Search is Dead, Long Live Gourmet Content

Three years ago I founded Dulcinea Media, a human-powered search alternative, on the premise that most Internet users cannot find the information they need online, and that search engines will never match the ability of a human curator to find the best content. At the time, this was a contrarian view. A  report by Pew Internet in January 2005 had shown that 75% of Internet users were satisfied with search engine results, and most pundits believed Google and Wikipedia would dominate the online information landscape forever.

In the past three years, the market has warmed to my view that uncurated, general search engines are a less-than-perfect tool for finding information online. More recent studies from the USC's Center for the Digital Future showed that user satisfaction with search results declined to 62% in 2006, and again to 51% in 2008. A survey commissioned by Yahoo! curiously issued a damning indictment of search engine results: 85% of initial search queries fail to return the information users were seeking, causing the users to try and try again, resulting in "search engine fatigue." A study from the UK exposed as a myth the notion of a “Google Generation” of young people with native ability to find information online.

Next, Nicholas Carr, who famously asked “Is Google Making Us Stupid?", and a number of other columnists bemoaned the reality that most users today read an Internet that is a mile-wide and an inch-deep. The center of their media world is a technology driven algorithm and “the wisdom of crowds” that simply uncover the same recycled headlines and updates from a slew of news sources. Google dipped a toe in the “human-powered” waters to tweak some of its search rankings, although it still accords technology most of the weight in the equation. Roger Schank, an artificial intelligence expert from Yale University, reversed his 30-year-old prediction that we would create machines as smart as humans in his lifetime. Schank came to recognize that "[h]umans are constantly learning. ... [e]very new experience changes what we know and how we see the world." Schank attributed this to "an unconscious indexing method that all people learn to do without quite realizing they are learning it."

Now a growing chorus of observers is acknowledging that search engines often fail the user. The impetus is the rise of “content farms,” which assure that search engines are only going to get worse at delivering quality results on the first search results page. Demand Media, Associated Content, Mahalo, Bukisa, eHow, HubPages and a voracious pack of others are paying freelance writers a modest per-article fee to create tens of thousands of articles each day. These companies excel at getting their content to rank high in search engines, regardless of quality.

The biggest challenge with these sites, paradoxically, is that some of the content is actually good, and most of it reads well. But, as Richard MacManus of ReadWriteWeb wrote after analyzing Demand Media’s content, it “lacks passion and often also lacks knowledge of the topic at hand.” Worse, the quality varies greatly from article to article – these brands stand for nothing other than “we create lots of content cheaply, SEO it superbly, and monetize it well.” Since no flashing neon lights warn “keyword-ridden trash” for weak submissions, the average Internet user does not know that the article was written in 20 minutes by someone with no expertise on the topic.

Many pundits agree that this spells trouble for search engines, but differ on further implications. Michael Arrington of TechCrunch sees the “end of hand crafted content.” In his view, the “fast food content” created by content farms has produced a “race to the bottom situation, where anyone who spends time and effort on their content is pushed out of business.” Fred Wilson of Union Square Ventures is more hopeful. He sees search engines being displaced by social media tools, where, “with the help of machines, our friends and trusted sources” will tell users what content matters.

What I see is that this avalanche of fast food content will lead to the “clarion call” that I predicted, in a July 2008 article  would cause a “flight to safety.” Internet users will turn to the "new portal" - trusted sources that consistently deliver “gourmet content” - important, relevant, reliable and comprehensive information, from a wide variety of resources across the Internet. The only sites that will succeed at this are those that rely on a human touch. For all the dismissive talk about Yahoo, its audience is massive - because it gathers content from around the Web, albeit of inconsistent quality across the verticals.  Companies that aggregate and organize content in an elegant way, and combine that with their own proprietary, high-quality content, will inherit the position at the top of the Web food chain that search engines abdicate.

So where is Dulcinea Media in all this ? Naturally, planning to be one of those trusted sources, or perhaps an engine that powers the new portals to which users flock. We are still executing  on the business plan we created three years ago. findingDulcinea now offers Web Guides to the best information alone about more than 700 broad topics, and we’ve created thousands of Beyond the Headlines and Features articles that provide a full context view of news stories. A Spanish-language sister site, encontrandoDulcinea, replicates much of this content in Spanish.

To make all this content easier to access, we’ve introduced SweetSearch, a custom search engine that harnesses Google’s technology and the 100,000+ hours of Web site evaluation that is the bedrock of findingDulcinea. SweetSearch returns results only from a “whitelist” of 35,000 sites that we’ve evaluated and approved. And we are tweaking SweetSearch to ensure that it remains the best search engine for students, and indeed, the only one they can use effectively. We've also introduced SweetSearch4Me, which is the only search engine that displays prominently high quality sites created for younger learners.

As our audience continues to grow steadily, we’ve also found that our “best customers” – those who visit our site the longest and consume the most pageviews, and thus are most likely to return – are college, high school and middle school students. And thus we focus our content on subjects that would be of interest to teachers, librarians, and students.  We presented at two national conferences in the fall - the AASL conference for school librarians, and the NCSS conference for social studies teachers - and we received an overwhelmingly positive response to our products.  We learned there is a critical need in the marketplace for free products that promotes effective, efficient, safe and responsible use of the Internet, and that ours fit the bill magnificently. And we've also had some very encouraging discussions with forward-thinking media companies about partnering with us to help make them trusted sources for content from around the Web.

We remain steadfast in our principles that (i) we will not use technology to aggregate links for Web Guides or articles; everything will pass through the prism of human judgment; and (ii) we will never compromise on the quality of our product, which will all continue to be created by our full-time editorial staff or subject-matter expert freelancers, and edited by a full-time editor.

To address scaling issues while holding form to these principles, we plan to introduce a program early next year in which we invite librarians and educators to submit content. Practitioners of these professions are trained to find, evaluate and recommend outstanding information resources, and library Web sites have always been the closest comparable to our Web Guides. We envision findingDulcinea and SweetSearch becoming a repository of the knowledge and insight of tens of thousands of librarians and teachers.

And we’ll stick with that vision, for as long as it takes to make it a reality.

From low expectations, great satisfaction

As I've written before, Marathon Fever is highly contagious. In 2004, two colleagues dragged me to the start of the NYC Marathon. Since then, I've run 5 more marathons, and have recruited a dozen first-timers. Days after my Boston Marathon last April, my very good friend Liz Perlman opened the door: "hey Mark - I've been thinking alot about your marathon shove. Your love of it is really contagious..." Soon I had persuaded her to register for the Philadelphia Marathon, by promising to run it with her. I had decided that I wanted to run only one more serious marathon for time, and then run several each year in a casual mode, with first-timers or people just having fun. So I started to look for an early marathon for my final serious run, to be followed by a casual stroll with Liz in Philly. While I ran 4 half marathons over the summer, my training was not very consistent. In September, a spate of minor injuries that I was too busy to properly address were followed by a nasty virus. In early October, 7 weeks before the race, I ran 5 miles of a half marathon with Liz before telling her to go ahead. I struggled mightily to breathe, and ran finished in 2:34, 46 minutes off my best. The next day, I was in a doctor's office contemplating a diagnosis of pneumonia and a chest x-ray pocked like a shooting range target. My "serious race" was out the window, and even a casual run Philly seemed highly unlikely. Fortunately, I was able to use an elliptical and stationery bike despite my condition, and soon both the pneumonia and x-ray cleared up. But things at findingDulcinea are more exciting than ever; we attended several national educator conferences down south and are pursuing some enormous opportunities, and I struggled to find time for the training I desperately needed, and didn't do a single run over 12 miles, and only about 5 runs of 6-12 miles. The few runs I did get in were ugly, since I didn't find time to stretch adequately beforehand. But two episodes told me I might be able to compete; one was the night before I went to the doctor for a definitive reading of the x-ray; I was so anxious to convince myself I was OK that I put myself through a vicious routine of calisthenics and fast miles, and held up very well. A week later, I ran 7.5 hard miles nearly as fast as I could a year ago, when I set a PR in Philly. Not wanting to leave my new convert in the lurch, I decided I may accompany her for 15 miles and then walk the rest of the way. I set an internal goal of 4:38, the average of my best & worst times so far, but would have been thrilled to break 5:00. A friend told me that when you are undertrained, hydration & nutrition become ever more important. Because I had fallen down on that front in Boston, I spent the past two days eating and drinking lots of good stuff and lining up the perfect breakfast for today. The race weather was utterly perfect. It was very crowded early on, and we ran the first three miles in 10:45, 10:30 and 10:15. I felt great, but worried that this would be as fast as we run. The next 5k was done in close to the same time. Then we picked up steam, running a 9:20 mile. Recalling that running miles 6-9 too fast in Boston came back to haunt me, I reined us in, and we soon were back to steady 10:00s. When we hit 10 miles, I realized I was in less distress than I had ever been at that point in the race, but the 3rd lesson I learned in Boston was not to celebrate too early, since there I very quickly went from thinking I'd break my 4-hour goal by 10 minutes to thinking I wouldn't finish. But our steady pace and stress-free feeling continued till Mile 19, when I was far beyond the distance of any training run and began to worry about running in a quality way for another 70 minutes. I focused only on the next half mile, and took that approach all the way in, never looking ahead more than 800 meters. Around 22 miles, I realized Liz was gaining strength, so I became determined not to hold her back any more than necessary. I realized that at our current pace, we'd just miss 4:30, and focused on that target, a half mile at a time. We finished in 4:28:48, and were utterly thrilled. I realized that the last 10k, which I thought I would largely walk, I had run faster than I ever had before, and our splits of 2:12 / 2:16 were the closest I had come to running even halfs. The realization that I can run decent marathons without adequate training was a big surprise. But fortunately I am focused on the realization that breaking 4:00, which I nearly did in Boston but which seemed remote as I struggled this fall, is very much within my reach if I have a healthy few months, and I plan to go after it hard early next year.

A Digital Defense of School Librarians

Books are giving way to e-books; newspapers to news aggregators; encyclopedias to Wikipedia. And that is why we need librarians, especially school librarians, now more than ever.

If you think that librarians are archaic, you’re most likely thinking of a 1950s bespectacled stereotype. Librarians are no longer – if they ever were – those hushing and shushing guardians of books. They are media specialists, guiding children and adults through every form of media, from books to databases, newspapers to blog posts, and even from YouTube to Twitter.

In the libraries of old – the ancient days of 1990, say – mastering the Dewey Decimal system was enough to get you started on your research. But there is no card catalogue 2.0. In order to use the Internet as a library, you need 21st-century research skills: the ability to pick out reliable sources from an overwhelming heap of misinformation, to find relevant material amid an infinite array of options, and to navigate the shifting ethics of creative commons and intellectual property rights. As good as your kid may be on Facebook, she is not born with a digital M.L.S. These skills are learned, not instinctive, and the only way for students to learn them is for someone else to guide and teach them.

This seems as elementary as the ABCs – but apparently nobody’s told the school districts. Librarians and teacher librarians, who are double-credentialed, are being driven out of their increasingly stripped-down libraries. Painful as it is, it’s no surprise to come across a tweet like Shankhead’s: “being an engaged school librarian, at least in my neck of the woods, now means being an ‘Austerity Specialist.’ Whatever it takes.”

I’m not sure what it takes to convince the school districts of common sense, but it definitely takes librarians to teach students how to evaluate credibility, create content of their own, and conduct research in their increasingly interconnected world. And it might take the SKILLs Act, a bill in the House of Representatives, to ensure we still have school librarians to train the next generation.

Students will create and consume online content, and even social media will find a way into their research. Should a student trust a blog as a source in a paper? If not, then how about a blog on The New York Times website? A blog run by an online magazine? Can they use collaborative technology, like wikis? Even teachers need help answering these questions. There are no official guidelines to using the Web, and even if there were, they would change by the minute.

As the information landscape becomes more and more complex, why would we abandon our professional guides to it?

A curated search engine for students

Most educators today struggle to harness the potential of technology and the Internet, which have only just begun to change the skills that students need to succeed.

David Ligon's post on "New Media Literacy," is a comprehensive look at the opportunities and challenges 21st century teachers face. It is exhilarating - and terrifying. As Joyce Valenza and Doug Johnson recently wrote in the School Library Journal blog, "School librarians, as we once knew them, may no longer be relevant. And, yet, this is undoubtedly the most exciting time in history to be a librarian."

I agree emphatically. In my school days, evaluating the accuracy of information was rarely part of the mix - if it was in the library, it was accurate. To fully develop critical 21st century learning skills, students will have to be taught how to find, evaluate, use and communicate information gleaned from a dozen credible sources, chosen out of the millions of resources available a fraction of a second after the click of a "search" button, and in countless other places. It is a transformative moment in the history of our education system.

So how do we teach students these critical skills ? When they conduct online research, most students heavily on major search engines, and review only the first few results. Students sense that some results are better than others, struggle to distinguish effectively, and worry about wasting time on the wrong one. So they put their faith in the search engine, hoping it has somehow placed the best results are the top.

This is not something that can be overcome in a 30-minute tutorial. Just as students develop writing skills by reading hundreds of great books, they learn to recognize a credible Web site by using hundreds of them, and learning from the experience. Teachers and librarians should refer students to the best Web resources, and let students devote most of their energies to distilling it and communicating the information they find.

With that in mind, our first product, findingDulcinea, directs users to credible and comprehensive information online about thousands of subjects. It is accessed through both an internal search function and by rooting through the category-driven tree-structure. After we spent 75,000 hours over 18 months creating it, we realized all this information we had amassed could be accessed in a manner that was more consistent with established user behavior.

And thus was born SweetSearch, a custom search engine that is derived from our work on findingDulcinea. SweetSearch only searches 35,000 Web sites that have been evaluated by our staff. It does not include results from the unreliable sites that often rank high in other search engines and waste students’ time. You know those hundreds of bookmarks you have to lists of great sites selected by teachers, librarians and subject-matter experts ? Think of SweetSearch as a giant, searchable repository of all of them.

This curated pool of Web sites allows students to choose, from a list of credible results, which ones are most relevant to their research, rather than spend much of their time deciding which sites are worth their consideration.

We launced SweetSearch in February of this year and have been improving it ever since. We've been gratified to see usership grow dramatically in recent weeks. Most encouraging has been the fact that every day, we see more school librarians linking to SweetSearch from the school library Web site. Some examples can be found here and here.

We invite the educational community to help us curate SweetSearch; please send any suggested additions or deletions to SweetSearch@DulcineaMedia.com

A Gallop for the Great Grete Waitz

I had an "I'll never wash this hand again" moment today. I didn't merely touch greatness, I high-fived it, wrapped my fingers around its fingers and stared into its eyes. As I crossed the starting line of "Grete's Great Gallop," a half marathon today in Central Park, enthusiastically greeting and high-fiving as many runners as possible. was the Great Grete Waitz. Yes, the word "Great" is misplaced in the title of the race. Wayne Gretzky is called "the Great One" because he annihilated the hockey record books. Similarly, the Great Waitz laid waste to world records in women's marathoning in the 1970s, and to the notion that women could not compete among the top men in the marathon.

In 1978, Fred Lebow, founder of the New York City Marathon, invited Waitz to participate as a "rabbit", pacing the top runners and dropping out. But fate had other plans. After setting the early pace, Waitz decided to complete the entire 26.2 miles. Despite not having done any training runs beyond 12 miles, she won the race and set a women’s world record of 2:32:30.

The following year, Sports Illustrated's cover story was about an epic duel in the men's New York Marathon; it noted that, shortly after the men finished "all of them were near the finish line, and Rodgers, at least, was cheering when Grete Waitz, the Norwegian schoolteacher who insists she has always been, is now and ever will be a track runner, not a marathoner, crossed the finish line in 2:27:33, almost five minutes faster than the world record she set last year in New York, and 11 minutes faster than any other woman in the race."

SI reported that Rodgers said admiringly, "She's pretty outrageous. I saw her come across the line, and, well, she's inspirational." SI also noted that, the prior year, nobody, including the announcer at the finish line, knew who she was, but that "this year she spent hours signing autographs wherever she went."

At the starting line today, they rattled off a list of Waitz' accomplishments, including her winning the New York City Marathon an unprecedented nine times and a litany of other records.

What can get lost in her long list of stunning accomplishments are two particular points worth noting:

Over the course of 7 years, Waitz lowered the women’s marathon world record by more than 9 minutes. In the 25 ensuing years, despite great advancements in training methods, nutrition, etc., today's women have managed to lower Waitz' record by only 3 minutes.

And most importantly of all was Waitz' influence on all the women who followed her. In 1979, she was the only woman to finish in the top 100 overall in the New York City Marathon, and the notion that a woman could do such a thing was staggering. In the 2008 New York City Marathon, 12 women finished in the top 100 finishers.

In the spring of 2005, Waitz began battling cancer. Throughout her treatment and recovery, she has been a tireless promoter for many charities, particularly around children's health, and of course a great ambassador for the sport of running. It was because of the latter role that I will never wash my right hand again.

Teaching Kids Web Research Skills

Dulcinea Media has a lot of ambitions for a little company. And we madly pursue them all at once. And thus we take great pride that all of our initiatives have gelled together in a single endeavor: findingEducation's On This Day Challenge.

Our mission is driven by the fact that most people cannot find online, evaluate, and put to use the critical information they need in their daily lives. This is true even of students, who facilely use the Web for social purposes, but can’t effectively research online. They rely heavily on major search engines, and review only the first few results. They can’t discriminate between a credible resource and a suspect one, can’t locate primary sources and, above all, don’t know how to digest what they’ve learned and communicate it to others.

Our company mission is to help educators change it.

In furtherance of our mission, our first product was findingDulcinea, a content Web site that helps users find credible and comprehensive information online about thousands of subjects. Its most popular feature is "On This Day," which each day details an important event in history, including the preamble and the denouement. In November 2008, I explained the genesis of our On This Day.

The second site we launched was EncontrandoDulcinea, a Spanish language sister site that offers much of the content on findingDulcinea, translated into Spanish. Its traffic has begun to boom in large part due to the popularity of the On This Day feature in Spanish.

The third site we launched was SweetSearch, a custom search engine that is derived from our tens of thousands of hours of work on findingDulcinea. SweetSearch only searches 35,000 Web sites that have been evaluated by our staff. It eliminates results from the junky sites that rank high in other search engines and waste students’ time.

Earlier this month, we introduced findingEducation, a free tool that serves as a meeting place for educators to share insight and outstanding links, assignments and lesson plans with each other and their students. We have seeded the links library with hundreds of links contributed by the teachers and librarians who created findingDulcinea's Web Guides on education topics, and SweetSearch is available to help find other high quality links. Our hope is that teachers from all over the world come to view findingEducation as their community site, one whose continued development they largely direct.

And the endeavor that arises from all of our years of hard work on each of this products is findingEducation's On This Day Challenge. Through this Challenge, students, working individually or in teams, will learn to find and evaluate quality Web sites for online history research, learn how to organize and write a research article about historical events, and gain an appreciation of how historical events have shaped the world.

Drawing on the plethora of material on findingDulcinea, we provide extensive guidance to teachers and students on how to find and evaluate Web resources and organize them into an article about an important historical event. SweetSearch can be used to search all of these resources at once. All articles in the Challenge will be published to the teacher's public page on findingEducation, and a broad range of impressive entries will be highlighted in our newsletter and on findingDulcinea.

The number of early sign-ups to the Challenge has been gratifying, and we eagerly look forward to sharing reports about its progress throughout the school year. We believe that, for students who participate in the Challenge will learn critical life skills of finding online, evaluating, organizing, using and communicating information.

May their love give us love

Last year, I wrote about how, when I think of 9/11, I choose to remember the love.

The love shown by thousands of rescue workers and ordinary citizens, in NYC, Washington, D.C. and on Flight 93, when they knowingly put their lives at grave risk so that others may live. The love expressed by the many victims who lived for a desperate hour or more after their fate was sealed, and used the last precious minutes of their lives to call loved ones; all that mattered at the end of their lives was the love they created along the way. The loved expressed by victims’ family members, many of whom spent weeks desperately hoping that their loved ones had somehow miraculously been spared. And the love that I witnessed when I emerged from my office building in midtown Manhattan in early afternoon, and indeed throughout America and much of the rest of the world, for months afterward.

And so tomorrow, I will once again think about the manner in which so many people responded to the certain end of their lives by calling their loved ones. Though I hopefully won’t be imperiled myself, I will call or email many of my loved ones that I have been meaning to get back in touch with, and with that small gesture let them know how I feel about them.

I will also recall my Aunt Eileen’s comment to my post about how the heroes of 9/11 and the ensuing months (and indeed years) saved us “from having that day be remembered as one of being simply victims, totally demoralized…. and turned the story into one of great pride in our values as a country and in the bravery and devotion to duty that our people can show.” And I will beam with pride about our values as a country, and how brave and devoted to duty our people can be.

And recalling the timeless words of firefighter Mike Moran, who in October 2001 declared that his brother and his many close friends and crewmates who perished “are not gone, because they are not forgotten,” I will recall all of those ordinary people who lovingly did remarkable things that day so that others may live.

May their love give us love.