Google tweaked search 450 times in 2007

Google is typically tight-lipped about it the inner workings of its search business, but there are a few nuggets worth looking at in a Popular Mechanics interview with Udi Manber, the Google vice president who oversees search quality. Among them: Google rejiggered its search algorithm 450 times last year.

The job of the algorithm is to best match Web pages with people’s search terms. One tweak the company tried last week was increasing the “diversity” of search results so the listed Web pages would cover a broader scope in an attempt to compensate for the ambiguities of search terms, he said.

And while some might see the industry of search engine optimization (SEO), which strives to get Web sites higher placement on search sites, as gaming the system, Manber said that at least a basic amount would make his life easier.

“I wish people would put more effort into thinking about how other people will find them and putting the right keywords onto their pages,” he said.

He also said Google doesn’t adjust search results by hand.

“If we find, for a particular query, that result No. 4 should be result No. 1, we do not have the capability to manually change it,” he said. “We have to find what weakness in the algorithm caused that result and find a general solution to that, evaluate whether a general solution really works and if it’s better, and then launch a general solution.”

For those interested in the subject, I also recommend the New York Times interview with Manber from last year and another from Eric Enge at SEO firm Stone Temple Consulting. (I can’t help but note that the latter piece shows up higher in Google search results.)

---

June 3, 2007

Google Keeps Tweaking Its Search Engine

By SAUL HANSELL

Mountain View, Calif.

THESE days, Google seems to be doing everything, everywhere. It takes pictures of your house from outer space, copies rare Sanskrit books in India, charms its way onto Madison Avenue, picks fights with Hollywood and tries to undercut Microsoft’s software dominance.

But at its core, Google remains a search engine. And its search pages, blue hyperlinks set against a bland, white background, have made it the most visited, most profitable and arguably the most powerful company on the Internet. Google is the homework helper, navigator and yellow pages for half a billion users, able to find the most improbable needles in the world’s largest haystack of information in just the blink of an eye.

Yet however easy it is to wax poetic about the modern-day miracle of Google, the site is also among the world’s biggest teases. Millions of times a day, users click away from Google, disappointed that they couldn’t find the hotel, the recipe or the background of that hot guy. Google often finds what users want, but it doesn’t always.

That’s why Amit Singhal and hundreds of other Google engineers are constantly tweaking the company’s search engine in an elusive quest to close the gap between often and always.

Mr. Singhal is the master of what Google calls its “ranking algorithm” — the formulas that decide which Web pages best answer each user’s question. It is a crucial part of Google’s inner sanctum, a department called “search quality” that the company treats like a state secret. Google rarely allows outsiders to visit the unit, and it has been cautious about allowing Mr. Singhal to speak with the news media about the magical, mathematical brew inside the millions of black boxes that power its search engine.

Google values Mr. Singhal and his team so highly for the most basic of competitive reasons. It believes that its ability to decrease the number of times it leaves searchers disappointed is crucial to fending off ever fiercer attacks from the likes of Yahoo and Microsoft and preserving the tidy advertising gold mine that search represents.

“The fundamental value created by Google is the ranking,” says John Battelle, the chief executive of Federated Media, a blog ad network, and author of “The Search,” a book about Google.

Online stores, he notes, find that a quarter to a half of their visitors, and most of their new customers, come from search engines. And media sites are discovering that many people are ignoring their home pages — where ad rates are typically highest — and using Google to jump to the specific pages they want.

“Google has become the lifeblood of the Internet,” Mr. Battelle says. “You have to be in it.”

Users, of course, don’t see the science and the artistry that makes Google’s black boxes hum, but the search-quality team makes about a half-dozen major and minor changes a week to the vast nest of mathematical formulas that power the search engine.

These formulas have grown better at reading the minds of users to interpret a very short query. Are the users looking for a job, a purchase or a fact? The formulas can tell that people who type “apples” are likely to be thinking about fruit, while those who type “Apple” are mulling computers or iPods. They can even compensate for vaguely worded queries or outright mistakes.

“Search over the last few years has moved from ‘Give me what I typed’ to ‘Give me what I want,’ ” says Mr. Singhal, a 39-year-old native of India who joined Google in 2000 and is now a Google Fellow, the designation the company reserves for its elite engineers.

Google recently allowed a reporter from The New York Times to spend a day with Mr. Singhal and others in the search-quality team, observing some internal meetings and talking to several top engineers. There were many questions that Google wouldn’t answer. But the engineers still explained more than they ever have before in the news media about how their search system works.

As Google constantly fine-tunes its search engine, one challenge it faces is sheer scale. It is now the most popular Web site in the world, offering its services in 112 languages, indexing tens of billons of Web pages and handling hundreds of millions of queries a day.

Even more daunting, many of those pages are shams created by hucksters trying to lure Web surfers to their sites filled with ads, pornography or financial scams. At the same time, users have come to expect that Google can sift through all that data and find what they are seeking, with just a few words as clues.

“Expectations are higher now,” said Udi Manber, who oversees Google’s entire search-quality group. “When search first started, if you searched for something and you found it, it was a miracle. Now, if you don’t get exactly what you want in the first three results, something is wrong.”

Google’s approach to search reflects its unconventional management practices. It has hundreds of engineers, including leading experts in search lured from academia, loosely organized and working on projects that interest them. But when it comes to the search engine — which has many thousands of interlocking equations — it has to double-check the engineers’ independent work with objective, quantitative rigor to ensure that new formulas don’t do more harm than good.

As always, tweaking and quality control involve a balancing act. “You make a change, and it affects some queries positively and others negatively,” Mr. Manber says. “You can’t only launch things that are 100 percent positive.”

THE epicenter of Google’s frantic quest for perfect links is Building 43 in the heart of the company’s headquarters here, known as the Googleplex. In a nod to the space-travel fascination of Larry Page, the Google co-founder, a full-scale replica of SpaceShipOne, the first privately financed spacecraft, dominates the building’s lobby. The spaceship is also a tangible reminder that despite its pedestrian uses — finding the dry cleaner’s address or checking out a prospective boyfriend — what Google does is akin to rocket science.

At the top of a bright chartreuse staircase in Building 43 is the office that Mr. Singhal shares with three other top engineers. It is littered with plastic light sabers, foam swords and Nerf guns. A big white board near Mr. Singhal’s desk is scrawled with graphs, queries and bits of multicolored mathematical algorithms. Complaints from users about searches gone awry are also scrawled on the board.

Any of Google’s 10,000 employees can use its “Buganizer” system to report a search problem, and about 100 times a day they do — listing Mr. Singhal as the person responsible to squash them.

“Someone brings a query that is broken to Amit, and he treasures it and cherishes it and tries to figure out how to fix the algorithm,” says Matt Cutts, one of Mr. Singhal’s officemates and the head of Google’s efforts to fight Web spam, the term for advertising-filled pages that somehow keep maneuvering to the top of search listings.

Some complaints involve simple flaws that need to be fixed right away. Recently, a search for “French Revolution” returned too many sites about the recent French presidential election campaign — in which candidates opined on various policy revolutions — rather than the ouster of King Louis XVI. A search-engine tweak gave more weight to pages with phrases like “French Revolution” rather than pages that simply had both words.

At other times, complaints highlight more complex problems. In 2005, Bill Brougher, a Google product manager, complained that typing the phrase “teak patio Palo Alto” didn’t return a local store called the Teak Patio.

So Mr. Singhal fired up one of Google’s prized and closely guarded internal programs, called Debug, which shows how its computers evaluate each query and each Web page. He discovered that Theteakpatio.com did not show up because Google’s formulas were not giving enough importance to links from other sites about Palo Alto.

It was also a clue to a bigger problem. Finding local businesses is important to users, but Google often has to rely on only a handful of sites for clues about which businesses are best. Within two months of Mr. Brougher’s complaint, Mr. Singhal’s group had written a new mathematical formula to handle queries for hometown shops.

But Mr. Singhal often doesn’t rush to fix everything he hears about, because each change can affect the rankings of many sites. “You can’t just react on the first complaint,” he says. “You let things simmer.”

So he monitors complaints on his white board, prioritizing them if they keep coming back. For much of the second half of last year, one of the recurring items was “freshness.”

Freshness, which describes how many recently created or changed pages are included in a search result, is at the center of a constant debate in search: Is it better to provide new information or to display pages that have stood the test of time and are more likely to be of higher quality? Until now, Google has preferred pages old enough to attract others to link to them.

But last year, Mr. Singhal started to worry that Google’s balance was off. When the company introduced its new stock quotation service, a search for “Google Finance” couldn’t find it. After monitoring similar problems, he assembled a team of three engineers to figure out what to do about them.

Earlier this spring, he brought his squad’s findings to Mr. Manber’s weekly gathering of top search-quality engineers who review major projects. At the meeting, a dozen people sat around a large table, another dozen sprawled on red couches, and two more beamed in from New York via video conference, their images projected on a large screen. Most were men, and many were tapping away on laptops. One of the New Yorkers munched on cake.

Mr. Singhal introduced the freshness problem, explaining that simply changing formulas to display more new pages results in lower-quality searches much of the time. He then unveiled his team’s solution: a mathematical model that tries to determine when users want new information and when they don’t. (And yes, like all Google initiatives, it had a name: QDF, for “query deserves freshness.”)

Mr. Manber’s group questioned QDF’s formula and how it could be deployed. At the end of the meeting, Mr. Singhal said he expected to begin testing it on Google users in one of the company’s data centers within two weeks. An engineer wondered whether that was too ambitious.

“What do you take us for, slackers?” Mr. Singhal responded with a rebellious smile.

THE QDF solution revolves around determining whether a topic is “hot.” If news sites or blog posts are actively writing about a topic, the model figures that it is one for which users are more likely to want current information. The model also examines Google’s own stream of billions of search queries, which Mr. Singhal believes is an even better monitor of global enthusiasm about a particular subject.

As an example, he points out what happens when cities suffer power failures. “When there is a blackout in New York, the first articles appear in 15 minutes; we get queries in two seconds,” he says.

Mr. Singhal says he tested QDF for a simple application: deciding whether to include a few news headlines among regular results when people do searches for topics with high QDF scores. Although Google already has a different system for including headlines on some search pages, QDF offered more sophisticated results, putting the headlines at the top of the page for some queries, and putting them in the middle or at the bottom for others.

 

GOOGLE’S breakneck pace contrasts with the more leisurely style of the universities and corporate research labs from which many of its leaders hail. Google recruited Mr. Singhal from AT&T Labs. Mr. Manber, a native of Israel, was an early examiner of Internet searches while teaching computer science at the University of Arizona. He jumped into the corporate fray early, first as Yahoo’s chief scientist and then running an Amazon.com search unit.

Google lured Mr. Manber from Amazon last year. When he arrived and began to look inside the company’s black boxes, he says, he was surprised that Google’s methods were so far ahead of those of academic researchers and corporate rivals.

“I spent the first three months saying, ‘I have an idea,’ ” he recalls. “And they’d say, ‘We’ve thought of that and it’s already in there,’ or ‘It doesn’t work.’ ”

The reticent Mr. Manber (he declines to give his age), would discuss his search-quality group only in the vaguest of terms. It operates in small teams of engineers. Some, like Mr. Singhal’s, focus on systems that process queries after users type them in. Others work on features that improve the display of results, like extracting snippets — the short, descriptive text that gives users a hint about a site’s content.

Other members of Mr. Manber’s team work on what happens before users can even start a search: maintaining a giant index of all the world’s Web pages. Google has hundreds of thousands of customized computers scouring the Web to serve that purpose. In its early years, Google built a new index every six to eight weeks. Now it rechecks many pages every few days.

And Google does more than simply build an outsized, digital table of contents for the Web. Instead, it actually makes a copy of the entire Internet — every word on every page — that it stores in each of its huge customized data centers so it can comb through the information faster. Google recently developed a new system that can hold far more data and search through it far faster than the company could before.

As Google compiles its index, it calculates a number it calls PageRank for each page it finds. This was the key invention of Google’s founders, Mr. Page and Sergey Brin. PageRank tallies how many times other sites link to a given page. Sites that are more popular, especially with sites that have high PageRanks themselves, are considered likely to be of higher quality.

Mr. Singhal has developed a far more elaborate system for ranking pages, which involves more than 200 types of information, or what Google calls “signals.” PageRank is but one signal. Some signals are on Web pages — like words, links, images and so on. Some are drawn from the history of how pages have changed over time. Some signals are data patterns uncovered in the trillions of searches that Google has handled over the years.

“The data we have is pushing the state of the art,” Mr. Singhal says. “We see all the links going to a page, how the content is changing on the page over time.”

Increasingly, Google is using signals that come from its history of what individual users have searched for in the past, in order to offer results that reflect each person’s interests. For example, a search for “dolphins” will return different results for a user who is a Miami football fan than for a user who is a marine biologist. This works only for users who sign into one of Google’s services, like Gmail.

(Google says it goes out of its way to prevent access to its growing store of individual user preferences and patterns. But the vast breadth and detail of such records is prompting lust among the nosey and fears among privacy advocates.)

Once Google corrals its myriad signals, it feeds them into formulas it calls classifiers that try to infer useful information about the type of search, in order to send the user to the most helpful pages. Classifiers can tell, for example, whether someone is searching for a product to buy, or for information about a place, a company or a person. Google recently developed a new classifier to identify names of people who aren’t famous. Another identifies brand names.

These signals and classifiers calculate several key measures of a page’s relevance, including one it calls “topicality” — a measure of how the topic of a page relates to the broad category of the user’s query. A page about President Bush’s speech about Darfur last week at the White House, for example, would rank high in topicality for “Darfur,” less so for “George Bush” and even less for “White House.” Google combines all these measures into a final relevancy score.

The sites with the 10 highest scores win the coveted spots on the first search page, unless a final check shows that there is not enough “diversity” in the results. “If you have a lot of different perspectives on one page, often that is more helpful than if the page is dominated by one perspective,” Mr. Cutts says. “If someone types a product, for example, maybe you want a blog review of it, a manufacturer’s page, a place to buy it or a comparison shopping site.”

If this wasn’t excruciating enough, Google’s engineers must compensate for users who are not only fickle, but are also vague about what they want; often, they type in ambiguous phrases or misspelled words.

Long ago, Google figured out that users who type “Brittany Speers,” for example, are really searching for “Britney Spears.” To tackle such a problem, it built a system that understands variations of words. So elegant and powerful is that model that it can look for pages when only an abbreviation or synonym is typed in.

Mr. Singhal boasts that the query “Brenda Lee bio” returns the official home page of the singer, even though the home page itself uses the term “biography” — not “bio.”

But words that seem related sometimes are not related. “We know ‘bio’ is the same as ‘biography,’ ” Mr. Singhal says. “My grandmother says: ‘Oh, come on. Isn’t that obvious?’ It’s hard to explain to her that bio means the same as biography, but ‘apples’ doesn’t mean the same as ‘Apple.’ ”

In the end, it’s hard to gauge exactly how advanced Google’s techniques are, because so much of what it and its search rivals do is veiled in secrecy. In a look at the results, the differences between the leading search engines are subtle, although Danny Sullivan, a veteran search specialist and blogger who runs Searchengineland.com, says Google continues to outpace its competitors.

Yahoo is now developing special search formulas for specific areas of knowledge, like health. Microsoft has bet on using a mathematical technique to rank pages known as neural networks that try to mimic the way human brains learn information.

Google’s use of signals and classifiers, by contrast, is more rooted in current academic literature, in part because its leaders come from academia and research labs. Still, Google has been able to refine and advance those ideas by using computer and programming resources that no university can afford.

“People still think that Google is the gold standard of search,” Mr. Battelle says. “Their secret sauce is how these guys are doing it all in aggregate. There are 1,000 little tunings they do.”

 

This is good news for all considering or deploying SEO, as the value of natural placement can be bench marked to Google AdWords.   While Google’s revenue is in part a fuction of the premium Advertisers place on the Adwords (since it is a bit system) and therefore the more people who buy ads, the more potential revenue google stands to make, the realized revenue is nonetheless due entirely to traffic. 

This is good news for all who deploy SEO.   Traffic to Adwords is trending in the right direction – which means that traffic to natural results is too.   Search analysts estimate traffic into the “natural results” to be as high as 70-80%, whereas PPC/Adwords receives between 20-30% (estimated); so if Google is hitting their revenue goals and doing so while only capitalizing on only 20-30% of the value they provide to the market, this supports what we know to be true: First page google exposure is valuable!  

Here’s the link to the NYT article about Google’s after hours performance announcement last night  

http://www.nytimes.com/2008/04/18/technology/18google.html?ex=1366257600&en=478f3a6615213fc5&ei=5124&partner=permalink&exprod=permalink

Most of the time wall street’s valuation process can be used as an indicator of trends, and can often be used as a litmus test to determine conditions in various sectors of the economy, perhaps not to measure the real time “value” of this or that but to guage how investors view the future of certain industries.

For the past 5 years we have witnessed the market reward Google, which comes as no surprise to anyone who has witnessed the revenue shift from print to digital.  Rather, the surprise has been the lack of penalization (in market terms) in the print advertising industry, in particular the Yellow Page industry the Google’s direct competitor.  Over the past year Google’s stock has been as high as $700 and currently hovers around $450 per share, up from around $100 since the IPO.   Meanwhile, RHD (R.H. Donnelley the leading US Yellow Page Company http://www.rhd.com/) has been as high as $80, and up from $50 during Google’s IPO. 

Initally, it looked like Wall Street was betting on RHD’s ability to shift from a player in the print market to a player in the digital market.

Well — the shift never happened.  Over the past several weeks RHD has hit an all time low of $4.27, and is currently hovering around $5.25.

This comes as little surprise to SEO/SEM companies like The Search Engine Guys, LLC.   Yet another example of an “ocean liner” unable to tack in time to survive.

The fact that ”no one really knows” precisely how much market share Google actually has… is a bit like saying no one really knows exactly what the weather will be like in Los Cabos in January.   True true.   It might be in the mid 50’s, around 65, or it might be in the high 80s… either way I”ll bet you my winter boots that it will be much warmer in Los Cabos than in Chicago! 

The point is - Google is WAY out in front. 

We’ve all heard estimates as high as 85% but I’ve never seen data to back it up, though in some industries I’m sure there is evidence.   A cursory study of five of our clients (each with roughly equivalent natural placements on Google, Yahoo and MSN) revealed a range of Google market share between 51-66%.   Though this is a poor measure of total market share it is a relevant statistic and reveals the imporatance of optimizing for Yahoo, MSN / Live, Ask and Aol.

Hitwise (the most respected market research firm on the topic) recently published a press-release suggesting around 65% market share with Yahoo trailing around 21% (see release below or go to http://www.hitwise.com/press-center/hitwiseHS2004/searchengines200711us.php)

NEW YORK, NY - December 11, 2007 - Hitwise, the leading online competitive intelligence service, today announced that Google accounted for 65.10 percent of all U.S. searches in the four weeks ending December 1, 2007. Yahoo! Search, MSN Search and Ask.com each received 21.21, 7.09 and 4.63 percent respectively. The remaining 46 search engines in the Hitwise Search Engine Analysis Tool accounted for 1.96 percent of U.S. searches. 

We are commonly asked about Google-Local and whether a Google-Local listing has any impact on page rank in the natural results on Google.  As far as we know, there is no positive SEO value (in terms of natural results) for listing your business on Google-Local.  However, we do recommend that you get your business listed.  Why not - it’s free and well… google has pretty darn good market share! 

To list your business, simply go to Google’s local business center (follow the link below) and follow the instructions.     

  https://www.google.com/accounts/ServiceLogin?continue=http%3A%2F%2Fwww.google.com%2Flocal%2Fadd%2FbusinessCenter%3Fhl%3Den_US%26gl%3DUS&service=lbc&hl=en-US&gl=US

At first glance there may seem to be an infinite number options to market your business on the Internet.  And while there are countless options there are only a few distinct strategies… four to be exact.

I will briefly describe each, in order of my personal preference which I believe is perfectly aligned with the effectiveness of each strategy.

1. SEO ( Search Engine Optimization) or as I sometimes like to call it “The Long Tail Strategy”

SEO is built upon the Long Tail Strategy.  You could say that SEO is the vehicle that rides on top of the “long tail” boulevard.   The Long Tail theory is now part of the daily vernacular of the web.  The theory was developed and popularized by Chris Anderson, Editor of Wired Magazine who coined the phrases as a way of describing how infinite demand  (picture a very very long never ending tail) can be met in the digital age with media suppliers such as Amazon, Itunes and Rhapsody, etc., where traditional bricks and mortar suppliers of media, books, periodicals, music where limited by shelf space and forced to cater inventory only main stream authors and musicians that appeal to the masses (or fat belly of the bell curve also known as “pop culture”).  Simple enough.

As applied to marketing, the long tail theory explains how businesses / customers can connect directly without relying on traditional “interstate like intermediaries” such as catalogs, directories or traditional stock-everything-you-sell retailers.  Fifteen years ago, an organic soap maker in Montana would have to develop rely wholly on retailers to sell soap, except for the one-off sale here and there from someone who happened by his business.  What I am calling the “the long tail strategy” allows this same organic soap maker in Montana to implement SEO and respond to people who are searching for “organic soap.”

Search Engine Optimization empowers small and mid-market business owners; allowing for previously impossible levels of efficiency and precision in attracting and communicating to prospective customers.

Another aspect that is deeply intriguing about Chris Anderson’s analysis of the web is how infinite taste is, when unfettered by the limitations of traditional producers / retailers of goods, services and media.   Something all budding and seasoned entrepreneurs should be reminded of… in the digital age, the challenge is finding a niche and delivering value to a specialized audience / customer base.  Success for business men, film makers, artists, authors, musicians no longer requires that one scale the corporate ladder (or break into Hollywood’s very narrow line of vision) or crack the code of mass culture.

Embedded in this theory, of course, is the assumption that people use search engines which allow them and empower them to “go off road” in search for precisely what they are looking for.  If traditional directories were elevators, or train cars (vertical / horizontal search) search engines are magic carpets… allowing for infinitely precise search across many dimensions.   Consider that people may use search engines to find directories (indicating that directories are no longer a main conduit for behavior but no one (at least it is absurd to imagine) would use a directory to find a search engine.

For more on The Long Tail Theory Visit Wikipedia: http://en.wikipedia.org/wiki/The_Long_Tail

For more on SEO visit Wikipedia: http://en.wikipedia.org/wiki/Search_engine_optimization

#2 Lead Generation

In short, lead generation is a form of web marketing (and traditional marketing) that qualifies a prospective customer for a supplier/vendor while simultaneously providing passive value to the prospect, typically in the language of “access to a qualified network of vendors for free.” There are countless niche examples across many industries. Legal Match comes to mind for the legal industry and Service Magic for general contractors, plumbers, painters, roofers, etc.  I have seen lead gen work, however there are some inherent drawbacks to the service. For starters, the prospective client typically must fill out an in depth questionnaire, which limits reach as not everyone is willing exert the time, or be willing to disclose information; though you could argue that the step eliminates casual shoppers and refines a set of prospects who are serious about resolving a particular need.   Of greater concern, is the fact that a lead that has been generated is never, or I should say… rarely… exclusive, or passed on to the most qualified vendor. Rather a lead is sent forth to many vendors who are then given the opportunity to contact the prospective customer. You could argue that this diminishes the value of the lead (from the view of the business / supplier); and could  be potentially misleading if a prospective buyer assumes that the most qualified (based on presumed customization of their need to a particular skill).  That said, lead generation services can be very effective, I merely believe they are a distant second to the effectiveness of implementing search engine optimization as a web marketing strategy.  S

For more on Lead Generation visit Wikipedia: http://en.wikipedia.org/wiki/Lead_generation

#3 Directories

Directories are familiar.  We all grew up with print directories, yellow pages, etc.  On-line, this strategy includes a wider range of options: open source directories such as Craigslist, free or paid on-line classifieds, aggregates such as Ebay, Amazon, etc., and pay-for-access directories such as Angie’s List. The value is directly related to two factors (and this rings true for all Internet Marketing) usage (and usability) and content. If a directory has only a few listings — will anyone use it?  No.  If no one uses the directory, is a listing valuable?  There can be tremendous value in listing your business in a directory if and only if the directory is reputable, user friendly and receives volume usage (also known as eye balls, searches, hits, traffic, impressions, etc.).  However, there is danger in being listed as one of many and failing to distinguish yourself in large directories, getting “lost in the shuffle.”
For many reasons, SEO is superior, that is assuming that people are in fact utilizing search engines like Google in greater numbers for the particular good or service you provide, than the leading directory that may be the industry speciality.  But even then… numbers can lie.  Would you rather be 1 of 523 vendors listed in a directory that gets 10,000 searches a month, or ranked on the first page of Google in the organic results (1- for 1,000 searches a month?

#4 Paid Placement (Ad-words, Banner Ads, Pop-ups, etc.)

Paid Placement is far and away the most abused form of Internet marketing, that said… it can be effective.  An ad-words campaign on Google, is relatively effective… but not as effective as a natural SEO campaign.  That could change, if Google were to phase out the real estate devoted to natural results (which I don’t anticipate happening given the fact that their business model relies on natural results, even if their revenue model does not).  There are many factors to consider when evaluating paid placements — is the ad intended to reinforce a brand?  Generate initial interest in a new product or concept? Convert casual shoppers by offering a sale?

More importantly, what website, and where on the site will the ad be purchased?  ESPN is a better site to purchase a pop-up ad, promoting a male-focused product… but is the pop-up ad on the first page, will it be viewed every click,  every 20th click, or will it pop up only on a back page that caters to NBA fans, in the middle of summer (hint NBA fanatics/weirdos and much less traffic than the home page).  More-over, the local FOX News Channels web sports page in Green Bay Wisconsin may be a better buy than ESPN for a Wisconsin Medical Spa offering hair replacement treatments catering to men… for many reasons, the fact that Packer’s fans bald prematurely may just be one of the many reasons.

The point here is to measure and determine good better best, and do so by using a cost per 1000 measurement.  What does it cost to communicate to 1000 individuals, over the contract period for the ad?  Likewise, use common sense… does your target audience use the site? Can you pin-point whether they are looking for you, and the ad makes you more visible (such as with Ad Words) or are you trying to generate / create a need? The value of paid placement is based on cost vs. the benefit of the resulting exposure how often the ad is seen, and should be, like any form, measured obsessively.

Summary / Conclusion:

Regardless of what strategy you implement ask questions, demand data, make comparisons, try to determine the behavior of your prospective audience, ask your friends and family how they would search for you.

If possible demand proof, demand exclusivity, ask for testimonials.

SEO is easy to measure and track; furthermore it is intuitive to determine who your audience is, because they are looking for you.  Lead Generation has it’s advantages and disadvantages, remember that you and your competition will be receiving the same leads.  Directories are the oldest form of marketing known to the modern world, but have will they survive the digital age, so far few have successively migrated to the web.  Directories that aggregate information and have rich local content are phenomenal for certain kinds of transactions, poor for others.  Would you look for a Lawyer on Ebay?  A Hotel in Paris on Craig’s list?  Probably not… you might look for a house-swap on Craig’s list and a Lawyer on Lawyers.com.  But is this superior to optimizing a site to show up when someone searches “Patent Lawyer Boston” or “Hotel Paris” on Google?  I think not.  Lastly, be aware of what your buying with Paid Placement — are you buying it for vanity, branding, or is there quantifiable proof that the premier placement makes you the obvious choice ahead of your competition?

It all things, remember that everything on-line can be measured and quantified.

-

The YP industry exists beneath most peoples radar, as it’s not nearly as sexy as TV or Magazine advertising. Nevertheless, is estimated to control more than 25 Billion in local advertising revenue: a chuck of change that digital search companies like Google Yahoo and Live as well as SEO companies like The Search Engine Guys are jockeying for as people continue to migrate away from print media to digital search.While large YP companies such as RH Donnelley (currently the largest Yellow Page company in the US) www.rhd.com still trade above $70 a share, their stock performance may not be the best indicator of their future. I would argue that it has far more to do with consolidations, acquisitions and internal efficiencies, and insanely high margins (good management in short) than projected growth.

Simply look at your local book — and notice that it has shrunk in size. Though in many cases, the YP industry has been able to maintain the appearance of “lots of content” by including filler content often local art or filler ads for other YP services on the individual pages, not to mention the extraneous shopping guides, golf guides, maps to museums etc. More noticeable many YP books now cover a much larger geographic scope than they did 10 years ago. Often a book will cover a whole county now, which was once covered by 3 or 4 city directories, etc. The pitch to the small business owner might be — more turf for the same buck.

But come-on is that real growth? Not in my book.

Bill Gates - founder and Chairman of Microsoft made a hefty statement in an interview this past May (simply search “Bill Gates Yellow Pages” and you can find the article in 5 seconds) when he claimed that, “Yellow Page usage amongst people below 50, will drop to near zero over the next five years.”

And while Bill Gates was discussing the future of live.com (formally MSN) and his statement could be taken as evangelism for his cause, it is very safe and reasonable to see that the future of the yellow page industry is in decline and not positioned for growth.

A few factors:

1. The Yellow Page Industry thrives on two things: usage and content.

2. Usage in decline — without question, I saw it with my own eyes over from 2002-2005. As broadband access increases, access via smart phones, more people rely on search engines for information than on the Yellow Pages.

3. Content in decline - As fewer businesses choose to ante up for display ads, content declines, as content declines so does usage. A classic chicken-egg relationship, the impulse to pick up the Yellow Pages gives way to reluctance the minute you can’t quickly find what your looking for.

4. Availability - Next time your in your own (or neighbor’s) kitchen or home office, do a quick scan for a copy of the yellow pages. Is it easier to go on the web?

5. Markets - Most trends begin in large metropolitan cities then move to smaller markets. We saw the YP industry begin to hemorrhage in places like Chicago and New York long before the suburban markets, and smaller tier 2 or tier 3 metropolitan cities like Cleveland or Milwaukee. In some markets, the YP industry will remain strong for years to come. In others, it’s long past the hemorrhage stage.

6. Age Demographics - Bill Gates honed in on age as the predominant factor. Have you met anyone under 20 who has ever used the Yellow Pages? That said, the meaty belly of the population bell curve is still the group between 30-75 in America. The boomers and their kids- where the money in America is. This group clearly uses Google. The trend in this group is not back toward the Yellow Pages, but toward Search on Google, Yahoo and Live.

7. Obscurity of the search - Still to this day, if I am looking for something very obscure (say a hardware store in a certain part of town, or a shoe repair shop) I will first search on google, then if I can’t find what I’m looking for, I will pick open the Yellow Pages. Again, the growth trend will not be back to the YP, but rather toward Google & Co. That said, there are business types that have not shifted to digital search and until they do, there will continue to be some value in the content and the usage in the YP industry.

8. Additional factors -With digital search you can receive loads more information than you can receive in a static print format. You can quickly determine which store is close to home, which plumber is nearby, etc., you can more readily educate yourself and evaluate dynamic information about the product or service you are considering. Think of a ticket broker for a sports game — in the YP era, ticket brokers would jockey for placement in the Yellow pages hoping for a phone call. Now brokers can list tickets online, display prices and manage inventory electronically — drastically reducing their overhead while creating a more efficient internal market, where they can measure and better predict how to position their future ticket purchases. The end result — it is more efficient for a ticket broker to direct buyers to their website. The phone call interaction may still prove effective as the interaction may still be preferred by the buyer… but the point here, is to establish in principle how much more effective and efficient a dynamic website is, for this particular business type, than is a static display ad in the Yellow Pages.

In conclusion - there may still be value in the YP, the question is how much? and for how much longer?

The best litmus test is to test it for yourself. Look at the numbers. Think objectively about how people search. Broad strokes do you think your neighbors and friends, colleagues and aunts would pick up the yellow pages or go online to find your business? Do you think online directories are valid? Or would you rather put your website in front of the largest pool of savvy buyers (people who use digital search).

The nice thing about the digital world is the availability of data.

Test it for yourself.

You can very quickly determine and compare how people are searching online.

Go to Aaron Walds site: www.seobook.com and use his free key word analysis tool, or pay for a subscription to Wordtracker.com. You can quickly determine search trends and estimated search volumes for particular phrases and variations online.

Then ask your local yellow page company to provide stats (they can do call tracking numbers, but might not be inclined to share the results) for your business type.

It is rare to find a designer of anything who favors function over form. When you do, the “designer” may be downgraded by their peers into “engineer” status. Most designers prioritize form over function, and to their credit they probably know full well that by the time the engineers get their hands on their design, some of the form/ style will be shaved off to save money or to appease the market research crowd. In most industries there is a natural tension between design and function. Call it tension or a difference in priority — it is easy to see in a few industries (the auto and fashion industries for starters). The gestation process is clear in the auto industry as we see the aggressively new-age prototypes referred to as “concept cars” at auto-shows in London and Chicago morph into slightly improved (on a design level) if noticeable at all models by the time the cars actually go into mass production and hit the showroom. A similar process, thankfully occurs in the fashion industry. Famous designers create aggressively edgy fashion “couture” (a word I recently learned when my wife used it while shopping for ridiculously over-priced jeans) that only 1% of 1% of the world’s population would ever wear in public or fit into for that matter. And so between the runways in Milan where they showcase these designs and the actual display rack at Nordstrom most of the edgy design is lost in favor of more salable merchandise.

I guess with cars and fashion most people are just a bit too conservative to wear or drive prototypes.

What does this have to do with websites you ask?

Quite a bit actually.

The conflict between designers and engineers is very clear in the realm of websites. Web designers are aesthetically literate. They build “beautiful” looking websites. Web designers see in terms of lines, negative space, color, font, spacing, size, etc.

Where as web engineers (SEO guys) are functionally literate. They see in terms of performance, page rank, outcome, practicality, usefulness and efficiencies.

When it comes to websites: the conflict at first glance is not as obvious as the design of a car. Automakers must wrestle with far more compromise. Mainly because of most people cannot afford to spend $85,000 on a car, or $85 at the pump. So auto makers choose fuel efficiency over horsepower, and give up voice guided navigation and hand stitched Italian leather for a compass and cloth seats (and 45 other factors) to deliver a car in a more reasonable price points. Of course this analogy is not comprehensive, luxury auto brands sacrifice precious little and favor high price-point low volume production over lower margin mass-production — and from the looks of it, theirs no-shortage of people who are willing to ante up.

Back to web-design.

At The Search Engine Guys, we have a very simple agenda. We favor performance and function over design. We are pragmatists on this issue.

What is the point of having a website if prospective clients don’t view it?

What is the point of having a beautiful website if prospective clients don’t view it?

That said, we believe you can have both. We just prioritize performance ahead of design. Once a website has achieved rank, which in our book is first page placement on Google. Then let’s focus on tweaking the look, the lines, the color, the photos etc.

We are quite happy to work with web designers and commonly work on re-designs for clients.

Unlike other industries — a website is a digitally-organic thing. It can be revised continually. It never comes off the production line. So the conflict is not a natural either or.

If you have spent any time on the web, you have seen both extremes.

On a regular basis, we come across beautiful looking websites that very well could be framed and hung on the wall in the Tate Modern in London. Sites that with well balanced negative space, visually interesting hi-res photos and lines that observe all the religiously precise 1/3rd rules that they preach in art school.

The point is, that often these sites are buried on the 132nd page of Google.

The other extreme, is one you inevitably see more often. I call this extreme the “flee market approach.” A website so packed full of links, ads, content that it emits that intense vibe you get when standing in a flee-market. It’s overwhelming. Difficult to look at. It may be SEO friendly but not easy on the eyes.

Clearly there is a balance.

At The Search Engine Guys we recognize that our clients have different priorities, different personalities, and work within different industries. We strive to understand the nuances of our individual clients and implement our proven SEO processes to achieve optimal rank. Once we achieve rank, we are able to study the performance of the site in terms of how long people are spending on various pages, etc., from there we are able to test and refine both the content and the look and feel to maximize performance.

The challenge when it comes to content vs. beauty is to not “let the tyranny of either / or ruin the possibility of both / and” how’s that for a parting cliche’ for you?