Why You Cannot Reverse Engineer Google’s Algorithm

Some people just cannot seem to learn what it means when Google says “We made 400+ algorithm changes last year!”

There was a time, many years ago, when you could design a simple Web document, submit it to a search engine, and evaluate how the search engine processed the document. We called that reverse engineering the algorithm. There were no lists of “ranking factors”. People weren’t concerned with much more than using the right keyword density and loading up the keywords meta tag with all the terms they were afraid to write copy for.

Inktomi changed all that. They made links important to have. With too few links you were dropped from the main index of Inktomi and you vanished from dozens of search engines. When we learned that building out links across multiple sites fixed the Inktomi problem we set ourselves on the path of overcoming Google’s anti-spam proposition: that links somehow determined the quality of a document.

That was never true, although people still believe the nonsense that Larry Page and Sergey Brin cooked up in 1998 with their Backrub project. By 1998 Web spammers were already using links to lead people down the garden path of deception. And innocent Webmasters were loading up their simple pages with lists of links because they couldn’t think of anything else to share on the World Wide Web. Links were worthless as “endorsements” before Google won its first $100,000 angel investment.

Still, here we are 12 years later and people are now trying to figure out how to topple Google from its pedestal. Google sneezes several million dollars an hour and people fall down and worship the Great Google Algorithm. Truth be told, Google’s Web indexing and searching processes may indeed be one of the greatest algorithms ever developed in human history — and the ironic thing is that links play a microscopic part in that process.

The Great Google Algorithm is not a set of ranking factors; rather, it is a collection of protocols, operating systems, applications, databases, and occasional information retrieval processes. I suspect that Google devotes more resources to maintaining and managing its resources than it does to actually providing its core search services.

Given the immensity of the Great Google Algorithm one should think the search engine optimization community would have given up trying to figure out what Google is actually doing. And yet, year after year we see new speculations published on what and how Google is doing. People like Mike Blumenthal and Michael Gray have shared some of the darnedest, uncanniest, most insightful speculative articles about Google’s electronic doodling I have seen in a long, long time over the past year.

They have been accorded too little recognition for their insightfulness.

And it’s too late to do anything about that because…well, because everything they have figured out about Google in the past few months is pretty much algorithmic history by now. By the time you read this article, any speculations I might have shared about the Google algorithm will be pushing the edge of aginess. By the time this article is 3 months old, all our dreams of Google algorithmic ranking factors will be little more than fantasy and nonsense.

It takes more time to think up a couple hundred potential “ranking factors” than Google needs to change the mix several dozen times. The Great Google Algorithm changes at an exponential rate, faster than the SEO community perceives, more flexibly than would-be algorithm chasers are prepared to understand. I doubt there is anyone at Google who fully understands the process — it’s too big, too everfluxy, for one human mind to comprehend it.

Sure, they probably have various models that show what state the system is in: maybe project status charts, engineering assessments, lists of priority deliverables, and other traditional metrics that map ordinary business projects. But the Great Google Algorithm (and the Great Bing Algorithm) is too big, too complex. All the masters of the machinery can do is look at the dials, turn the valves according to their training, and hope that the steam doesn’t build up past the point where something has to blow out.

Google and Bing are rather steampunky at the moment. They require thousands of engineers, tweakers, researchers, testers, and other experts and technicians to keep the machinery functioning. They have not yet moved into the Star Trekkian nuclear age of search algorithmic science; they cannot simply develop the concept and tell the computer to put together all the pieces to bring it into existence.

Someone is still attaching nuts and bolts to iron gears and braces down in the engine room, but both search engines have really, really big engine rooms. They each have at least a dozen Commander Scotts trying to eek a few more Warp factors out of the boilers. Maybe there are hundreds of Scottys at both search engines all bustling around their little search engineering fiefs, yelling at their deckhands to use “the right tool for the right job!”

The search engine optimization community is certainly fascinated by the whole Black Boxian world that the search engineers have constructed. We eagerly peruse every video and interview featuring someone from the search engineering teams, gleaning tidbits of Web information retrieval wisdom from their glib, vaccuous answers.

We browse the same stupid questions over and over again, looking for the extra little insight that guys like Matt Cutts and John Mueller might share out of pity for some struggling Webmaster’s frustration. And we compile these lists of “ranking factors” thinking that maybe we’ll develop some keen insight into the inner workings of the Great Web Search Engines.

Unfortunately for the SEOs who put their faith into ranking factors there are two pieces of the puzzle that are seldom discussed and never evaluated. The first missing piece of the puzzle could be called the Factor Weights. That is, let’s say a search engine’s algorithm uses 1,000 factors to figure out the order in which to list Websites in search results. How does the search engine process those factors? Which of them matters the most? And how can that be quantified?

Naive people who despite years of being shown just how complex search algorithms are still say stupid things like, “Links are the most important factor”. There is absolutely no credible public proof that ANYTHING is “the most important factor”. But even if Google and Bing were to announce, “Hey, Factor 561 is the most important ranking factor” today, next week they would tweak their systems and then Factor 327 would become more important than 561.

But it’s really not even that simple. After all, different queries call for different selection and ranking criteria. There isn’t really just one big algorithm figuring out how to handle billions of queries — it’s more like there are lots of little algorithms. Scotty is in the background yelling, “How many times do I have to tell you? ‘The Right Tool For The Right Job!'”

Search engines change the mix every week, probably every day. Matt Cutts once disclosed on his blog that at any given time there are probably three alternate algorithms serving Google search results around the world. Perhaps today there are even more than that. So if you’re going to try to evaluate “ranking factors”, which set of ranking factors are you looking at? Algorithm 1? Algorithm 2? Algorithm 3?

Factor Weights are, to my knowledge, never disclosed. But people like Vanessa Fox (who no longer works for Google) and Matt Cutts (who still works for Google) have stated time and time again, “Things are tweaked every week.” You’re never dealing with the same algorithm twice. Your analysis of search results today won’t be relevant to the search results tomorrow.

Of course, some people will be quick to point out that for many queries the results are pretty stable. Hence, if the results are the same today, tomorrow, and next week, we can still analyze them and derive some insight into what Google and Bing are doing. Right? Wrong. It doesn’t work that way.

There is more than one way to skin a cat, more than one way to drive from New York to Los Angeles, and more than one way to spell “color” (or “colour”). Given that the same result can be achieved through multiple paths, analyzing the search results with the hope of knowing which ranking factors are being used or how those ranking factors are being weighted is a waste of time.

Today’s Sudoku ranking factor puzzle is different from yesterday and tomorrow’s Sudoku ranking factor puzzles. Even if you can show scientifically (and no one can) that your correlations and analyses are correct, you’re only painting a picture of a rock that the boat has long since passed by in the river of everchanging search algorithms.

And that leads us to the other missing piece of the puzzle: Analytics Latency. As the search engines figure out new ways to mix up their algorithms at more frequent paces, our analyses of those algorithms fall farther and farther behind. The scale of analysis calls for weeks or months (even if you can cull the results of 20,000 queries tonight) of planning, coding, and testing.

Maybe if we had 100 analytics teams working together in some sort of grand procession we could keep up with the search engines. They have dozens if not hundreds of engineering teams tackling all sorts of problems. Google proved just a few weeks ago that it can assemble a team of engineers and effect an algorithmic change in a matter of days. (Conversely, they can also face problems that require five months to sort out — but they have resources which no SEO company can match.)

This is why lists of ranking factors have become such useless pieces of drivel. They provide no insight into the inner workings of search engines. At best, like correlation studies, all that ranking factor lists do is offer some insight into the way SEOs think search engines work.

You get far more information about what the SEO community is doing from these types of “studies” than you can ever hope to get about what the search engines are actually doing. And this is the dangerous thing about correlation studies and ranking factor surveys: they assume false veneers of authority. People point to these things and say, “See? This is how Google works!”

If analyzing Google and Bing’s algorithms were really as simple as asking 600 SEOs what they think is most important, Google and Bing would have figured out a way to make the process more complex by now. Why? Because their financial incentives lie in providing their users with acceptable search results, not in providing Webmasters with reliable monetization.

Web search is an odd thing: those who pay the bills (Web publishers) are not those who decide where the money goes. There is no Golden Rule in the searchable Web ecosystem. Those who have the gold just don’t get to make the rules.

Now, all that said, we can still claim some small victories in the war against the algorithms. It is certainly possible to test many of the statements that search engineers make about the way search engines work. SEOs do that pretty successfully every day. But these types of tests are gut instinct-level toe-dips in the water. We only do what doesn’t hurt for any length of time and we stop doing what does hurt as soon as we figure out why we’re in pain.

Sure, you can perform simple tests to see if the keywords meta tag is indexed (not for Web search), if pages with keywords in the title and URL have an advantage over pages that only have keywords in body copy (they might), and if links’ anchor text can help boost rankings for a specific phrase (they can). You can even put multiple links to the same destination on a page and argue that only the first link counts or the second link also passes anchor text.

These tests don’t reveal anything about how important any ranking factors may be to a search engine. They don’t even provide insight into how important those factors are to the SEO community.

Ranking factor surveys and correlation studies have a pretty good track record for documenting what SEOs think is important. If you want to assess the maturity of the SEO industry’s grasp of search technology, then look at the correlation studies and ranking factor surveys. They teach us how naive we still can be, as well as how creative we strive to be in influencing algorithms we cannot reverse engineer.

The funny thing is, SEO opinions don’t seem to be static. I have watched them change. Sometimes it takes a great intervention to change SEO opinions. The PageRank Sculpting Debacle lasted for two years before Google put a stop to that nonsense by revealing that it had long since changed the way it handles PageRank for documents that contain links with the “rel=’nofollow'” attribute. Very few people talk about sculpting PageRank any more.

Yet even more astounding is the recent gradual shift in SEO opinions about the value of links versus content. It seems many more people are now challenging the long-held view that links are all you need. I still don’t see much robustness in the assessment of content’s value to search engine optimization but give it a couple of years: eventually most SEOs will come around to realizing, thinking, and insisting that you need rich, lengthy content that lasts forever rather than thin, quickly churned-out keyword-injected fluff pieces.

And probably in about two years, when new correlation studies are being published, and new ranking factors are being compiled, SEOs will be trumpeting their recent discoveries that complex, rich content is highly valued; that trust and reputation are bound up together; and that search algorithms really are too complex to be reverse engineered.

We can figure out a few things day by day but the searchable Web ecosystem is bigger than us, changes faster than we change our minds, and has evolved even beyond the most advanced theories offered by the SEO community.

If that seems disheartening then take consolation in this, at least: remember that the academic community is at least 5 years behind the SEO community in identifying Web spam techniques. That’s because there are more of us than there are of them, and they have no hope of outthinking all our desperate attempts to outthink the search engines.

See also:

• 5 Reasons Why You Cannot Use Science to Show How Links Influence Search Results
• Why Correlation Studies Hurt Your Web Marketing
• Google Correlation Studies are Sham Search Engine Optimization

Watch this video titled “Marketing Correlation Studies: How Misleading Are They?” recorded on June 25, 2016 where Kent Yunk and I discuss correlation studies.