by Joe Miller
In late May, Google announced a whole suite of changes to its core product: Google search.
The biggest of those changes is that the company is further integrating AI into search results. The company is doubling down on its efforts to answer your questions directly, rather than simply giving you links to websites where you can find answers to your questions.
AI skeptics were vocal in their displeasure – my social media feeds were full of variations on “remember when you could type your query into Google and get 10 links to answers written by humans?”
The critics have a real point. I tried using Google search to research this column, but it just sent back pages of AI-generated You Tube videos. I finally asked Claude to help sort through the clutter.
That research confirmed what I’d vaguely remembered – low-quality content isn’t a new problem.
Between 2009 and 2011, the web was overrun with content farms – companies whose entire business model was to identify search terms that were not well-served by newspapers or e-commerce. These farms then paid writers a few dollars per article to draft copy that would rank highly on those searches. They then stuffed the pages full of advertisements.
If you’ve been around the web a long time, you might remember eHow or Associated Content. As media researcher and professor Dave Karpf pointed out, these sites didn’t need to provide good answers; they merely needed to provide relevant answers.
Google eventually updated its search algorithm to deprioritize content farms. Sites like eHow lost somewhere between 20 and 40% of their visibility within months of the change.
In 2012, Google made another big update aimed at improving information quality, adding what it called “a knowledge panel” to the right of search results. That box surfaces all sorts of factual details about your search topic.
That was part of a move that Google called “things, not strings.” Computers notoriously do not understand the meaning of text – a bunch of text is, in tech-speak, a string, and Google’s algorithms consisted of elaborate sets of rules for matching the string entered into a search query with strings that appeared on webpages.
Google’s change was part of a larger project called the semantic web. Semantics is the study of the meaning of language. The semantic web is about helping computers understand the meaning of words.
Take, for example, something like the hours of operation for a business. A human can just read a chunk of text that says, “Monday to Saturday, 10–6” and know what that means. A computer does not.
So, a bunch of big tech companies got together and developed a set of standardized tags that developers could use to markup webpages. If you were to open up the source code of a webpage, you’d see a bunch of code, some of which would say stuff like “Opening Hours Specification” and “Day Of Week.”
That markup tells computers that the text strings that follow will specify the days of the week that an establishment is open.
The result is that if you Google “Rivertown cafe Marlinton WV” you’ll see the hours of operation in the knowledge panel to the right. Google gets that information from the cafe’s website, and it knows which string to use because the website structures its markup in a standardized way.
The big drawback to the semantic web is that it’s incredibly labor intensive to implement. You need a tag for every concept you want to express and then you need a way to apply those tags. It’s possible to do that for an ecommerce website. It’s just not feasible to tag every idea in an essay.
Large language models – the technology behind what we think of as AI these days – are a different approach to the same problem.
Instead of relying on a bunch of standardized rules for looking up the meanings of strings of text, LLMs apply sophisticated statistical algorithms to huge bodies of text – and by huge, I mean a sizable percentage of everything ever written.
That process allows LLMs to interpret and use natural language in meaningful ways.
Google isn’t really doing anything new in rolling out AI into search. The semantic web is simply a different approach to AI, and Google added that in 2012. It’s part of Google’s attempt to understand what its users are trying to do and then help them accomplish that task.
And it turns out that what most people want isn’t a link to a webpage that will answer their question. They just want an answer to their question.
The trick today – much as it was at the peak of the content farm era – is how to sort out meaningful content from low-quality stuff whose creators spend far more time optimizing to show up high in search results than they have in providing useful information. Google is betting that AI mode is a solution to the current flood of AI-generated content.
