For a detailed exposé of the inner workings of Google searches, read the book How Google Works, written by two high ranking Google executives. As such, it can be considered the definitive source by the people that know it the best.
Spoiler alert: they don’t really know how it works themselves.
The various search engines, such as Google, Yahoo, or Bing, all operate with different complicated algorithms. Most people wonder about this from time to time, but it isn’t a serious thing to learn until a website owner tries to be ranked high in search results. Then, there isn’t enough information in the world. However, there are some clear basics.
The internet works like a spider web, in that everything is connected and the spider goes out to retrieve what is needed and collects it back in the web.
There is a huge mess of information on the internet, dispersed onto trillions of websites. A search engine sends a spider/bot to collect all the relevant information based on a given keyword search. When someone enters a search term into Google, the bots scramble to find all the occurrences of that term found on websites and then return a list of the most relevant and useful results.
Sounds simple enough, only it’s not simple at all. There are more than 60 trillion web pages that bots must crawl within a matter of seconds. More importantly, just like in the real world, some neighborhoods are better than others, so the bots must consider each hit for quality and relevance.
In the simplest way of operating, a search would return results in order of the keywords most-frequently tied to the search phrase. However, not all websites are created equally, so there is a ranking system for the authority of sites. For example, a Google search for “live music in Boise” returns a list of sites that give information about live music in Boise. However, the bot needs to understand that the search intends to see results such as Boise Weekly’s music calendar or Go Listen Boise, which are appropriately the top two ranked results.
A hundred listings later, the results become things like radio stations or Tumblr pages displaying tags about live music in Boise. These types of results are somewhat related, but not very comprehensive or the most relevant. Two hundred listings later, the results have become things like Ticketleap selling tickets to individual concerts. While also related, it isn’t helpful to know that you could have seen Ryan Bingham at the Knitting Factory three months ago when you want to know what the options are for next weekend.
To rank pages in order of what the search is intended to return, the bots must have be able to consider relevance.
Websites that have more links going to them get a higher rank. Now, this idea encourages website owners to put a lot of links on pages so that Google bots will crawl them more often. That can increase a ranking, if the linked sites also have a high ranking and there is a logical reason for the links being included. Otherwise, Google bots will shy away from them.
Google will not return dubious things like porn sites, unless they are explicitly asked for. If the website contains a lot of spammy keywords, the bots will avoid it. Google may even decide to block a site from its bots if it is deemed too spammy to be of any use. If the links on the website are “not followed” or otherwise hidden from view, then the bots won’t even crawl there.
Websites that specialize in the subject of the search, or community-focused websites in the location where the search was made, will rank higher. Thus, with the above example, sites that have updated live music listings in Boise rank very high because they do exactly what the search terms request and in the correct location. This infographic from the University of Alabama-Birmingham on the Science Behind Google shows some of the features that the main search engines deem important.
The complexities within search engines has given rise to search engine optimization (SEO) and search engine marketing (SEM) as two major industries. To have an optimally performing website is to be an authoritative source in the 21st century. It may well be that a website is excellent and relevant and still not showing up in search results.
There are many steps that can be taken to make sure a website is performing optimally for the way the search engine operates. Is the website indexed? If not, it won’t show up in search results at all. It’s a glaring oversight that many websites never even realize. Is the site cached? Does the site use basic techniques like title tags or meta-descriptions? Does the website do further These steps make a huge difference in how well a site performs in search.
There’s a lot to learn about how to navigate the web to get the result you need first and even more to learn about how to get your website to become a high-ranking authoritative source. No matter how much you learn, it is an ever evolving process and in the end no one knows all the tricks, not even Google themselves.