So one fine day, you decide to search your name in Google. But then you are hit with over 75 million search results, in just under a moment’s notice. What kind of sorcery is this?
It is no magic (neither are you that famous), but rather just a clever implementation of big data, something that Google is an expert at.
A brief guide to how Google search works
Through Big Data, Google is able to pick the appropriate results out of trillions of web pages and several yottabyes of data within a few milliseconds. But that’s not all, as all these search results are arranged and displayed in accordance with their relevancy and number of hits.
So what is Google’s secret sauce? Well, to answer that question, we must first look at how exactly a search query is processed by the Google Search Engine –
Okay that is impressive. But how does Google manage all that data in their servers while also retrieving the information in less than a second?
As mentioned before, Google utilizes the concept of Big Data, or massive data set storage, to fill their servers with webpage information. To search through the billions of indexed pages that have stored in their multitudes of servers worldwide, the automated Google bots compile a MapReduce algorithm and form clusters of relevant datasets in a moment within a parallelizable environment. It’s like finding a needle in 2 haystacks, in less than 1 second.
Let us see how Map Reduce works in Google searches –
- User sends a search query request to the Google servers.
- Google bots compile the list of information related to the search query.
- A separated Big Data table (or Google file system) is created containing the file list and their subsequent logs.
- The files are then sorted using Google’s PR algorithm and displayed according to it.
With an average of 3.5 billion user requests being handled every day, one can only imagine how many results are displayed on a day-to-day basis by the largest and most popular search engine – Google.
It’s ok. You can gasp in amazement now.