11/19/22, 4:47 PM How Search Engines Work

Thanks for trying out Immersive Reader. Share your feedback with us.

How Search Engines Work

1/11
11/19/22, 4:47 PM How Search Engines Work

Let’s begin by exploring what search engines are, why they exist, and how they make money.

What are search engines?

Search engines are searchable databases of web content. They’re made up of two main parts:

1. Search index. A digital library of information about webpages.

2. Search algorithm(s). Computer program(s) tasked with matching results from the search
index.

What is the aim of search engines?

Every search engine aims to provide the best, most relevant results for users. That’s partly how they
gain market share.

How do search engines make money?

Search engines have two types of search results:

1. Organic results from the search index. You can’t pay to be here.
2. Paid results from advertisers. You can pay to be here.

Each time someone clicks a paid search result, the advertiser pays the search engine. This is known as
pay-per-click (PPC) advertising, and it’s why market share matters. More users mean more ad clicks
and more revenue.

2/11
11/19/22, 4:47 PM How Search Engines Work

Each search engine has its own process for building a search index. Below is a simplified version of
the process Google uses.[1]

3/11
11/19/22, 4:47 PM How Search Engines Work

Let’s break it down.

URLs
Everything begins with a known list of URLs. Google discovers these in many ways, but the three
most common are:

From backlinks. Google has an index of hundreds of billions of webpages.[2] If someone

links to a new page from a known page, Google can find it from there.
From sitemaps. Sitemaps tell Google which pages and files you think are important on
your site.[3]
From URL submissions. Google lets site owners request crawling of individual URLs in
Google Search Console.

Crawling
Crawling is where a computer bot called a spider visits and downloads known URLs. Google’s crawler
is Googlebot.[4]

Processing and rendering

Processing is where Google works to understand and extract key information from crawled pages. To
do this, it has to render the page, which is where it runs the page’s code to understand how it looks
for users.

Nobody outside of Google knows every detail about this process. But it doesn’t matter. All we really
need to know is that it involves extracting links and storing content for indexing.

Indexing
4/11
11/19/22, 4:47 PM How Search Engines Work

Indexing is where processed information from crawled pages gets added to the search index.

The search index is what you search when you use a search engine. That’s why getting indexed in
major search engines like Google and Bing is so important. Users can’t find you unless you’re in
the index.
Did you know?

Google owns 91.43% of the search engine market. It can send you more traffic than other search
engines, as it’s the one most people use.[5]

Discovering, crawling, and indexing content only make up the first part of the puzzle. Search engines
also need a way to rank matching results when a user performs a search. This is the job of search
algorithms.

What are search algorithms?

Search algorithms are formulas that match and rank relevant results from the index. Google uses many
factors in its algorithms.

Key Google ranking factors

Nobody knows every Google ranking factor because Google hasn’t disclosed them. But we do know
some key ones. Let’s look at a few of them.

Backlinks

Backlinks are links from a page on one website to another. They’re one of Google’s strongest ranking
factors.[6] This is probably why we saw a strong correlation between linking domains and organic
traffic in our study of over a billion pages.[7]

5/11
11/19/22, 4:47 PM How Search Engines Work

It’s not all about quantity, though. Quality matters too. Pages with a few high-quality backlinks often
outrank those with many low-quality backlinks.
Did you know?

You can check backlinks to your website for free in Ahrefs.

Sign up for a free Ahrefs Webmaster Tools account, enter your domain in Site Explorer, and go to the
Backlinks report.

6/11
11/19/22, 4:47 PM How Search Engines Work

Our crawler is the fifth most active on the web,[8] so you’ll see a pretty complete view of your
backlinks here.

Relevance

Relevance is the usefulness of a given result for the searcher. Google has many ways of determining
this. At the most basic level, it looks for pages containing the same keywords as the search query. It
also looks at interaction data to see if others found the result useful.[9]

Freshness

Freshness is a query-dependent ranking factor. It’s stronger for searches that call for fresh results.[9]
That’s why you see a recently published top result for “new netflix series” but not “how to solve a
rubik’s cube.”

7/11
11/19/22, 4:47 PM How Search Engines Work

Page speed

Page speed is a ranking factor on desktop and mobile.[10][11] But it’s more of a negative ranking factor
than a positive one. This is because it negatively affects the slowest pages rather than positively affect
lightning-fast pages.
Did you know?

You can check your page speed for free in Ahrefs.

Sign up for a free Ahrefs Webmaster Tools account, crawl your website with Ahrefs’ Site Audit, then
go to the Performance report. In general, the less red you see, the better.

8/11
11/19/22, 4:47 PM How Search Engines Work

Mobile-friendliness

Mobile-friendliness has been a ranking factor on mobile and desktop since Google’s switch to mobile-
first indexing in 2019.[12]

Google tailors search results for each user. It uses information such as your location, language, and
search history to do this.[9] Let’s take a closer look at these things.

Location
Google uses your location to personalize the results for searches with local intent. That’s why all
results for “italian restaurant” are from or about local restaurants. Google knows you’re unlikely to fly
halfway around the world for lunch.

Language
Google knows there’s no point in showing English results to Spanish users. That’s why it ranks
localized versions of content (if available) to users who speak different languages.

9/11
11/19/22, 4:47 PM How Search Engines Work

Search history
Google saves the things you do and places you go to give you a more personalized search experience.
[13] You can opt out of this, but most people probably don’t.

Key takeaways
A search engine consists of two main parts: index and algorithms.
To build its index, it crawls known pages and follows links to find new ones.
The aim of search algorithms is to return the best, most relevant results.
Search result quality is important for building market share.
Nobody knows all of Google’s ranking factors for organic results.
Key ranking factors include backlinks, relevance, and freshness.
10/11
11/19/22, 4:47 PM How Search Engines Work

Google personalizes its results based on your location, language, and search history.

References
1. “Understand JavaScript SEO Basics”. Google. Retrieved August 16th 2022.
2. “Organizing Information — How Google Search Works”. Google. Retrieved 16th August
2022
3. “Learn about sitemaps”. Google. Retrieved 16th August 2022
4. “Googlebot”. Google. Retrieved 16th August 2022
5. “Search Engine Market Share Worldwide”. Statcounter. Retrieved 16th August 2022
6. “Google Q&A+ #March”. YouTube. Retrieved 16th August 2022
7. “90.63% of Content Gets No Traffic From Google. And How to Be in the Other 9.37%”.
Ahrefs. January 31, 2020. Retrieved 16th August 2022
8. “CloudFlare Radar”. CloudFlare. Retrieved 16th August 2022
9. “Ranking Search Results — How Google Search Works”. Google. Retrieved 16th August
2022
10. “Using site speed in web search ranking”. Google. Retrieved 16th August 2022
11. “Using page speed in mobile search ranking”. Google. Retrieved 16th August 2022
12. “Mobile-first indexing best practices”. Google. Retrieved 16th August 2022
13. “Find & control your Web & App Activity”. Google. Retrieved 16th August 2022

11/11