How Search Engine Optimization works - lesson 3

Introduction to how search engine works

To improve a web page position on SERPs, first of all we have to know how search engine works. In this article we will expose all about Internet search engines such as definition of search engines, tutorials on web crawling, indexing and searching of web search engines.
how search engine works
Fig: Search Engines

Web search engines are special sites on Internet which are designed to search for information on the world wide web.Search results are represented in a list consisting of web pages, images, information and other types of files. There are basically three types of search engines.

  • search engines which are powered by robots (called crawlers; ants or spiders)
  • search engines which are powered by human submissions and
  • those that are a hybrid of the two.
Robots based search engine use automated software named as spiders, bots ,crawlers to search the web and analyze the individual pages, then data collected from each pages are added to the search engine index. Human powered search engines depend on humans to list a web page on their database. Manually you have to submit information of a site to these search engines or directories, then it is subsequently indexed and catalogued based on your submitted
information. In Hybrid search engines both crawler-based results or human-powered listings are used. Here we will discuss only first type of search engines. All search engines basically perform three tasks:
  • They search the web to gather document of web pages and their web addresses using spider or crawler software.
  • The indexing software extracts information from the documents, index them based on keyword and store it in a database.
  • They allow users to look for words or combinations of words found in that index.
To do their task properly each day search engines have to crawl hundreds of million pages
, index new web pages and update their old database continuously. Now we will discuss basic three tasks of search engines in details.

Web crawling of search engines

Search Engines use automated program called "Spiders" to find new pages or changed information on the hundreds of millions of web pages. The process of building lists of the words found on the web sites is called web crawling.

web crawling,indexing of search engine
Fig : Web Crawling by Spider

One question may arise, How does any spider start its travels over the Web? The spider will begin with a high ranking or popular sites, indexing the words on its pages and then follows every links found within the sites. In this way, spider starts to travel and visits the most widely used pages in the web. The content of each pages is analyzed to determine how it should be indexed. Search engines looks for relevant keyword of the content of the web pages. Then index that page with that keyword. When spider looks at a HTML page, it gives special weight to keywords that appear in :
If a site has high page rank, it is spider ed more often and more deeply. So spider crawl the entire web starting from popular sites to all links on those sites. But if your site or blog is very new then there is less probability to get inbound links or backlinks from higher page ranked sites. Then a question arise, how does spider crawl a new site or blog? There is a option of manual submission on every search engines. If you are new site or blog owner then first submit to popular search engines.

Indexing and Searching

Now spider have collected the information from the web and search engines must store those in index database for later queries. The main purpose of indexing is to find a information as quickly as possible. In simply search engine could store the keyword and the URL of a page in their database. But it would not be useful for them. To make efficient and useful result most search engines store more than just keyword and the URL. Normally the principle of search engines are not revealed for their business, so nobody can guarantee a #1 ranking on any SERPs. An engine might save total numbers that the keyword appears on a page. The engine might give a rank to each web page depends on the lot of factors such as no of backlinks, keywords, relevancy of content etc. But each search engine uses different algorithms on how to index a page. So this is the one reason why different search engines produce different search lists for a specific keyword query.

tutorials on search engine

"Spiders" take a Web page's content and create key search words that enable online users to find pages they're looking for.

When a user searches specific keywords on search engine website, the engine returns a listing of best-matching web pages by examining it's index database. They provide a short description of the content containing the title and the URL in the SERPs. To build a complex query most search engines support the use of Boolean operator that allow to extend the terms of the search. To use the search engines more efficiently you can follow the instructions below :
  • AND - All the keywords joined by "AND" must be in the web pages or documents. You can replace "AND" with "+" in some search engines.
  • OR - At least one of the terms joined by "OR" must be in the web pages or documents. You can replace "OR" with "|" in some search engines.
  • NOT - If you want a keyword not showing in the web pages , you can use this operator. Some search engines substitute the operator "-" for the word NOT.
  • FOLLOWED BY - One of the keywords must be followed by the other.
  • NEAR - One of the terms must be within a specified number of words of the other.
  • Quotation Marks - The word between the quotation marks is considered as a phrase. It must be on the web pages

You can find the remainder of these lessons below:

Comments Post a Comment

SeoNext - September 09, 2009

Search engine optimization is a pretty involved process and methods and opinions can sometimes differ. All search engines use different algorithms to rank how websites are displayed and they are usually kept secret, but some of it is known and can be inferred. Older search engines were much easier to exploit (such as Infoseek) as they often only considered a single page so repetition of keywords and placement of terms could bump you to the top. Newer search engines such as Google use other factors often beyond your control to ensure that the top results are more relevant and trustworthy. There are entire sites and books devoted to search engine optimization which provide various tips, but a lot of it just takes time. Thanks for sharing this information with us.Really an informative post.

IEDig - September 13, 2009

IEDig is a social content website where your readers or you can
submit content to. If you have a good story, members will 'Vote' the post
and write comments. As a blog owner, you may want
to make it easy for and encourage your readers to Visit on Your Blog.


Cinderella - September 14, 2009

thks....useful info - September 19, 2009

IEDig is a social content website where your readers or you can
submit content to. If you have a good story, members will 'Vote' the post
and write comments. As a blog owner, you may want to make it easy for and encourage your readers to Visit on Your Blog.


Rohit Rajgaria - September 23, 2009

thanks a lot for the valuable information about how SEO works....its a good articls...

Blogging Tips-Tricks and Tutorials

Chris - October 06, 2009

Excellent post. I learned a lot from this and it will certainly help me in my understanding of SEO. Thanks.

SEO Services - October 27, 2009

Thanks for your tips .It is very helpful & informative for doing SEO.I still newbie in this field, so this article pretty helpful for me. Keep up the good work.I am hoping the same best work from you in the future as well.

Anonymous - October 01, 2010

Thanks for u r information

its very useful

Handsome Seo - October 13, 2010

great artical awesome tips thanx for the share with us great information thanx alot and great blog keep it up!

SEO services.

Leave a Comment