SEO, Web Server Logs And Science

Web Server Logs, SEO and Science

When you talk with SEO consultants or in general with people about SEO, you often hear them saying "SEO is not an exact science" ,"Search engines are driven by algorithms which are updated constantly, it is impossible to follow them" , "Nobody can fully understand how google works even the engineers who are working on it". I could agree with all  those statements only if the web server logs do not exist.

There is one simple fact everyone whether being an SEO professional or not will confirm is "If a page is not crawled by a search engine bot, it can not bring in any organic visits from that search engine". This basic truth alone is the entry point of SEO in science. Although we can not completely perceive how google as a search engine works, there is a way to find out how google perceives a website entirely by analyzing that website's web server logs. 

Web server logs can answer two important questions.

In a certain period of time which pages of the website

  1. Have been crawled by a search engine bot.
  2. Are bringing in organic traffic by that search engine.

Introduction to Web Server Logs

What is a server?

A server simply is a computer which provides data to other computers.

Many types of servers exist, including web servers, mail servers and file servers.

What is a web server?

In computer science, the term "web server" refers to both a physical machine and software.

In the first case, it is a computer connected to the Internet and hosting resources. These resources can be files, programs or databases.

In the second case, a web server is a set of programs which operate and publicize a website or web application.

What are the popular web server softwares?


• Open Source

• PHP,Perl, Python, Ruby

• Linux,Unix, Windows, Apple OS X


• Open-Source

• PHP, Perl, Python, Ruby

• Linux,Unix, Windows, Apple OS X


• Commercial


• Windows

How to identify a web server software?

It is essential to know which web server software is running on a web server of a website before analyzing its logs.

There are several methods

1) Curl

2) Live HTTP Headers

On Chrome or Firefox, with this add on, it is possible to get HTTP headers of a page on a browser

3) Online Tools

Netcraft is also good for learning about the history of used web server softwares on the web server.

Builtwith is also handy to have an overall idea about the technologies used on the website as well as the web server software on the web server

Have comments, questions or feedback about this article? Please do share them with us here.

If you like this article

Follow Me on Twitter

Follow Searchdatalogy on Twitter


About Us

Our objective is bringing all our experience and expertise together to deliver solid technology solutions that can take your search traffic acquisition to the next level. Our main goal is to assist you in building and maintaining your search marketing analytics platforms. Our will is to leverage your marketing and IT teams search knowledge while bridging the gap between two.

Legal Terms Privacy

Recent Posts

BrightonSEO 5 days, 10 hours ago
1 Million #SEO Tweets 7 months, 2 weeks ago
SEO, Six Blind Men & An Elephant 8 months, 2 weeks ago
Best SEO Conferences In 2017 8 months, 4 weeks ago
SEO Hero 2017 9 months, 2 weeks ago
3 Ways For Free HTTPS 11 months ago
Crawl Dictionary 11 months, 3 weeks ago
Bing Strikes 1 year ago
HTTPS On Top Sites 1 year ago
SEO Web Server Log Files 1 year, 1 month ago
SEO, Web Server Logs And Science 1 year, 1 month ago
HTTP2 On Top Sites 1 year, 1 month ago

Recent Tweets