The purpose of this thesis is to analyse studies that evaluate Web search engines. This is done in four categories; the researchers’ purpose, the evaluation measurements, the relevance, and the time aspect. Our method is based on a text analysis in which we use the direction of analysis of the content of sixteen evaluation experiments. Our results indicate fundamental differences in the way the researchers are tackling the problem of evaluation of Web search engines. We think that, despite the differences that we have been able to identify, it is necessary to perform evaluation experiments, so that methods can be developed that can guarantee the quality of the Web search engines. To provide people with the kind of information they need is the main task for Web search engines. In an increasing flow of information that task will be even more important. Evaluation of Web search engines can be a part of improving the efficiency of the Web search engines and in that way strengthen their roll as important information resources.