| |
Online ISSN:1349-8606 |
| Progress in Informatics |
|
| No.1 March 2005 |
|
| Page 59-73 |
PDF(1,266KB) | References |
| doi:10.2201/NiiPi.2005.1.5 |
| The test collection for navigational retrieval on WWW data-Design and characteristics |
| Keizo Oyama1, Haruko Ishikawa2, Koji Eguchi3, Akiko Aizawa4 |
1, 2, 3, 4 National Institute of Informatics
1, 3, 4 The Graduate University for Advanced Studies (SOKENDAI)
|
(Received: November 22, 2004)
(Revised: February 2, 2005)
(Accepted: February 21, 2005)
|
Abstract:
This paper describes the design and characteristics of a test collection for navigational retrieval of WWW data that was built through the WEB Task of the Fourth NTCIR Workshop to evaluate the retrieval effectiveness of Web search systems. This reusable test collection consists of 100 gigabytes of Web document data and 300 topics of various types and corresponding relevance judgments. Among the several types of ‘Navigational Retrieval’, we selected the ‘Known Item Search’, which simulates a situation where a user searches for one or a few ‘representative Web pages’ of a known item. It is assumed that the user knows about the item but may not have seen its Web page. Relevance judgments were performed on the probable documents mainly from the viewpoint of representativeness of respective known items represented by the topics. Using the judgment results, several evaluation measures were applied to various retrieval results. Based on the evaluation results, relationships among the types of topics, Web-page styles and search methods are discussed. The stability of the evaluation results with different numbers of topics is also analyzed.
|
Keywords:
Web information retrieval, evaluation methods, test collections
|
|
|
| PDF(1,266KB) | References |
National Institute of Informatics is a member of CrossRef.
|
