||This paper gives an overview of the Web Retrieval Task that
was conducted from 2001 to 2002 at the Third NTCIR Workshop.
In the Web Retrieval Task, we attempted to assess the
retrieval effectiveness of Web search engine systems using a
common data set, and built a re-usable test collection
suitable for evaluating Web search engine systems. With
these objectives, we constructed 100-gigabyte and
10-gigabyte document data that were mainly gathered from the
`.jp' domain. Participants were allowed to access those
data only within the `Open Laboratory' located at the
National Institute of Informatics. Relevance judgments were
performed on the retrieved documents, which were written in
Japanese or English, by considering the relationshiop
between the pages referenced by hyper-links. Some
evaluation measures were also applied to individual system
results submitted by the participants.