- Lecture 3
- Meta-data driven multimedia retrieval and piggy-back retrieval
- 26 March, 13:30 - 15:00 Lecture room 2001, 20fl
- Trivial program specialization. Interpretation overhead, including self-interpretation. How specialization can be done; binding-time analysis. Speedups from self-application in the Futamura projections. Optimal program specialization.
- Lecture 4
- Automated image annotation
26 March 15:30-17:00, Lecture room 2001
Automated image annotation can be seen as a special case of piggy-back retrieval, but assigning meaningful text snippets to images, scenes or objects in them, has far wider uses. We will discuss algorithms such as non-parametric density estimation and label transfer, and study how co-occurrence and semantic world knowledge might be able to improve image annotation.
- Lecture 5
- Visual content-based retrieval I (features and distances)
23 April 13:30-15:00, Lecture room 2001
- Lecture 6
- Visual content-based retrieval II (practical considerations)
23 April 15:30-17:00, lecture room 2001
These two lectures are dedicated to the paradigm of content-based retrieval, where the query consists of a media excerpts and the returned media are expected to be similar in nature to the query. There are a number of difficulties to overcome in this approach. Unlike the near-duplicate example, it is not at all the case that the query is a slightly different representation of a known item in the database. "Similarity in nature" is a vague concept. For example, an image of a red toy Ferrari as query may well be expected to match a video of a black Cooper Mini as both represent cars. As such content-based retrieval needs to overcome both the challenges of polysemy (that a media excerpt as query can have many meanings), the semantic gap (that we already know from the lecture on automated image annotation) and scalability. I do not have a general solution for these challenges. Instead, both lectures will cover technical details of the best current practice for content-based retrieval. Through this we will come to an understanding where content-based retrieval is eminently useful and what its limitations are. We will analyse popular features and distance measures for visual content-based retrieval and their interplay. In particular, we will cover colour histograms, statistical moments, ways to turn texture into feature vectors, and shape encodings, as well as how to retain spatial information in the feature representation. Amongst the geometric, statistical and probabilistic distances we will see that the choice of distance measure matters, particularly so for high-dimensional spaces, and work out recommendations for an appropriate choice. We will study the most important practical considerations when building content-based retrieval systems: the choice of features, distances, their respective standardisation, fusion of feature spaces and query results, and the perils of high-dimensional indexing brought about by the curse of dimensionality.
- Lecture 7
- Evaluation of Multimedia Information Retrieval Systems
7 May 13:30-15:00, Lecture room 2004&2005
How can we measure the effectiveness of multimedia retrieval systems? TREC, TRECVid, ImageCLEF and similar evaluation workshops have long been fora of communities that try to model and assess retrieval tasks in a laboratory setting. The lecture gives examples of typical annual cycles of these evaluation workshops, and discusses a few typical tasks that try to capture particular aspects of retrieval quality.
We discuss important metrics that quantify the retrieval effectiveness of result lists returned by the search engine. These are precision, recall and derived measures. The lecture covers how image annotation systems differ in their evaluation need, and I develop and demonstrate alternative measures for their effectiveness.
- Lecture 8
- Added value: visualisation, multimedia browsing and geography
7 May 15:30-17:00, Lecture room 2004&2005
Search is only one aspect of multimedia retrieval. Even if the challenges of the preceding lectures in this series were all solved and if the automated methods we discussed so far enabled a retrieval process with high precision and high recall, it would still be vital to present the retrieval results in a way so that the users can quickly decide to which degree those items are relevant to them. In this lecture we examine a few paradigms of information visualisation, relevance feedback methods for visual search, browsing and, in particular, the relevance of geography to multimedia information retrieval.