Chapter 4
Video and Image Content
Representation and Retrieval

Introduction

Why do we need video? -- to illustrate motion.

What can we do with video data?

• capture, store, play back, edit -- almost without regard to the contents

But we also may want to do more things with video:

• query, retrieve, search, index -- depending on context

Some terms:

• Video sequence = Video stream (physical, raw data) + Video information (meaning)

We may need more than just raw video data or image sequences. For instance, for faster, more precise query execution.

Maximalist approach and Minimalist approach to content representation and indexing -- with respect to information extraction ammount from the video data.

Maximalist approach: David Marr proposed 3 levels of video data representation [Marr,D., Vision, Cambridge, MA: MIT Press, 1978]:

• primary sketch (encoding of local boundary elements in an image)
• 2.5D scetch (viewer centered representation of visible surfaces)
• 3D structure (object centered representation of shape using hierarchical decomposition)
• Example: Human as a cilinder. Then decompose it to head, torso and limbs that also are represented as cylinders.

Minimalist approach: Extract as minimal info as possible from the video.

May be we do not even need the raw video data ? (There was an experiment where small lights were attached to joints of a human subject dressed in dark cloths and observing the lights on dark background.Not only the object (human being) could be reckognized but also the actions and the events taking place).

How video content information can be used in multimedia query processing? Thats what this chapter is about.