|
Group F: Multi-modal Issues, Systems and Applications
Mission Statement
Our mission is to develop the technology necessary to support fully-integrated multi-media database systems including many different types of data, such as text, images, signals, presentations, videos, audio clips, dynamic data, scientific data, and software.
The new technology will include
New data representations and storage schemes
To encode data in ways that facilitate retrieval and other operations
Full integration (fusion) of multimedia information
Combining different types of information from different sources
New types of queries, new definitions of "matching"
New ways of organizing, indexing, and retrieving data
New multimodal interfaces that allow users to interact with diverse data
Constructing multimedia documents
Data acquisition tools and systems
Mining of multimedia data
Research Directions: Most Important
What are the most important problems to solve? Short term? Long term?
Features
Efficient algorithms and tools for manipulation and retrieval of compressed data.
New features and new extraction techniques for complex or nontraditional data types such as volumetric images, dynamic sequences, hyperspectral images.
Feature extraction in transformed domains.
Pose-invariant features.
Data analysis and Modeling
Content analysis of multimedia data
- Feature extraction
- Pattern recognition
- Spatio-temporal analysis
- Multi-level analysis
(Collaborative work with related disciplines) - Content modeling
Search and indexing
For multimedia documents
With multimedia queries*
For high-dimensional data
For nontraditional data
By similarity measures at multiple levels
* Include text (NLP), images and signals (IP, SP), videos (VP), audio (SP), scientific data (visualization), source code.
Methods of evaluation
Benchmarks
Quality metrics
Query formulation
What kinds of queries
What kinds of user interfaces
Impact
Use of multimedia data in:
Interfaces for the handicapped
Improving k-12 education (the digital earth, the solar system, art…)
Distance learning
Electronic commerce
Multimedia patient records (he electronic patient)
Needs:
What research facilities are needed? (i.e. repositories, computing facilities, major infrastructure)
Standard test databases, ground truth
Images, outdoor scenes, video clips,indoor scenes, medical, remote sensing, faces.
Research Directions: Midterm Plans
Nontraditional Data types
3D (volumetric data)
VR data
Animations
Interoperability
Establish a common language for content-based search.
Self-creating, self-maintaining databases
Accepts data of different multi-media types
Creates structures based on metaknowledge
Learns from sequences of user interactions
Research Directions: Not Our Emphasis
System concerns
Efficient storage of multimedia data
Optimization of multimedia queries
Time/quality tradeoffs for rapid access | 
