http://www.kmed.cs.ucla.edu

• Award Number: IRI-9619345
• Duration: 08/15/1997 – 07/31/2000, 1st year in progress
• Title: Knowledge-based Image Retrieval with Spatial and Temporal Constructs

A knowledge-based approach to retrieve medical images by feature and content with spatial and temporal constructs is being developed. Selected objects of interest in a medical image (e.g. x-ray, MR image) are segmented, and contours are generated from these objects. Features (e.g. shape, size, texture) and content (e.g. spatial relationships among objects) are extracted and stored in a feature and content database. Knowledge about image features can be expressed as a hierarchical structure called a Type Abstraction Hierarchy (TAH). TAHs can be generated automatically by clustering algorithms based on feature values in the databases and hence are scalable to large collections of image features. Further, since TAHs are generated based on user classes and applications, they are context- and user-sensitive.

A knowledge-based semantic image model which provides a mechanism for accessing and processing spatial, evolutionary, and temporal queries is also being developed. A knowledge-based spatial temporal query language (KSTL) will be developed which supports operators for approximate matching via feature and content, conceptual terms, and temporal logic predicates. Further, a visual query language is also proposed that will accept point-click-and-drag iconic input on the screen that maps into KSTL. The proposed system will be implemented in a testbed to evaluate the functionality of the proposed tasks. The results from this research are applicable to other multimedia information systems as well.

Our research is focused on the retrieval of medical images by feature and content represented by spatial and temporal constructs. We have developed a comprehensive data model methodology for knowledge-based query processing. We plan to implement the iconic query language, Mquery, and a knowledge-based query processing technique in the Knowledge-based Medical Image Databases (KMeD) testbed.

One of the important aspects of our content-based image retrieval research is to perform automatic image segmentation. More specifically, we want to generate the contour of an object (e.g. tumor) in an image. Recently, we have collaborated with Professor Jesse Jin, an expert in the field of image segmentation, and jointly developed a segmentation technique for our applications. This enabled us to semi-automatically perform segmentation for a large volume of images, which is a good breakthrough for our research. Our KMeD research has also received good reviews by the press and was reported as a feature article in the ‘Cutting Edge Technology’ of the Business Section of the Los Angeles Times (July 1997).

Our Knowledge-based query processing technology is being transferred to the ongoing NIH Program Project Grant at the Radiology Department in the UCLA Medical School to be part of specific radiology workstations.

Wesley W. Chu, A. F. Cardenas, and R. K. Taira. KMeD: A Knowledge-based Multimedia Medical Distributed Database System. Information Systems, 20(2): 75-96, 1995.

John David N. Dionisio and A. F. Cardenas. Mquery: A Visual Query Language for Multimedia, Timeline, and Simulation Data. Journal of Visual Languages and Computing. Vol. 7, Academic Press, 1996, pp.377-401.

Chih-Cheng Hsu, W. W. Chu, and R. K. Taira. A Knowledge-based Approach for Retrieving Images by Content. IEEE Transactions on Knowledge and Data Engineering (TKDE), 8(4): 522-531, 1996.

John David N. Dionisio, A.F. Cardenas, R. B. Lufkin, A. DeSalles, K. L. Black, R. K. Taira, and W. W. Chu. A Multimedia Database System for Thermal Ablation Therapy of Brain Tumors. Journal of Digital Imaging. 10(1):21-26, February 1997.

Wesley W. Chu, C. C. Hsu, I. T. Leong, and R. K. Taira. Content-Based Image Retrieval Using Metadata and Relaxation Techniques. In Managing Multimedia Data, edited by A. Sheith and W. Clas, McGraw Hill, 1998.

Database systems, knowledge-based systems

H. K. Huang and R. K. Taira. Infrastructure Design of a Picture Archiving and Communication System. American Journal of Roentgenology, 158:742-749, 1992.

There are several content-based image retrieval projects (e.g. Agouris, Chang, Chu, Faloutsos, Ghafoor, and Yu) in the IDM. I think they can all be leveraged on each other’s research work.