Research

My recent projects are:

• RingCam
  

  Today's video conferencing systems have had limited success. While this is due to many factors, we think they at least have the following technical deficiencies: poor camera viewpoints, insufficient resolution, and inaccurate speaker detection, among others. The RingCam system is designed to overcome some of these problems. See our paper here . Note that an important piece of information that any teleconferencing system wants to know is how many people are in the meeting, their position, and who is talking. These are exactly the problems I am attaking now. Specifically, I am interested in real-time multi-person tracking, sound source localiaztion, sensor fusion and speaker timeline clustering.

• Real-time multi-person tracking using particle filters
  

  Tracking multiple people reliably in real time in a meeting is not easy. For example, the tracker needs to be initialized automatically, lighting condition can change, and people can occlude one another. We tackle this problem by using a statistical tool: particle filters (PF). For linear Guassian systems, we all know the elegant solution is Kalman filter. Unfortunately, in real life, we have to handle non-linear non-Guassian systems. PF is such a tool. It uses a set of weighted particles (samples) to approximate the posterior probability. We have advanced this technique at various fronts: Demo Better Proposal Distributions: Object Tracking Using Unscented Particle Filter , Proc. of IEEE CVPR 2001, pp. II-786 to 793, Kauai, Hawaii, December 11-13, 2001. JPDAF Based HMM for Real-Time Contour Tracking , Proc. of IEEE CVPR 2001, pp. pp.I-543 to 550, Kauai, Hawaii, December 11-13, 2001. Mode-based Multi-Hypothesis Head Tracking Using Parametric Contours , Proc. of IEEE Automatic face and gesture recognition 2002, Washington DC, May 20-21.

• Real-time 360 sound source localization (SSL)
  

  Knowing where people are is one thing, knowing who is currently talking is another. The goal of SSL is to detect where the sound source is coming from with respect to a microphone array. One of the most successful SSL techniques is based on time delay estimation. If two microphones are located at A and B, and the sound source is at C, the sound source will reach A and B at slightly different time. But based on this slight difference (time dealy), we can figure out the sound location. I am preparing a draft of the full technique. But if you cannot wait, take a look at Section 5 of this paper first.

• Timeline clustering for better meeting browsing
  
  For archived meetings/lectures, a very important value-added feature is participant clustering. If after the meeting, a timeline map can be generated based on the different participants, it will allow the end user to jump between interesting points, listen to a particular participant, and understand the dynamics of the meeting. The input to the system is the output from SSL, and the output from the system is the timeline clusters. There are two components in this system: pre-filtering and clustering. During pre-filtering, the noisy SSL output will be filtered and outliers thrown away. During clustering, K-mean's clustering is used during the first a few iterations to boot strap. Mixture of Gaussian is then used to refine the result.

• iCam: a fully automated lecture room for recording and broadcasting
  

  Given rapid improvements in storage devices, network infrastructure and streaming-media technologies, a large number of corporations and universities are recording lectures and making them available online for anytime, anywhere access. However, producing high-quality lecture videos is still labor intensive and expensive. Fortunately, recent technology advances are making it feasible to build automated camera management systems to capture lectures. In this paper we report our design of such a system, including system configuration, audio-visual tracking techniques, software architecture, and user study. Motivated by different roles in a professional video production team, we have developed a multi-cinematographer single-director camera management system. The system performs lecturer tracking, audience tracking, and video editing all fully automatically, and offers quality close to that of human-operated systems. See paper .

• OmniCam
  

  For teleconferencing and meeting recording, we are not limited by the RingCam. We can use a parabolic mirror coupled with high-res camera sensor: Omnicam.

• Automatically extracting highlights from a baseball game
  

  Tired of watching a three-hour baseball game, but do not want to miss any highlights? This project is the answer. We focus on detecting highlights using audio-track features alone without relying on expensive-to-compute video-track features. We use a combination of generic sports features and baseball-specific features to obtain our results, but believe that many other sports offer the same opportunity and that the techniques presented here will apply to those sports. We present details on relative performance of various learning algorithms, and a probabilistic framework for combining multiple sources of information. We present results comparing output of our algorithms against human-selected highlights for a diverse collection of baseball games with very encouraging results. See paper .

• Content-based Texture Image Retrieval with Relevance Feedback
  

  This is a very nice generalization of the relevance feedback techniques from the text-based document retrieval domain to the content-based image retrieval domain. For detailed description, please see related publications.

• An Integrated Relevance Feedback Architecture for Content-based Image Retrieval
  

  But ..., image is more complicated than text. That is why there is a old saying "An image worth a thousand words". The major complexity comes from human's perception subjectivity of image content. So, we need to come up with a better model to deal with this. My old team at UIUC is one of the earliest teams in the nation having looked into such a situation. This system is an effective interactive image retrieval system which achieves the relevance feedback at all the feature level, representation level, and vector level. User's initial query can be dynamically refined such that the new query is a better approximation to the user's information need. For more complete description, please see the related publications .

• Constructing Table-of-Contents (ToC) for Videos
  Raw video is a long and unstructured data stream, which is difficult to access. This project explores the automatic extraction of video structures from both the physical shots and the semantic scenes to assist user's access to the video.

• Browsing and Retriving Video Content in a Unified Framework
  How does a reader efficiently access a 1000-page book's content? Without reading the whole book, he will probably first go to the book's Table-of-Contents (ToC), finding which chapters or sections suit his need. If he has specific questions (queries) in mind, such as finding a terminology or a key word, he will go to the index page and find the corresponding book sessions containing that question. In short, book's ToC helps a reader browse and book's index helps a reader retrieve. The former is useful when the reader does not have any specific question in mind and will make his information need more specific and concrete via browsing the ToC. The latter is useful when the reader has a specific information requirement. Both aspects are equally important in helping users access the book's content. For current videos, unfortunately, we lack both the ToC and the index. Techniques are needed for constructing ToC and index to facilitate the video access.

  What is even more important in video domain is that the ToC and index should be inter-related. For a continuous long medium type like video, such ``back and forth'' mechanism between browsing and retrieval is crucial. The video library users may have to browse the video first before they know what to retrieve. On the other hand, after retrieving some video objects, it will guide the users to browse the video in the correct direction.
  For more detailed description, please see related publications .

• Efficient Multi-Dimentional Indexing
  As multimedia databases are getting bigger and bigger, the speed of retrieval is becoming a bottle neck. Exsiting indexing techniques, such as R-Tree, K-db Tree, etc. do not scale to dimentionality higher than 20. However, it is normal that a feature vector in a multimedia database can be of dimention more than 100. We are currently exploring dimension reduction and clustering techniques to achieve the multi-dimentional indexing goal.

• Multimedia Analysis and Retrieval System (MARS)
  Back in the old days at UIUC, we started our MARS project as part of our Digital Library project in 1994. It is a Java-written web-based system supportting content-based image retrieval based on color, texture, shape, and any Boolean combinations of them. The noval part is the integration of DBMS techniques (query processing), IR techniques (boolean retrieval model), and Image Processing techniques (image features).

• Image Segmentation and Shape Coding
  This project explores the segmentation of image regions using color, texture, and spatial information. The segmented regions are then coded with our proposed modified Fourier Descriptor, which is invariant to translation, scaling, and rotation. For a more detailed description, please see related publications .

• Electronic Commerce
  In this project, we study the advantages of on-line home shopping in terms of greater product customization, faster access and lower cost. As an example, we propose a framework for integrated design, advertisement and retailing of garment over the Internet. This on-line shopping framework would make use of the latest research in computer graphics, image processing, computer vision and artificial intelligence. We describe these technologies, and explain how they can be used to build a visually attractive and easy to use interface to an intelligent integrated system that fulfils most of the functions of the traditional production chain, while allowing for mass-customization. For a more detailed description, please see related publications .

• Efficient Representation for Image/Video Storage and Transmission
  This project studies the problem of representing images within a multimedia Database Management System (DBMS), in order to support fast retrieval operations without compromising storage efficiency. To achieve this goal, we propose new image coding techniques which combine a wavelet representation, embedded coding of the wavelet coefficients, and segmentation of image-domain regions in the wavelet domain. A bitstream is generated in which each image region is encoded independently of other regions, without having to explicitly store information describing the regions. The proposed algorithms achieve coding performance which compares favorably, both perceptually and objectively, to that achieved using state-of-the-art image/video coding techniques while additionally providing region-based support. For a more detailed description, please see related publications .

• MPEG-7
  Observing the increasing availability of digital audiovisual content, MPEG recently started a new work item, MPEG-7, formally called Multimedia Content Description Interface . The objective is to provide an interoperable solution to extend the capabilities of today's proprietary solutions in identifying multimedia content. That is, MPEG-7 will specify a standard set of descriptors that can be used to describe various types of multimedia information.
  I have 4 MPEG-7 contributions and some papers in this topic, please have a look .

• More ......
  I am also interested in many internet related techniques, including Multimedia Web Search Engines, Digital Watermarking, Multimedia OS for the Web, etc. More contents will come soon.

Home | Next

Copyright © 1995-1999