Research

This page contains a brief description of the projects that I have been involved in.

• Sentient: People monitoring
• Leaving Flatland: 3D scene mapping
• Real-time path detection for LAGR
• Markerless Motion capture
• Marker-based motion capture
• Human Identification based on Gait Analysis

Sentient: Realtime Multi stereo sensor based tracking

People: Chris Connolly, Aravind Sundaresan, Bob Bolles

Leaving Flatland: 3D scene mapping

People: Radu Rusu, Aravind Sundaresan, Benoit Morisset, Motilal Agarwal, Kris Hauser, Jean-Claude Latombe, Michael Beetz

"Leaving Flatland" is an exploratory project that attempts to surmount the challenges of closing the loop between autonomous perception and action on challenging terrain. The proposed system includes comprehensive localization, mapping, path planning and visualization techniques for a mobile robot to operate autonomously in complex 3D indoor and outdoor environments. In doing so we integrate robust Visual Odometry localization techniques with real-time 3D mapping methods from stereo data to obtain consistent global models annotated with semantic labels. These models are used by a multi-region motion planner which adapts existing 2D planning techniques to operate in 3D terrain. All the system components are evaluated on a variety of real world data sets, and their computational performance is shown to be favorable for high-speed autonomous navigation.

Real-time path detection for LAGR

People: Kurt Konolige, Motilal Agarwal, Rufus Blas, Aravind Sundaresan, Bob Bolles

This project outlines an approach for learning the brightness, color and texture of scene objects with the final goal for identifying a path in real-time for the LAGR (Learning Adapted for Ground Robots) project. At the lowest level, high dimensional vectors in the form of textons are used to represent local image measurements. An efficient two-step k-means implementation is then used to cluster image regions into separate groups. This is done by clustering the textons into a number of distinctive texton primitives for a given scene. Histograms are then constructed of these texton primitives in image neighborhoods and re-clustered into image regions with similar histograms. The Earth Movers Distance is used to merge clusters in order to minimize over-segmentation. Integral images are used allowing for construction of arbitrarily large histograms at constant time. Recent advances in accelerating k-means allow for a real-time implementation. Results are shown for a robotic application of outdoor path following where a context-aware approach allows for automatic learning of the visual cues for a path using 3D spatial information.

Towards markerless motion capture

People: Aravind Sundaresan, James Sherman, Rama Chellappa

Motion capture methods traditionally use active or passive markers, and there exist applications where it is desirable to do away with markers for a variety of reasons not the least of which is their invasive nature. In particular, biomechanical and clinical applications, where marker-based motion capture is the state-of-the-art technique would benefit greatly from such a system. We have published some work on image-based 3-D tracking as well as pose estimation and human body model estimation from voxels.

Since much of my work has revolved around multi-camera capture I have a fair bit of experience with different multiple capture systems. I have (with James Sherman) designed and built the Hydra project, a portable and scalable multi-camera capture system for human motion analysis.

Model driven human body model estimation in Laplacian Eigenspace

This project has two components. The first is model-driven segmentation in Laplacian Eigenspace. The input abstraction layer is voxels and the neighbourhood relationship of the voxels is used to compute the Laplacian of the adjacency graph. The nodes are then mapped to 6-D Laplacian eigenspace using the eigenvectors corresponding to the smallest non-zero eigenvalues of the Laplacian matrix. We show that this transformation maps segments whose lengths are greater than their thicknesses to 1-D curves in eigenspace. We can then fit splines to these 1-D curves and segment them at their joints. The two images on the left correspond to the 6-D eigenspace and we have segmented one segment by fitting a spline. This work was awarded the best student paper in the Computer Vision track at the biennial International Conference on Pattern Recognition, 2006.

The second part of the project is to acquire a set of key frames where the voxels have been segmented and registered using a prior model and a probabilistic registration method. This set of key frames can be used to estimate the human body model in two steps: estimate a skeleton based human body model and joint locations using human body statistics and computed skeleton, and then fit a super-quadric model using the segmented voxels. The images (from left to right) denote the voxels (unsegmented), voxels (segmented), computed skeleton curve and estimated super-quadric skeleton model. Five frames were used to estimate the model.

Articulated 3-D Tracking using shape and motion cues

We perform 3D tracking of articulated subjects using both motion and shape cues information in the images obtained from multiple cameras. The motion information used is the computed pixel displacement. The shape information used is both the silhouette information as well as the "motion residue". These cues are complementary and when fused in the tracking algorithm prevent drifting (through use of shape features) and avoid non-optimal local minima (through prediction of motion using optical flow). The two images on the right illustrate the super-quadric model superimposed on the image for two views.

Real-time marker based motion capture to control robots

People: Allen Yang, Aravind Sundaresan, James Davis, Hector Gonzalez-Banos

The project was with Allen Yang, James Davis, Hector Gonzalez-Banos and Victor Ng-Thow-Hing at Honda Research Institute. The objective is to retarget motion from a subject wearing markers to different robots such as Asimo. Images were obtained from eight cameras attached to two servers. The server can be controlled over network to capture, calibrate, obtain 2D marker locations and 3D marker locations. The markers are located in real time and their position in space found. The pose is estimated from the markers and the motion is retargetted to the robot. The motion retargetting was done by Allen and Hector.

Human Identification at a Distance (HID)

People: Amit Kale, Aravind Sundaresan, Naresh Cuntoor, Rama Chellappa, Amit Roy-Chowdhury,Volker Kruger, A. Rajagoplan.

The objective is to obtain methods of representing and recognizing humans in video sequences. We use 2-D binary silhouettes in an HMM framework for modelling gait and human shape. This simple approach enables us to analyse gait and gain an understanding of the problem. We aim to ultimately build 3-D models for human motion. Silhouettes are obtained from the sequence and are used to build an exemplar image-based human shape and gait model using the Baum-Welch algorithm. This HMM model can be used to obtain the identity of an unknown subject by maximising the posterior probability of the model given the sequence. The UMD data is available at HID UMD Database page.