Moving foreground objects can be classified into relevant categories. Statistics about the appearance, shape, and motion of moving objects can be used to quickly distinguish people, vehicles, carts, animals, doors opening/closing, trees moving in the breeze, etc. Our system classifies objects into vehicles, individuals, and groups of people based on shape features (compactness and ellipse parameters), recurrent motion measurements, speed and direction of motion (see following Figure). From a small set of training examples, we are able to classify objects in similar footage using a Fisher linear discriminant followed by temporal consistency.
Figure 1. Left: Result of classification system on a frame from video data provided by the IEEE Workshop on the Performance of Tracking in Surveillance 2001. Right: (left to right) The mask output from the background subtraction, the ellipse fitting and contour and the recurrent motion image used in the object classification for the person (top) and the car (bottom). Notice how the lower third of the person recurrent motion indicates the leg motion due to walking.
We are working to incorporate three significant algorithmic enhancements by including the following information: (i) probabilistic information regarding the likelihood that a pixel is associated with a real moving object or noise, (ii) the likelihood that the pixel is occluded and in which depth layer; (iii) reliable segment of object groups, object shadows; (iv) explicit motion (e.g., walking, driving) pattern detection; and (v) inferring true scale of objects and their motions from calibration information. We believe such a system will supply important quantitative bounds to object classification and allow us to achieve higher accuracy for a wide range of circumstances while maintaining real-time performance.
Watch this demo (video 4MB MPEG1). “P” = person, “C” = car, "M" = multiple people, numerical values on the image represent the image velocity.
