As shown in above figure, the background subtraction module combines evidence from differences in color (top left), texture (middle left), and motion (bottom left). The use of multiple modalities improves the detection of objects in cluttered environments. The resulting saliency map (top right) is smoothed using morphology-like operators and then small holes and blobs are eliminated to generate a clean foreground mask (middle right).
The background subtraction module has a number of mechanisms to handle changing ambient conditions and scene composition. First, it continually updates its overall RGB channel noise parameters to compensate for changing light levels. Second, it estimates and corrects for AGC and AWB shifts induced by the camera. Finally, it maintains a map of high activity regions (lower right) and slowly updates its background model only in areas deemed as relatively quiescent.
The system also has several mechanisms for automatically updating the background model over time. These are demonstrated in the video (2.6MB). The upper left quadrant of the video shows the input frames, while the diagonally opposite (lower right) quadrant shows the current background model. The upper right quadrant shows the salience of each pixel relative to this background model (using color, motion, and texture), and the lower left quadrant shows the regions extracted after the salience has been thresholded and geometrically smoothed (using pseudo-morphology and connected components).
At the beginning of the video the background model has not been initialized and so it is completely back. The only contribution to the salience is therefore motion energy. After a number of frames, the system determines that a large portion of the scene is stable and adds these portions to the background model. The exception is where the two women have just entered. Once the background has been established, it can be seen that the salience values for the two women are much more complete. Eventually, the system determines that the lower left portion of the image is also now stable, so it adds it to the model also.
After the women enter the office they close the door behind them. This yields a large foreground object comprised of the door and the previous light spill that had emanated from the opening. The system continually monitors all foreground objects for internal motion. Eventually, if they are determined to be quiescent, objects are deliberately removed and pushed into the background model (after signalling the higher-level system). This is what happens to the door. Finally, toward the end of the video it can be seen how the background is incrementally updated over time, thus gradually erasing the remaining crescent of light near the base of the door.
For additional demos you can go to 2D Tracking or 3D Multi-Person Tracking. All the tracking work is based on background subtraction results.