Much of the current pervasive computing research concentrates on devices and the communication between them. However, an important aspect of pervasive devices is their interface with the physical world—particularly, how they acquire information about their users. One rich medium (and the dominant one through which people receive information) is vision. As such, we expect future pervasive computing environments to depend on vision for passive perception of people.

We are focusing on the privacy issues involved in such visual information gathering, both in pervasive computing environments and in video surveillance systems. The PeopleVision system takes an object-oriented approach to video. It understands the video stream, decomposing it into people, objects, and areas of interest. It then abstracts or selectively re-renders this information based on the intended user. Client processes receive abstract information about people in the environment according to issued requests. Access control lists that can grant privileged processes access to richer (and more intrusive) information verify these requests. For example, the list might grant a face recognition security system access to facial images but tell the air conditioning process only how many people are in each room.

Access control lists also govern the information delivered to security guards, supervisors, and ordinary users. Re-rendering delivers reconstructed video, which preserves some objects unchanged and blanks out other areas of the image or replaces the area with a computer-graphics rendering that preserves relevant information. However, it does not convey more privacy-sensitive details. During ordinary use, a guard may only view silhouettes of people in the surveillance area, hiding irrelevant but privacy-sensitive information such as race, gender, and appearance.

We have developed a “PrivacyCamera” – a single device combining camera and processor that implements, at 30 frames per second, the video-understanding algorithms. With this device we can ensure that the privacy-intruding video is never available or leaves the device only in encryptedform. All of the processed data leaving the device can also be encrypted ensuring maximum privacy protection for the people in the pervasive computing environment. Some demos are shown below. All demo videos are in MPEG1 format.

1. Original video (video 1.5MB)	2. Foreground masked (video 1.5MB)
3. Background masked (video 1.5MB)	4. Both masked (video 1.5MB)

We have designed a privacy-protecting console for surveillance video monitoring and also implemented the ideas in a PrivacyCam which is a "smart camera" (combined imaging and processing device) that outputs video with privacy-intrusive data removed: 

The PrivacyCam can be substituted for a standard video/surveillance camera and ensures that privacy-intrusive data never leaves the camera, while providing a video stream that is still appropriate for the video system's task. 

Other Research Areas:

• Robust Background Subtraction 
• Salient Motion Detection 
• Object Classification 
• 2D Tracking 
• 3D Multi-Person Tracking 
• Articulated Human Body Tracking 
• Active Head Tracking
• Coarse Head Pose Estimation 
• Position Independent Absolute Head Pose Estimation
• Face Cataloger
• Multi-scale Tracking & Index Browser
  
• Real Time Alerts
• Middleware for Large Scale Surveillance (MILS)
• Performance Evaluation of Surveillance Systems