Performance Evaluation of Surveillance systems

Effectively evaluating the performance of moving object detection and tracking algorithms is in an important step towards attaining robust digital video surveillance systems with sufficient accuracy for practical applications. As systems become more complex and achieve greater robustness, the ability to quantitatively assess performance is needed in order to continuously improve performance. To this end, we are providing video sequences and ground truth annotations for performance evaluation.


Ground Truth sequences have extension .pvann and are xml files.

 

An explanation of the ground truth procedure can be found in the paper [1] .

Performance metrics and an evaluation of the IBM Smart Surveillance System can also be found in this publication.

 

1. Outdoor Sequences ( 4 videos from PET2001 with ground truth )

    PetsD1TeC1.avi
    PetsD1TeC1.pvann

    PetsD1TeC2.avi
    PetsD1TeC2.pvann

    PetsD2TeC1.avi
    PetsD2TeC1.pvann

    PetsD2TeC2.avi
    PetsD2TeC2.pvann

2. Indoor Sequences  (11 videos with ground truth)

    IndoorGTTest1.avi
    IndoorGTTest1.pvann

    IndoorGTTest2.avi
    IndoorGTTest2.pvann

    ThreePerson_Circles_Comp_0_Quad0.avi
    ThreePerson_Circles_Comp_0_Quad0.pvann

    ThreePerson_Circles_Comp_0_Quad2.avi
    ThreePerson_Circles_Comp_0_Quad2.pvann

    ThreePerson_Circles_Comp_0_Quad3.avi
    ThreePerson_Circles_Comp_0_Quad3.pvann

    ThreePerson_Together_Split_Comp_0_Quad0.avi
    ThreePerson_Together_Split_Comp_0_Quad0.pvann

    ThreePerson_Together_Split_Comp_0_Quad2.avi
    ThreePerson_Together_Split_Comp_0_Quad2.pvann

    ThreePerson_Together_Split_Comp_0_Quad3.avi
    ThreePerson_Together_Split_Comp_0_Quad3.pvann

    TwoPerson_Line_Circle_Comp_0_Quad0.avi
    TwoPerson_Line_Circle_Comp_0_Quad0.pvann

    TwoPerson_Line_Circle_Comp_0_Quad2.avi
    TwoPerson_Line_Circle_Comp_0_Quad2.pvann

    TwoPerson_Line_Circle_Comp_0_Quad3.avi
    TwoPerson_Line_Circle_Comp_0_Quad3.pvann

[1] Performance Evaluation of Surveillance Systems Under Varying Conditions
Lisa M. Brown, Andrew W. Senior, Ying-li Tian, Jonathan Connell, Arun Hampapur, Chiao-fe Shu, Hans Merkl, and Max Lu
IEEE Int'l Workshop on Performance Evaluation of Tracking and Surveillance, Colorado
Jan., 2005 .  PDF

Other Research Areas: