The ESPS team has built various reference applications to highlight various issues unique to stream processing applications.
Dense Information Grinding (DIG) : Audio and video are typically compressed for storage and transmission efficiency, and audio and video analysis algorithms are typically bandwidth and processing intensive. Traditional analysis systems operate by having a concept detector load a video or video segment, decompress and decode individual video frames, and then perform analysis on the uncompressed signal. The decompression, decode, and analysis are all data and compute-intensive operations. The DIG project focuses on adaptive and scalable audio/video feature extraction and classification operators for stream-processing. For example, instead of decoding and decompressing video and performing analysis on the uncompressed video - algorithms are being developed to perform feature extraction and classification on compressed audio and video. In some cases, the accuracy of algorithms operating in the compressed domain is less than those operating on raw audio/video, however the complexity (i.e., resource requirements) are also reduced (since the content need not be decompressed or decoded). Additionally, these algorithms are being designed to be adaptive in terms of complexity. For example, using support vector machine technology resource requirements can be reduced by reducing the number of support vectors or extracted features for a given concept detector. This reduction also results in a decrease in classification accuracy. In addition to developing these adaptive and scalable media feature extraction algorithms, the DIG project seeks ways of managing the complexity-accuracy tradeoff in a manner consistent with the objectives of the stream-processing application. The project also explores operator compostion rules for this domain of applications, to enable resource-efficient multi-query optimization.
Progressive Stream Assimilation : This project explores the need for two important features in stream-processing systems, in which the profferred load is greater than the resources available to process it -- ( i ) progressive information extraction and (ii) data and control feedback. In such systems, there is a vast amount of sensor data coming in from various sources, with information of interest buried in it. In order to effectively use the available resources, the stream-processing application extracts information progressively, discarding irrelevant data at each stage. In this project, we explore the use of feedback mechanisms to send information to earlier processing stages to enable them to improve the confidence of the information extracted and also cull away irrelevant information earlier in the processing pipeline, thereby utilizing resources more effectively.
Who's Talking to Whom (WTTW) : This work studies the feasibility of revealing pairs of anonymous and encrypted conversing parties (caller/callee pair of streams). By exploiting the aperiodic inter-departure time of VoIP packets, we can reduce each VoIP stream to a binary time-series. We first define a simple yet intuitive metric to gauge the correlation between two VoIP binary streams. We then propose an effective technique that progressively pairs conversing parties with high accuracy and in a limited amount of time. Our metric and method are justified analytically and validated by experiments on a very large standard corpus of conversational speech. We obtain impressively high pairing accuracy that reaches 97% after 5 minutes of voice conversations.
Resource-Adaptive Semantic Sports Image Scene Classification : This work studies the performance of a distributed image stream classification application under dynamically varying resource constraints. We build a topology of state-of-the-art Support Vector Machine (SVM) based image classifiers for semantic labeling of sports images. We also centralized develop optimization algorithms to dynamically reconfigure the operating points of individual classifier processing elements based on the underlying resource constraints, in order to maximize the end to end application performance metric. The designed performance metric captures the accuracy of classification and penalizes any misclassification (false alarm or missed detection) based on application specified costs. We show that with optimal reconfiguration, we can significantly outperform load-shedding based stream mining schemes.
Related Publications :