compressive sensing of dynamical scenes


                  
Compressive sensing (CS) is a relatively new approach for the acquisition and recovery of sparse signals and images that enables sampling rates significantly below the classical Nyquist rate. Despite significant progress in the theory and methods of CS, little headway has been made in compressive video acquisition and recovery. Video CS is complicated by the ephemeral nature of dynamic events, which makes direct extensions of standard CS imaging architectures and signal models infeasible. We develop a new framework for video CS for dynamic textured scenes that models the evolution of the scene as a linear dynamical system (LDS). This reduces the video recovery problem to first estimating the model parameters of the LDS from compressive measurements, from which the image frames are then reconstructed. We exploit the low-dimensional dynamic parameters (the state sequence) and high-dimensional static parameters (the observation matrix) of the LDS to devise a novel compressive measurement strategy that measures only the dynamic part of the scene at each instant and accumulates measurements over time to estimate the static parameters. This enables us to lower the compressive measurement rate considerably. We validate our approach with a range of experiments involving both video recovery, classification and sensing hyper-spectral data; all of this highlights the effectiveness of the approach.

papers

Compressive acquisition of linear dynamical systems
Aswin C. Sankaranarayanan, Pavan Turaga, Rama Chellappa, and Richard G. Baraniuk
SIAM J. Imaging Sciences 2013 (to appear)
Compressive acquisition of dynamic scenes
Aswin C. Sankaranarayanan, Pavan Turaga, Richard G. Baraniuk and Rama Chellappa
ECCV 2010

talk Slides

results

video recovery results on the DynTex database
each result figure/animation has three panels which correspond to ground truth (left), video recovery at 20x compression (center), and video recovery at 50x compression (right), respectively.
See SIIMS paper for quantitative performance scores

(avi)

(avi)

(avi)

(avi)

(avi)

(avi)

(avi)

(avi)

(avi)

code and dataset


[Demo Code]