Recognition of Ongoing Complex Activities
In this project, we propose the first method to predict ongoing activities over a hierarchical label space. We approach this task as a sequence prediction problem in a recurrent neural network where we predict over a hierarchical label space of activities. Our model learns to realize accuracy-specificity trade-offs over time by starting with coarse labels and proceeding to more fine grained recognition as more evidence becomes available in order to meet a prescribed target ac- curacy. In order to study this task we have collected a large video dataset of complex activities with long duration. The activities are annotated in a hierarchical label space from coarse to fine.
![](/fileadmin/_processed_/b/8/csm_ongoingActivity-teaser_99a222a3ef.png)
![](/fileadmin/_processed_/c/f/csm_ongoingActivity-hierarchy_30092a4bc9.png)
Dataset
References
[1] Recognition of Ongoing Complex Activities by Sequence Prediction over a Hierarchical Label Space, W. Li and M. Fritz, IEEE Winter Conference on Applications of Computer Vision, (2016) to appear