Michael S. Ryoo: Publications by topic [by type][by year]

Robotics


Learning robot activities from human examples
  • J. Lee and M. S. Ryoo, "Learning Robot Activities from First-Person Human Videos Using Convolutional Future Regression", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September 2017. arXiv preprint video
  • T. Shu, X. Gao, M. S. Ryoo, and S.-C. Zhu, "Learning Social Affordance Grammar from Videos: Transferring Human Interactions to Human-Robot Interactions", IEEE International Conference on Robotics and Automation (ICRA), May 2017. arXiv preprint
  • T. Shu, M. S. Ryoo, and S.-C. Zhu, "Learning Social Affordance for Human-Robot Interaction", the 25th International Joint Conference on Artificial Intelligence (IJCAI), July 2016. arXiv preprint


Robot-centric video understanding
  • I. Gori, J. K. Aggarwal, L. Matthies, and M. S. Ryoo, "Multi-Type Activity Recognition from a Robot's Viewpoint", the 26th International Joint Conference on Artificial Intelligence (IJCAI), August 2017 (invited).
  • I. Gori, J. K. Aggarwal, L. Matthies, and M. S. Ryoo, "Multi-Type Activity Recognition in Robot-Centric Scenarios", IEEE Robotics and Automation Letters (RA-L), 1(1):593-600, February 2016. [ICRA 2016 presentation] arXiv preprint link
    [Best Vision Paper Award from ICRA 2016]
  • M. S. Ryoo, T. J. Fuchs, L. Xia, J. K. Aggarwal, and L. Matthies, "Robot-Centric Activity Prediction from First-Person Videos: What Will They Do to Me?", ACM/IEEE International Conference on Human-Robot Interaction (HRI), March 2015 (full paper). pdf dataset
    [Best Paper Award Nominee]
  • L. Xia, I. Gori, J. K. Aggarwal, and M. S. Ryoo, "Robot-Centric Activity Recognition from First-Person RGB-D Videos", IEEE Winter Conference on Applications of Computer Vision (WACV), January 2015. pdf


Human Activity Recognition


End-to-end learning of temporal structures in human activities using deep architectures
  • A. Piergiovanni+, C. Fan+, and M. S. Ryoo, "Learning Latent Sub-events in Activity Videos Using Temporal Attention Filters", the 31st AAAI Conference on Artificial Intelligence (AAAI), February 2017 (+indicates equal contribution). arXiv preprint [source_code]


Privacy-preserving activity recognition
  • M. S. Ryoo, B. Rothrock, C. Fleming, and H. J. Yang, "Privacy-Preserving Human Activity Recognition from Extreme Low Resolution", the 31st AAAI Conference on Artificial Intelligence (AAAI), February 2017. arXiv preprint


First-person activity recognition
  • C. Fan, J. Lee, M. Xu, K. K. Singh, Y. J. Lee, D. J. Crandall, and M. S. Ryoo, "Identifying First-person Camera Wearers in Third-person Videos", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017. arXiv preprint
  • M. S. Ryoo and L. Matthies, "First-Person Activity Recognition: Feature, Temporal Structure, and Prediction", International Journal of Computer Vision (IJCV), 119(3):307–328, 2016. link
  • M. S. Ryoo, B. Rothrock, and L. Matthies, "Pooled Motion Features for First-Person Videos", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015. arXiv preprint
  • Y. Iwashita, A. Takamine, R. Kurazume, and M. S. Ryoo, "First-Person Animal Activity Recognition from Egocentric Videos", International Conference on Pattern Recognition (ICPR), August 2014. pdf dataset
  • S. Mann, K. Kitani, Y. J. Lee, M. S. Ryoo, and A. Fathi, "An Introduction to the 3rd Workshop on Egocentric (First-Person) Vision", IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), June 2014.
  • M. S. Ryoo and L. Matthies, "First-Person Activity Recognition: What Are They Doing to Me?", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2013. pdf video dataset


Human activity prediction and forecast
  • C. Fan+, J. Lee+, and M. S. Ryoo, "Forecasting Hand and Object Locations in Future Frames", arXiv:1705.07328, May 2017 (+indicates equal contribution). arXiv preprint
  • M. S. Ryoo, "Human Activity Prediction: Early Recognition of Ongoing Activities from Streaming Videos", International Conference on Computer Vision (ICCV), Barcelona, Spain, November 2011. pdf results


Single-example activity recognition using active video composition
  • M. S. Ryoo, "Interactive Learning of Human Activities Using Active Video Composition", International Workshop on Stochastic Image Grammars (SIG), in Proceedings of International Conference on Computer Vision Workshops (ICCVW), Barcelona, Spain, November 2011.
  • M. S. Ryoo and W. Yu, "One Video is Sufficient? Human Activity Recognition Using Active Video Composition",IEEE Workshop on Applications of Computer Vision (WACV), January 2011. pdf example composed videos


Spatio-temporal relationship match
  • M. S. Ryoo and J. K Aggarwal, "Spatio-Temporal Relationship Match: Video Structure Comparison for Recognition of Complex Human Activities", International Conference on Computer Vision (ICCV), Kyoto, Japan, October 2009. pdf


group stealing                group arresting
click to play                                      click to play

Group activity recognition
  • M. S. Ryoo and J. K. Aggarwal, "Stochastic Representation and Recognition of High-level Group Activities", International Journal of Computer Vision (IJCV), 93(2):183-200, June 2011. pdf link
  • M. S. Ryoo and J. K Aggarwal, "Stochastic Representation and Recognition of High-level Group Activities: Describing Structural Uncertainties in Human Activities", 1st International Workshop on Stochastic Image Grammars (SIG), in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Miami, FL, June 2009 (invited). extended abstract.
  • M. S. Ryoo and J. K. Aggarwal, "Recognition of High-level Group Activities Based on Activities of Individual Members", Proceedings of IEEE Workshop on Motion and Video Computing (WMVC), Copper Mountain, CO, January 2008. pdf
  • M. S. Ryoo, "Semantic Representation and Recognition of Human Activities", Ph. D. Thesis, Track of Computer Engineering, Department of ECE, The University of Texas at Austin, August 2008.
    [Outstanding Dissertation Award Nominee]


Human-object interaction recognition
  • M. S. Ryoo and J. K. Aggarwal, "Hierarchical Recognition of Human Activities Interacting with Objects", 2nd International Workshop on Semantic Learning Applications in Multimedia (SLAM), in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Minneapolis, MN, June 2007. pdf


Human-human interaction recognition
  • M. S. Ryoo, J. Joung, S. Choi, and W. Yu, "Incremental Learning of Novel Activity Categories from Videos", 16th International Conference on Virtual Systems and Multimedia (VSMM), Seoul, Korea, October 2010 (invited). pdf
  • M. S. Ryoo and J. K Aggarwal, "Semantic Representation and Recognition of Continued and Recursive Human Activities", International Journal of Computer Vision (IJCV), 82(1):1-24, April 2009. pdf link
  • M. S. Ryoo and J. K Aggarwal, "Human Activities: Handling Uncertainties Using Fuzzy Time Intervals", Proceedings of 19th International Conference on Pattern Recognition (ICPR), Tampa, FL, December 2008. pdf
  • M. S. Ryoo and J. K. Aggarwal, "Semantic Understanding of Continued and Recursive Human Activities", Proceedings of 18th International Conference on Pattern Recognition (ICPR), Vol. 1, pp. 379~382, Hong Kong, August 2006. pdf
  • M. S. Ryoo and J. K. Aggarwal, "Recognition of Composite Human Activities through Context-Free Grammar based Representation", Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 2, pp. 1709-1719, New York, NY, June 2006. pdf
  • M. S. Ryoo, "Semantic Understanding of Continued and Recursive Activities using Context-Free Grammar", M. S. Thesis, Track of Computer Engineering, Department of ECE, The University of Texas at Austin, August 2006.
    [Outstanding Thesis Award Nominee]


Human Detection and Tracking


Human recognition from aerial videos
  • Y. Iwashita+, M. S. Ryoo+, T. J. Fuchs, and C. Padgett, "Recognizing Humans in Motion: Trajectory-based Aerial Video Analysis", the 24th British Machine Vision Conference (BMVC), 2013 (+indicates equal contribution). pdf video


Observe-and-explain tracking
  • M. S. Ryoo and J. K. Aggarwal, "Observe-and-Explain: A New Approach for Multiple Hypotheses Tracking of Humans and Objects", IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, AK, June 2008. pdf i-Lids_example_video CAVIAR_example_video


Computer Vision Applications


Personal driving diary: Life-logging of driving activities
  • M. S. Ryoo, S. Choi+, J. H. Joung+, J.-Y. Lee+, W. Yu, "Personal Driving Diary: Automated Recognition of Driving Events from First-Person Videos", Computer Vision and Image Understanding (CVIU), 117(10): 1299-1312, October 2013 (+indicates equal contribution). pdf link
  • J. H. Joung, M. S. Ryoo, S. Choi, and S. R. Kim, "Reliable Object Detection and Segmentation Using Inpainting", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Algarve, Portugal, October 2012. pdf
  • J. H. Joung, M. S. Ryoo, S. Choi, W. Yu, and H. Chae, "Background-aware Pedestrian/Vehicle Detection System for Driving Environments", IEEE Conference on Intelligent Transportation Systems (ITSC), Washington, D.C., October 2011. pdf
  • M. S. Ryoo, J. Lee, J. Joung, S. Choi, and W. Yu, "Personal Driving Diary: Constructing a Video Archive of Everyday Driving Events", IEEE Workshop on Applications of Computer Vision (WACV), January 2011. pdf video




Human-computer interaction (HCI): Intelligent workspace
  • M. S. Ryoo, K. Grauman, and J. K. Aggarwal, "A Task-Driven Intelligent Workspace System to Provide Guidance Feedback", Computer Vision and Image Understanding (CVIU), 114(5):520-534, May 2010. link
  • M. S. Ryoo and J. K. Aggarwal, "Robust Human-Computer Interaction System Guiding a User by Providing Feedback", Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, January 2007. pdf




Human-vehicle interaction recognition
  • M. S. Ryoo+, J. T. Lee+, and J. K Aggarwal, "Video Scene Analysis of Interactions between Humans and Vehicles Using Event Context", ACM International Conference on Image and Video Retrieval (CIVR), Xian, China, July 2010 (invited) (+indicates equal contribution). pdf
  • J. T. Lee, M. S. Ryoo, and J. K Aggarwal, "View Independent Recognition of Human-vehicle Interactions using 3-D Models", IEEE Workshop on Motion and Video Computing (WMVC), Snowbird, Utah, December 2009. pdf




Abandoned baggage detection
  • M. Bhargava, C.-C. Chen, M. S. Ryoo, and J. K. Aggarwal, "Detection of Object Abandonment Using Temporal Logic," Machine Vision and Applications (MVA), 20(5):271-281, June 2009. link
  • M. Bhargava, C.-C. Chen, M. S. Ryoo, and J. K. Aggarwal, "Detection of Abandoned Objects in Crowded Environments", Proceedings of IEEE International Conference on Advanced Video and Signal based Surveillance (AVSS), London, UK, September 2007. pdf




Illegally parked car detection
  • J. T. Lee, M. S. Ryoo, M. Riley, and J. K. Aggarwal, "Real-Time Illegal Parking Detection in Outdoor Environments Using 1-D Transformation", IEEE Transactions on Circuits and Systems for Video Technology (T-CSVT), 19(7):1014-1024, July 2009. link
  • J. T. Lee, M. S. Ryoo, M. Riley, and J. K. Aggarwal, "Real-time Detection of Illegally Parked Vehicles using 1-D Transformation", Proceedings of IEEE International Conference on Advanced Video and Signal based Surveillance (AVSS), London, UK, September 2007. pdf


Human Activity Recognition - Others


Survey paper and tutorials
  • M. S. Ryoo, A. Hoogs, A. Basharat, and S. Oh, "Activity Recognition for Visual Surveillance", Tutorials of IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS), Beijing, China, September 2012.
  • J. K. Aggarwal and M. S. Ryoo, "Toward a Unified Framework of Motion Understanding", Image and Vision Computing, 30(8):465-466, August 2012. link
  • J. K. Aggarwal and M. S. Ryoo, "Human Activity Analysis: A Review", ACM Computing Surveys (CSUR), 43(3), April 2011. pdf link
  • J. K. Aggarwal, M. S. Ryoo, and K. Kitani, "Frontiers of Human Activity Analysis", Tutorials of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, June 2011.
  • M. S. Ryoo, and K. Kitani, "Understanding Videos - Human Activity Analysis", Tutorials of 11th Pacific Rim International Conference on Artificial Intelligent (PRICAI), Daegu, Korea, August 2010.
  • M. S. Ryoo, "An Introduction to Description-Based Human Activity Recognition", Tutorials of 21st Workshop on Image Processing and Image Understanding (IPIU), Jeju, Korea, February 2009.


ICPR contest on human activity recognition 2010: SDHA 2010
  • M. S. Ryoo, C.-C. Chen, J. K. Aggarwal, and A. Roy-Chowdhury, "An Overview of Contest on Semantic Description of Human Activities (SDHA) 2010", International Conference on Pattern Recognition (ICPR) Contests, August 2010. pdf slides website


Other Topics


Robotics and affective computing
  • H. S. Yang, Y. Seo, M. S. Ryoo, and H. Jung, "Affective Communication System with Emotional Memories for Multimodal Interaction with Humanoids", Proceedings of the 11th international conference on virtual systems and multimedia (VSMM), October 2005.
  • M. S. Ryoo, Y. Seo, H. Jung, and H. S. Yang, "Affective Dialogue Communication System with Emotional Memories for Humanoid Robots", Proceedings of the First International Conference on Affective Computing and Intelligent Interaction (ACII), LNCS 3784, pp. 819-827, October 2005. pdf
  • D. Pardoe, M. Ryoo, and R. Miikkulainen, "Evolving Neural Network Ensembles for Control Problems", Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), June 2005. link
  • H. Jung, Y. Seo, M. S. Ryoo, and H. S. Yang, "Affective Communication System with Multimodality for Humanoid Robot AMI", IEEE-RAS/RSJ International Conference on Humanoid Robots (Humanoids), November 2004. link
  • M. S. Ryoo, "Affective Dialogue Communication System with Emotional Memories for Humanoid Robots", B. S. Thesis, Division of Computer Science, Department of EECS, KAIST, June 2004.
Home