Michael S. Ryoo

SUNY Empire Innovation Associate Professor
AI Institute; Department of Computer Science
Stony Brook University

Staff Research Scientist
Robotics at Google
Google Brain

Founder & CSO, EgoVid Inc.

mryoo-at-indiana.edu or mryoo-at-egovid.com

As of September 2019, I joined the Department of Computer Science (CS) at Stony Brook University as an associate professor. I also am with Google Brain's "Robotics at Google" as a research scientist. Previously, I was an assistant professor at Indiana University Bloomington, and was a staff researcher within the Robotics Section of the NASA's Jet Propulsion Laboratory (JPL). I received my Ph.D. from the University of Texas at Austin in 2008 and B.S. from Korea Advanced Institute of Science and Technology (KAIST) in 2004.

Recent News
2019/06: My research at Google Brain on neural architecture search for video understanding:
AssembleNet: Searching for Multi-Stream Neural Connectivity in Video Architectures
Evolving Space-Time Neural Architectures for Videos (to appear at ICCV 2019).
2019/06: I organized a Tutorial on Unifying Human Activity Understanding at CVPR 2019, together with Gunnar Sigurdsson.
2018/09: A new privacy-preserving activity recognition work at ECCV 2018: Learning to Anonymize Faces for Privacy Preserving Action Detection
We also did its real-time demo at the conference.
2018/05: New research on robot deep reinforcement learning with an environment model learning: Learning Real-World Robot Policies by Dreaming.
2018/03: A new paper on detecting multiple activities from continuous videos at CVPR 2018, obtaining the state-of-the-art performances:
Learning Latent Super-Events to Detect Multiple Activities in Videos. Its source code is now available at [github].
2017/09: Research on deep learning for robotics: Learning Robot Activities from First-Person Human Videos Using Convolutional Future Regression
The paper appeared at IROS 2017 [video]. It won the Best Paper Award at CVPR 2017 Workshop on Deep Learning for Robot Vision.
2017/02: Presented a paper on Privacy-Preserving Human Activity Recognition from Extreme Low Resolution at AAAI 2017.
2016/06: Organized the 4th workshop on Egocentric (First-Person) Vision at CVPR 2016 with Kris Kitani, Yong Jae Lee, and Yin Li.
2016/05: Won the Best Paper Award in Robot Vision from ICRA 2016.
2015/03: My robot-centric activity prediction paper was one of the two nominees for the Best Enabling Technology Award at HRI 2015.

Curriculum Vitae pdf

Publications [by topic] [by type] [by year]

List of selected recent publications

  • M. S. Ryoo, A. Piergiovanni, M. Tan, A. Angelova, "AssembleNet: Searching for Multi-Stream Neural Connectivity in Video Architectures", arXiv:1905.13209. [arXiv]
  • A. Piergiovanni, A. Angelova, and M. S. Ryoo, "Learning Differentiable Grammars for Continuous Data", arXiv:1902.00505. [arXiv]
  • A. Piergiovanni, A. Wu, and M. S. Ryoo, "Learning Real-World Robot Policies by Dreaming", IROS 2019. [arXiv] [project]
  • A. Piergiovanni, A. Angelova, A. Toshev, and M. S. Ryoo, "Evolving Space-Time Neural Architectures for Videos", ICCV 2019. [arXiv]
  • A. Piergiovanni and M. S. Ryoo, "Temporal Gaussian Mixture Layer for Videos", ICML 2019. [arXiv] [github_code]
  • A. Piergiovanni and M. S. Ryoo, "Representation Flow for Action Recognition", CVPR 2019. [arXiv] [github_code]
  • Z. Ren, Y. J. Lee, and M. S. Ryoo, "Learning to Anonymize Faces for Privacy Preserving Action Detection", ECCV 2018. [arXiv] [project]
  • A. Piergiovanni and M. S. Ryoo, "Learning Latent Super-Events to Detect Multiple Activities in Videos", CVPR 2018. [arXiv] [github_code]
  • M. S. Ryoo, K. Kim, and H. J. Yang, "Extreme Low Resolution Activity Recognition with Multi-Siamese Embedding Learning", AAAI 2018. [arXiv]
  • J. Lee and M. S. Ryoo, "Learning Robot Activities from First-Person Human Videos Using Convolutional Future Regression", IROS 2017. [arXiv] [video]
  • C. Fan, J. Lee, M. Xu, K. K. Singh, Y. J. Lee, D. J. Crandall, and M. S. Ryoo, "Identifying First-person Camera Wearers in Third-person Videos", CVPR 2017. [arXiv]
Google Scholar page: Google Scholar: Michael S. Ryoo


MLB-YouTube dataset: an activity recognition dataset with over 42 hours of 2017 MLB post-season baseball videos.
JPL-Interaction dataset: a robot-centric first-person video dataset.
DogCentric Activity dataset: a first-person video dataset taken with dogs.
UT-Interaction dataset: a dataset containing continuous/segmented videos of human-human interactions.

Lab members

AJ Piergiovanni (CS PhD student)
Alan Wu (Engineering PhD student)
Ziwei Zhao (CS PhD student)


B457/I400: Intro to Computer Vision (Spring 2018)
B659/I590: Vision for Intelligent Robotics (Fall 2017)

Updated 09/2019