Introduction to Robotics, Spring 2023

Course number: CSE 525

Meets: Monday/Friday 1-2:20pm

Location: Old CS 2120 (in-person)

Website: http://michaelryoo.com/course/cse525/

Instructor: Michael S. Ryoo

Email: mryoo "at" cs.stonybrook.edu

Office: NCS 231

Office hours: Friday 2:30-4pm

TA: Xiang Li xiangli8@cs.stonybrook.edu

TA office hours: Monday 2:30-4pm

Course description:

Building an intelligent robot operating robustly in everyday environment has been one of the ultimate objectives in artificial intelligence. In this course, the students will learn fundamental algorithms for robot learning and perception. The course will also focus on recent progress in deep reinforcement learning for robotics, providing the students an opportunity to learn about state-of-the-art robot learning approaches. This is a research-oriented course composed of a series of lectures by the instructor. Some basic deep representation learning concepts will occasionally be discussed, in order to provide the right background to the students.

The topics to cover includes:

Robot reinforcement learning

Dynamic programming
TD (Q-Learning)
Policy gradients and actor-Critic
Model-based RL and planning
Imitation learning
Transformers for robots
Large language models for robots

Vision for Robotics

Camera and stereo
Depth and ego-motion
Localization and mapping
Recognition in the wild

Textbooks:

There is no particular textbook and the course will rely heavily on the lecture slides and the state-of-the-art papers. For RL basics, Sutton and Barto’s book is highly recommended.

Prerequisites:

Interest in robot learning; basic programming skills (Python + PyTorch?); Machine Learning (e.g., neural networks) basics

Schedule:

Week	Topic	Slides
Week1	Introduction Supervised action learning	Intro BC
Week2	Reinforcement learning introduction RL - Dynamic programming	Basics DP
Week3	Monte Carlo	MC
Week4	TD - Q-learning Deep Q-learning	TD DQ
Week5	Deep Q-learning (cont’d) More TD	n-step TD
Week6	Policy Gradients	REINFORCE
Week7	Policy Gradients (cont’d) + Xiang’s data augmentation
Week8	Spring break
Week9	Planning and model-based RL	Planning Model-based
Week10	Visual representations (Decision) Transformers	CNN
Week11	Evolutionary Strategies	ES
Week12	Project proposal presentations Vision and Robotics Transformers	Transformers
Week13	Midterm exam (April 14) and summary
Week14	Ego-motion and 3D SLAM	Ego-motion SLAM
Week15	Object recognition Imitation learning	Objects IL
Week16	Guest lecture - Dmitry Kalashnikov at Google DeepMind Project presentations

Use your @stonybrook.edu account to access the slides.

Materials from the last year: http://michaelryoo.com/course/cse525/spring2022/

Course requirements and grading:

Programming assignments (30%): three programming assignments.

Final project (50%): research project with proposal and presentation.

Midterm exam (20%).

Acknowledgement:

This course has been inspired by the Deep Reinforcement Learning course by Sergey Levine (Berkeley), Reinforcement Learning course by Adam and Martha White (Alberta), Computer Vision course by Linda Shapiro (UW).