Introduction to Robotics, Fall 2023

Course number: CSE 378

Meets: Tuesday/Thursday 1-2:20pm

Location: FREY HALL 211 (in-person)

Website: http://michaelryoo.com/course/cse378/

Instructor: Michael S. Ryoo

Email: mryoo "at" cs.stonybrook.edu

Office: NCS 231 or Google Meet (check lecture slides “TD”)

Office hours: Thursdays 2:30-3pm, 3:30-4pm

TA: Jinghuan Shang jishang@cs.stonybrook.edu

TA office hours: Tuesdays 11am-12:30pm, NCS 109

Course description:

Building an intelligent robot operating robustly in everyday environment has been one of the ultimate objectives in artificial intelligence. In this course, the students will learn fundamental algorithms for robot learning and perception, as well as basics to better understand them. The course will also focus on recent progress in deep reinforcement learning for robotics, providing the students an opportunity to learn about state-of-the-art robot learning approaches. The course is composed of a series of lectures by the instructor. Some basic deep representation learning concepts will occasionally be discussed, in order to provide the right background to the students.

The topics to cover includes:

Robot reinforcement learning

Dynamic programming
TD (Q-Learning)
Policy gradients
Model-based RL and planning
Transformers for robots
Large language models for robots

Vision for Robotics

Camera and stereo
Depth and ego-motion
Localization and mapping
Recognition in the wild

Textbooks:

There is no particular textbook and the course will rely heavily on the lecture slides and the state-of-the-art papers. For RL basics, Sutton and Barto’s book is highly recommended. For CV basics, check Szeliski’s textbook.

Prerequisites:

Interest in robot learning; basic programming skills (Python + PyTorch?); Machine Learning (e.g., neural networks) basics

Schedule:

Week	Topic	Slides
Week1	Introduction Supervised action learning	Intro BC
Week2	Background - images, features, and classifiers	Basics Image, features, classify
Week3	Background - machine learning Deep learning (CNNs)	Regression1, 2 DL_intro
Week4	Deep learning 2 Simulation and CoLab	CNN1 CoLab and libraries
Week5	Deep learning 3 Background - robot control	CNN2 Control_intro Assignment#1
Week6	Reinforcement learning basics RL - Dynamic programming	RL_basic DP
Week7	(Fall Break) RL - Monte Carlo	MC
Week8	RL - Q-Learning Deep reinforcement learning	TD DeepQ
Week9	Intermediate summary Mid-term exam
Week10	More TD Policy gradients	n-step_TD Policy_gradients
Week11	Actor-critic Planning	Planning
Week12	Model-based RL Architectures for robot learning	Model-based Representations
Week13	(remote?) class on Tuesday Thanksgiving week
Week14	Transformers for robotics	RT Transformer basics
Week15	Summary Guest lectures

Use your @stonybrook.edu account to access the slides.

Course requirements and grading:

Programming assignments (40%).

Final exam (30%).

Midterm exam (20%).

Acknowledgement:

This course has been inspired by the Deep Reinforcement Learning course by Sergey Levine (Berkeley), Reinforcement Learning course by Adam and Martha White (Alberta), Computer Vision course by Linda Shapiro (UW).