Robotic Manipulation

Course Description

Introduces the fundamental algorithmic approaches for creating robot systems that can autonomously manipulate physical objects in unstructured environments such as homes and restaurants. Topics include perception, learning (behavior cloning and reinforcement learning), planning (robot kinematics and trajectory generation, collision-free motion planning, and task-and-motion planning), as well as elements of dynamics and control.

Homework assignments will guide students through building a software stack that will enable a robotic arm to autonomously manipulate objects in cluttered scenes. A final project will allow students to dig deeper into a specific aspect of their choosing. The homework and projects will make use of dynamic simulation.

6.4210 is the undergraduate version of the class. It serves as an Advanced Undergraduate subject (AUS), Independent Inquiry (II) and a Communication-intensive subject (CI-M). Our communications focus is on reviewing active research papers and producing an excellent final project. Due to the significant emphasis on communications and the final project, the course is 15 units (rather than the more standard 12) and includes a Friday recitation.

6.4212 is the graduate version of the class. It qualifies as a Technical Qualifying Exam (TQE) subject (in Group 3: Artificial Intelligence). It is an Approved Advanced Graduate Subject (AAGS), and can automatically be counted as an AUS, and/or grad_II. This version of the course is 12 units, but to match the high standards of the department TQEs, the expectations for students registered for 6.4212 are higher than for 6.4210. 6.4212 students have additional problems on problem sets. Relative to 6.4210, the grading rubrics for the final project place increased emphasis on technical depth, and reduced emphasis on communication.

Course Information

Prerequisites

Open Access

Collaboration Policy

People

You can contact the course staff at: manipulation-staff [AT] mit [DOT] edu, but we prefer contact via Piazza private message.

Instructors

Tomas Lozano-Perez

Leslie Kaelbling

Russ Tedrake

Teaching Assistants (Graduate)

Shao Yuan Chew Chia

Aditya Agarwal

Guillermo Mendoza

Belinda Vela

Teaching Assistants (Undergraduate)

Joy Zhuo

Kaitlyn Zhang

Grace Yuan

Communications Instructors

Nora Jackson

David Larson

Talia Shalev

Jeanne Wildman

FAQ

I don't have any robotics experience, is it okay if I take the class?
Yes. The course is designed to not assume any prior experience with robotics. If you have plenty of robotics experience, that's great too.

What will assignments be like?
The assignments will comprise a mix of math and programming. These will be approximately weekly for about the first 2/3 of the term. There will be two in-class mid-term quizzes covering material similar to the problem sets. During the rest of the term we will ask you to focus on your final project.

Programming assignments will be in Python and will feature use of Drake, a toolbox for planning, control and analysis for robotics. Drake was developed out of the Robot Locomotion Group and its development is now led by Toyota Research Institute.

All assignments will be graded using Gradescope.

I am not sure if I fit the prerequisites, what should I do?
Please speak to a staff member.

What can I do to best prepare for success in the class?
The course takes a rigorous mathematical and algorithmic approach to robotics. The only required prerequisites are basic familiarity with linear algebra. If you work hard, we can teach you the rest.

If, however, you would like to prepare as best as possible, here is what the course TAs recommend:

• Linear Algebra:
• A strong, intuitive understanding of Linear Algebra will very much help you with this class. If off the top of your head you don't remember what the rank of a matrix is, or how to do Singular Value Decomposition, we recommend you review your linear algebra. The entire world of robotics is rich in linear algebra -- you will not regret investing time in mastering fundamentals!
• To brush up on linear algebra, the content and video lectures from Gilbert Strang's classic course, MIT 18.06 have helped many students in the past.
• If you are interested in the a more theoretical treatment, that is still very approachable, consider Linear Algebra Done Right by Sheldon Axler.
• Programming in Python
• If you are somewhat familiar with Python but would like to brush up on syntax, this tutorial from Stanford CS231n provides a good overview.
• If you are an absolute beginner with Python, Codecademy provides a friendly introduction.
• Although you will not need to know C++ for the class, you might be happy to know that the underlying software is written in C++, and is very suitable for use in mature engineering pipelines.
• Mathematical Optimization
• We will make some use of mathematical optimization, and hope to teach you most of what you need to know. But acquiring a background in the subject will help you deepen understanding. If you know the following acronyms, you are totally set: LP, QP, MIP, SDP.
• Video lectures on Convex Optimization by Stephen Boyd are an excellent resource, as is the reference textbook Convex Optimization by Boyd and Vandenberghe.
• Machine Learning
• Basic background in Machine Learning will help. 6.390 is a good introduction. There is an on-line version of 6.036 (the previous number for 6.390) from 2020. There are also many other great introductory classes online, Andrew Ng's is one
• Robotics
• You will not need any prior robotics exposure to succeed in the class. If however you want to start absorbing fundamentals (frame transformations, manipulator equations, etc.) then Introduction to Robotics: Mechanics and Control by John Craig is a good reference. Modern Robotics is also free online and has excellent video lectures.