Robotic Manipulation

Perception, Planning, and Control

Fall 2021

Course Description

Introduces the fundamental algorithmic approaches for creating robot systems that can autonomously manipulate physical objects in unstructured environments such as homes and restaurants. Topics include perception (including approaches based on deep learning and approaches based on 3D geometry), planning (robot kinematics and trajectory generation, collision-free motion planning, task-and-motion planning, and planning under uncertainty), as well as dynamics and control (both model-based and learning-based).

Homework assignments will guide students through building a software stack that will enable a robotic arm to autonomously manipulation objects in cluttered scenes (like a kitchen). A final project will allow students to dig deeper into a specific aspect of their choosing. The class has hardware available for ambitious final projects, but will also make heavy use of simulation using cloud resources.

6.800 is the undergraduate version of the class. It serves as an Advanced Undergraduate subject (AUS), Independent Inquiry (II) and a Communication-intensive subject (CI-M). Our communications focus is on reviewing active research papers and producing an excellent final project. Due to the significant emphasis on communications and the final project, the course is 15 units (rather than the more standard 12) and includes a Friday recitation.

6.843 is the graduate version of the class. It qualifies as a Technical Qualifying Exam (TQE) subject (in Group 3: Artificial Intelligence). It is an Approved Advanced Graduate Subject (AAGS), and can automatically be counted as an AUS, and/or grad_II. This version of the course is 12 units, but to match the high standards of the department TQEs, the expectations for students registered for 6.843 are higher than for 6.800. Relative to 6.800, the grading rubrics for the final project place increased emphasis on technical depth, and reduced emphasis on communication.

Course Information

Class Time and Location

Fall semester (September - December, 2021)
Lectures: Tuesday, Thursday 2:30 - 4:00 pm in 32-144
6.800 recitations: Friday, 2-3pm. (Room TBC).

Our current plan is to offer the class in person this fall. We will also stream the lectures to YouTube (with recordings available immediately after).

See class schedule for complete details

Office Hours

Rachel Holladay (TA)
Friday 1-2pm or Appointment by email
Dani White (TA)
Monday 4-5pm or Appointment by email

Grading Policy

Late assignments will be penalized 10% every 24 hours. Additionally, we will grant a one time (one pset) extension of up to one week without penalty.
We do not have a midterm nor final exam.

6.800 grade distribution
Assignments: 40%
Journal club/peer reviews: 5%
Project proposal: 15%
Project report: 30%
Project presentation: 10%

6.843 grade distribution
Assignments: 40%
Project proposal: 5%
Project updates: 10%
Project report: 35%
Project presentation: 10%

Prequisites

Basic linear algebra, probability and algorithms. A basic understanding of neural networks. Coding assignments will be in Python using cloud-provisioned Jupyter notebooks.

The prerequisites 6.0001 and 6.036 listed in the course guide were chosen as an example of coverage for this material.

See the FAQ for the answers to some common questions about prerequisites.

Class Forums

Forums are on Piazza (sign up with your @mit.edu email address). If you do not have an @mit.edu address but believe you are eligible to participate via zoom/piazza, please contact the staff.

For 6.800, communication on CI-M material will primarily happen through Canvas. If you are enrolled in 6.800 but cannot access the canvas page, please contact the staff.

Open Access


All of the course materials are freely available.
  • Links to lecture notes, problem sets, and additional resources will be linked from the course calendar.
  • Lectures will be live streamed on YouTube (with recordings available immediately after).

We hope to actively engage with the anybody who wants to spend time on the material. Please ask us questions on the YouTube lectures and/or via the annotation tool on the course textbook.

Readings and Assignments

This is a new class, offered for the first time in this form in Fall 2020. The "textbook" is under very active development, and will be updated substantial during this semester. We will post relevant reading material refer to the schedule.

Assignments (to be released as they're assigned)

Links and due dates for assignments will be on the schedule.

Submission of Assignments
We'll be using Gradescope for problem set submission and grading. Each problem set is weighted equally. The login code for this class will be posted on Piazza -- please create an account and add yourself to this class using that code only if you are taking this class for credit. Grading will rely on submitted code passing an automated battery of tests, in combination with our review of your code and inline responses to questions. More details will be provided when assignments are released.

Collaboration Policy
Collaboration is encouraged, but the work you submit for assignments is expected to be entirely your own. That is, the writing and code must be yours, and you must fully understand everything that you hand in. Discussing the details of how to solve a problem is fine, but you must write the solution yourself. To avoid plagiarizing, you shouldn't be looking at someone else's solution while you write down your own. If you collaborated significantly (use your own discretion for "significantly") on a problem, list the people you collaborated with next to your solution.

Final Projects

The final project will be your opportunity to explore some of the topics introduced in the course more deeply; either in simulation or on real hardware.

Details about the final projects are availble on this page

FAQ

I don't have any robotics experience, is it okay if I take the class?
Yes. The course is designed to not assume any prior experience with robotics. If you have plenty of robotics experience, that's great too.

What will assignments be like?
The assignments will comprise a mix of hand-written math and programming. These will be approximately weekly, and for approximately only the first half of the term. During the second half we will ask you to focus on your final project.

Programming assignments will be in Python and will feature use of Drake, a toolbox for planning, control and analysis for robotics. Drake was developed out of the Robot Locomotion Group and its development is now led by Toyota Research Institute.

All assignments (both hand-written components and programming components) will be graded using Gradescope.

I am not sure if I fit the prerequisites, what should I do?

Please speak to a staff member.

What can I do to best prepare for success in the class?
The course takes a rigorous mathematical and algorithmic approach to robotics. The only required prerequisites are basic familiarity with linear algebra. If you work hard, we can teach you the rest.

If, however, you would like to prepare as best as possible, here is what the course TAs recommend:

  • Linear Algebra:
    • A strong, intuitive understanding of Linear Algebra will very much help you with this class. If off the top of your head you don't remember what the rank of a matrix is, or how to do Singular Value Decomposition, we recommend you review your linear algebra. The entire world of robotics is rich in linear algebra -- you will not regret investing time in mastering fundamentals!
    • To brush up on linear algebra, the content and video lectures from Gilbert Strang's classic course, MIT 18.06 have helped many students in the past.
    • If you are interested in the a more theoretical treatment, that is still very approachable, consider Linear Algebra Done Right by Sheldon Axler.
  • Programming in Python
    • If you are somewhat familiar with Python but would like to brush up on syntax, this tutorial from Stanford CS231n provides a good overview.
    • If you are an absolute beginner with Python, Codecademy provides a friendly introduction.
    • Although you will not need to know C++ for the class, you might be happy to know that the underlying software is written in C++, and is very suitable for use in mature engineering pipelines.
  • Mathematical Optimization
    • We will make some use of mathematical optimization, and hope to teach you most of what you need to know. But acquiring a background in the subject will help you deepen understanding. If you know the following acronyms, you are totally set: LP, QP, MIP, SDP.
    • Video lectures on Convex Optimization by Stephen Boyd are an excellent resource, as is the reference textbook Convex Optimization by Boyd and Vandenberghe.
  • Machine Learning
    • Basic background in Machine Learning will help. 6.036 is a good introduction. There are also many great introductory classes online, Andrew Ng's is one.
  • Robotics
    • You will not need any prior robotics exposure to succeed in the class. If however you want to start absorbing fundamentals (frame transformations, manipulator equations, etc.) then Introduction to Robotics: Mechanics and Control by John Craig is a good reference. Modern Robotics is also free online and has excellent video lectures.

People

You can contact the course staff at: manipulation-staff [AT] mit [DOT] edu

Instructor

Russ Tedrake

Teaching Assistant

Rachel Holladay

Teaching Assistant

Dani White

Communications Instructor

Nora Jackson

Communications Instructor

Laura McKee