This course will cover a range of topics related to the problem of how to get AI systems to do what we, as humans, actually want them to do. We will explore a range of topics including active learning, human intent and preference learning, algorithmic teaching, and AI safety. Classes will be a mix of lectures covering foundational materials as well as hands-on analysis and exploration of both seminal and recent research papers. Throughout the semester, students will also be engaged in a novel research project or in-depth literature review, culminating in a final presentation and written technical report. By taking this course, students will develop a broad understanding of the common techniques and unique research challenges involved in building AI systems that learn from, interact with, and assist humans. Additionally, students will learn and practice fundamental research skills, including how to read, write, and review research papers, how to quickly prototype and test research ideas, and how to give technical presentations.
At the end of this course, you will have gained both knowledge/abilities related to human-AI interaction and alignment, as well as to research and presentation skills:
Human-AI Interaction and Alignment Skills
Articulate the challenges of developing ML and AI algorithms that interact with and are aligned with humans.
Articulate the similarities and differences in motivation, formalism, and algorithmic solutions between RL, imitation learning, IRL, and planning.
Apply skills learned in class to create and evaluate agents that learn from human input.
General Research Skills
Communicate scientific content to a peer audience.
Analyze and critique a scientific paper's experimental design and analysis.
Create novel research questions, form hypotheses, design and execute experiments to test hypotheses.
Your performance in this course will be evaluated by:
Paper Analysis and Discussion Participation (30%): Each student will be involved in the analysis and/or presentation of research papers each week. We will be following a Role-Player Paper-Reading Seminar style) where students are assigned a role for each paper, such as Scientific Peer Reviewer, Archaeologist, Academic Researcher, Hacker, or Industry Practitioner. Students will be assigned different roles and will be graded based on their preparation, analysis, presentation. Students are also expected to participate in discussions, even when not assigned a particular role.
Homeworks (20%): There will be a series of short homework assignments (~4 hours each) to gain hands-on familiarity with the topics covered in class.
Quizes (10%): Short quizzes periodically throughout the semester.
Literature Survey and Proposal (10%): Students will perform a brief literature review and propose a novel research topic (2 pages total) that will form their final project (5+ relevant papers, organized by different features, identifying a gap in the state of the art and proposing a project to fill this gap).
Final Project (30%): You will have a final presentation, and submit a report (up to 6 pages) at the end.
Students should work in teams of 2-4 students for all homework assignments. Only one student needs to submit for each group, but please include all group members' names on the submission.
All assignments, unless otherwise announced, must be submitted to the designated area of Canvas. Do not submit assignments via email.
Final projects can be done individually or in groups of up to 4 students.
Grading for this course will follow the following scale. Note, failure to turn in final project will result in failing the class, regardless of the student's grade on other assignments.
Accommodations will be considered on an individual basis and may require documentation.
Please contact your instructor and/or teaching assistant as soon as possible (preferably shortly before the semester begins) to request accommodations of any kind.
Extreme personal circumstances
Please contact your instructor as soon as possible if an extreme personal circumstance (hospitalization, death of a close relative, natural disaster, etc.) is interfering with your ability to complete your work.
Religious Practice
To request an accommodation for religious practices, contact your instructor at the beginning of the semester.
Active Duty Military
If you are a student on active duty with the military and experience issues that prevent you from participating in the course because of deployment or service responsibilities, contact your instructor as soon as possible to discuss appropriate accommodations.
Disability Access
All written information in this course can be made available in an alternative format with prior notification to the Center for Disability Services (CDS). CDS will work with you and the instructor to make arrangements for accommodations. Prior notice is appreciated. To read the full accommodations policy for the University of Utah, please see Section Q of the Instruction & Evaluation regulationsLinks to an external site..
If you will need accommodations in this class, contact:
This syllabus is not a contract. It is meant to serve as an outline and guide for your course. Please note that your instructor may modify it to accommodate the needs of your class.
You will be notified of any changes to the Syllabus.
Students may drop a course within the first two weeks of a given semester without any penalties.
Students may officially withdraw (W) from a class or all classes after the drop deadline through the midpoint of a course. A “W” grade is recorded on the transcript and appropriate tuition/fees are assessed. The grade “W” is not used in calculating the student’s GPA.
It is assumed that all work submitted to your instructor is your own work. When you have used the ideas of others, you must properly indicate that you have done so.
It is expected that students adhere to University of Utah policies regarding academic honesty, including but not limited to refraining from cheating, plagiarizing, misrepresenting one’s work, and/or inappropriately collaborating. This includes the use of generative artificial intelligence (AI) tools without citation, documentation, or authorization. Students are expected to adhere to the prescribed professional and ethical standards of the profession/discipline for which they are preparing. Any student who engages in academic dishonesty or who violates the professional and ethical standards for their profession/discipline may be subject to academic sanctions as per the University of Utah’s Student Code: https://regulations.utah.edu/academics/6-410.phpLinks to an external site.
Plagiarism and cheating are serious offenses and may be punished by failure on an individual assignment, and/or failure in the course. Academic misconduct, according to the University of Utah Student Code,
“...Includes, but is not limited to, cheating, misrepresenting one’s work, inappropriately collaborating, plagiarism, and fabrication or falsification of information…It also includes facilitating academic misconduct by intentionally helping or attempting to help another to commit an act of academic misconduct.”
The Content is made available only for your personal, noncommercial educational, and scholarly use. You may not use the Content for any other purpose, or distribute, post or make the Content available to others unless you obtain any required permission from the copyright holder. Some Content may be provided via streaming or other means that restrict copying; you may not circumvent those restrictions. You may not alter or remove any copyright or other proprietary notices included in the Content.
Please see the Code of Student Rights and ResponsibilitiesLinks to an external site., Section III.A.5 regarding the use and distribution of class ContentLinks to an external site. and materials. Section III.A.5. prohibits the following: Sale or distribution of information representing the work product of a faculty member to a commercial entity for financial gain without the express written permission of the faculty member responsible for the course. (“Work product” means original works of authorship that have been fixed in a tangible medium and any works based upon and derived from the original work of authorship.)
The University of Utah values the safety of all campus community members. You will receive important emergency alerts and safety messages regarding campus safety via text message.
To report suspicious activity or to request a courtesy escort, contact:
Campus Police & Department of Public Safety 801-585-COPS (801-585-2677) dps.utah.eduLinks to an external site. 1735 E. S. Campus Dr. Salt Lake City, UT 84112
Your personal health and wellness are essential to your success as a student. Personal concerns like stress, anxiety, relationship difficulties, depression, or cross-cultural differences can interfere with a student’s ability to succeed and thrive in this course and at the University of Utah.
Please feel welcome to reach out to your instructor or TA to handle issues regarding your coursework.
For helpful resources to manage your personal wellness and counseling options, contact:
Center for Student Wellness 801-581-7776 wellness.utah.edu 2100 Eccles Student Life Center 1836 Student Life Way Salt Lake City, UT 84112
Title IX makes it clear that violence and harassment based on sex and gender (which includes sexual orientation and gender identity/expression) is a civil rights offense subject to the same kinds of accountability and the same kinds of support applied to offenses against other protected categories such as race, national origin, color, religion, age, status as a person with a disability, veteran’s status, or genetic information.
If you or someone you know has been harassed or assaulted, you are encouraged to report it to university officials:
Title IX Coordinator & Office of Equal Opportunity and Affirmative Action 801-581-8365 oeo.utah.eduLinks to an external site. 135 Park Building 201 Presidents' Cir. Salt Lake City, UT 84112
Campus Police & Department of Public Safety 801-585-COPS (801-585-2677) dps.utah.eduLinks to an external site. 1735 E. S. Campus Dr. Salt Lake City, UT 84112
If you do not feel comfortable reporting to authorities, the U's Victim-Survivor Advocates provide free, confidential, and trauma-informed support services to students, faculty, and staff who have experienced interpersonal violence.
To privately explore options and resources available to you with an advocate, contact:
Center for Student Wellness 801-581-7776 wellness.utah.edu 328 Student Services Building 201 S. 1460 E. Salt Lake City, UT 84112
The University of Utah seeks to provide equal access to its programs, services, and activities for people with disabilities.
All written information in this course can be made available in an alternative format with prior notification to the Center for Disability & Access (CDA). CDA will work with you and the instructor to make arrangements for accommodations. Prior notice is appreciated. To read the full accommodations policy for the University of Utah, please see Section Q of the Instruction & Evaluation regulationsLinks to an external site..
If you will need accommodations in this class, or for more information about what support they provide, contact:
Your success at the University of Utah is important to all of us here! If you feel like you need extra support in academics, overcoming personal difficulties, or finding community, the U is here for you.
Student Support Services (TRIO)
TRIO federal programs are targeted to serve and assist low-income individuals, first-generation college students, and individuals with disabilities.
Student Support Services (SSS) is a TRIO program for current or incoming undergraduate university students who are seeking their first bachelor's degree and need academic assistance and other services to be successful at the University of Utah.
For more information about what support they provide, a list of ongoing events, and links to other resources, view their website or contact:
The AIRC works to increase American Indian student visibility and success on campus by advocating for and providing student-centered programs and tools to enhance academic success, cultural events to promote personal well-being, and a supportive “home-away-from-home” space for students to grow and develop leadership skills.
For more information about what support they provide, a list of ongoing events, and links to other resources, view their website or contact:
Using a pan-African lens, the Black Cultural Center seeks to counteract persistent campus-wide and global anti-blackness. The Black Cultural Center works to holistically enrich, educate, and advocate for students, faculty, and staff through Black-centered programming, culturally affirming educational initiatives, and retention strategies.
For more information about what support they provide, a list of ongoing events, and links to other resources, view their website or contact:
Our mission is to support and coordinate information, program development, and services that enhance family resources as well as the availability, affordability, and quality of child care for University students, faculty, and staff.
For more information about what support they provide, a list of ongoing events, and links to other resources, view their website or contact:
The Center for Disability and Access is dedicated to serving students with disabilities by providing the opportunity for success and equal access at the University of Utah. They also strive to create an inclusive, safe, and respectful environment.
For more information about what support they provide and links to other resources, view their website or contact:
The Center for Ethnic Student Affairs offers several programs dedicated to the success of students with varied cultural and ethnic backgrounds. Its mission is to create an inclusive, safe campus community that values the experiences of all students.
For more information about what support they provide, a list of ongoing events, and links to other resources, view their website or contact:
English as a Second/Additional Language (ESL) Students
If you are an English language learner, there are several resources on campus available to help you develop your English writing and language skills. Feel free to contact:
Immigration is a complex phenomenon with broad impact—those who are directly affected by it and those who are indirectly affected by their relationships with family members, friends, and loved ones. If your immigration status presents obstacles that prevent you from engaging in specific activities or fulfilling specific course criteria, confidential arrangements may be requested from the Dream Center.
Arrangements with the Dream Center will not jeopardize your student status, your financial aid, or any other part of your residence. The Dream Center offers a wide range of resources to support undocumented students (with and without DACA) as well as students from mixed-status families.
For more information about what support they provide and links to other resources, view their website or contact:
The LGBTQ+ Resource Center acts in accountability with the campus community by identifying the needs of people with a queer range of [a]gender and [a]sexual experiences and responding with university-wide services.
For more information about what support they provide, a list of ongoing events, and links to other resources, view their website or contact:
The mission of the Veterans Support Center is to improve and enhance the individual and academic success of veterans, service members, and their family members who attend the university; to help them receive the benefits they earned, and to serve as a liaison between the student veteran community and the university.
For more information about what support they provide, a list of ongoing events, and links to other resources, view their website or contact:
The Women’s Resource Center (WRC) at the University of Utah serves as the central resource for educational and support services for women. Honoring the complexities of women’s identities, the WRC facilitates choices and changes through programs, counseling, and training grounded in a commitment to advance social justice and equality.
For more information about what support they provide, a list of ongoing events, and links to other resources, view their website or contact:
The Office for Inclusive Excellence is here to engage, support, and advance an environment fostering the values of respect, diversity, equity, inclusivity, and academic excellence for students in our increasingly global campus community. They also handle reports of bias in the classroom as outlined below:
Bias or hate incidents consist of speech, conduct, or some other form of expression or action that is motivated wholly or in part by prejudice or bias whose impact discriminates, demeans, embarrasses, assigns stereotypes, harasses, or excludes individuals because of their race, color, ethnicity, national origin, language, sex, size, gender identity or expression, sexual orientation, disability, age, or religion.
For more information about what support they provide and links to other resources, or to report a bias incident, view their website or contact:
The syllabus page shows a table-oriented view of the course schedule, and the basics of
course grading. You can add any other comments, notes, or thoughts you have about the course
structure, course policies or anything else.
To add some comments, click the "Edit" link at the top.
Course Summary:
Date
Details
Due
Mon Aug 28, 2023
Assignment
Reading Assignment for Imitation Learning via Behavioral Cloning (Mon 28th Aug, 2023)