CS 288 Graduate NLP

Archive

Notes and Cheat Sheet

Review

The graduate follow-up to EECS 183. As promised, CS 288 focused on transformers and LLMs. The first unit built up to Llama 3 pretraininghe next unit was retrieval with RAG, and the associated assignment was rather creative: build a RAG QA system for the EECS directory, doing all of the scraping and chunking by hand. The final unit got us from Llama 3 to 2026 SOTA by covering MoE, RLHF, agents, and test time compute.

Prof. Min is a young, lively addition to Berkeley NLP (she comes across as a bit of a workaholic, but maybe that’s just me) and she put a lot of effort in modernizing the curriculum. Unfortunately, Prof. Suhr ended up reusing much of her 183 slides, so I didn’t end up learning anything when she lectured.

For the final research paper, my teammates and I created a video RAG system for lecture videos, then rendering animation-dense instructional videos post-retrieval.

CS 288 was a rather project-heavy class, 3.5/5 in difficulty and 4.5/5 in fun.