Back to Course home page
See also Daily Schedule - Part 2
Since Python’s classes are something we didn’t do much with, but they are often considered part of an introduction to Python, you could do a quick introduction to classes (aka “object-oriented programming”) such as Section 1.16 of Python Distilled by Beazley. The three things that are generally considered important about object-oriented programming are encapsulation (controlled access to the object’s data), inheritance (behavior is inherited from superclasses in the class heirarchy and extended by subclasses), and polymorphism (behavior can be overriden in subclasses). Chapters 4 and 7 of Beazley are a much more complete introduction to object-oriented programming.
Although the chapter we finished with did not attempt to cover the 2017 “Attention is All You Need” paper, Grus has almost perfectly set you up for a junior-level course in natural-language processing and LLMs that almost every computer science department is now or shortly going to have on offer. Before or instead of taking such a course, you can review or go beyond where we have gotten as of Chapter 19 of Grus in multiple ways:
(1) A concise and mathematically-sophisticated review of all that we have done (and then some) is “A high-bias, low-variance introduction to Machine Learning for physicists.”
(2) Grus considers how LLMs have changed and will continue to change the workflow of a data scientist starting at the 15:00 mark in this late-2023 video “Doing Data Science in the Time of ChatGPT.” This is a casual survey that may only serve to cement what you have already discovered you can do with a current-generation LLM like Grok 3 or ChatGPT 4.5.
(3) Recapitulate what we have done and then look inside the mathematics and implementation of LLMs, without actually doing any more implementation, by watching all seven of the 3Blue1Brown videos by Grant Sanderson. Sanderson’s mathematics visualizations are a joy to watch even when he is presenting something you already understand, but perhaps the first four in the series are not worth your time given how much we have covered in Grus. The last three in the series (“Transformers Explained Visually,” “Attention in Transformers,” and “How Might LLMs Store Facts?”) will certainly be new and worthwhile.
(4) How this is going to affect industry after industry is anybody’s guess, but a recent and informed guess (from venture capitalist Marc Andreessen) is in this late-2024 Lex Fridman interview of Marc Andreessen (the link deliberately jumps you to a point over three hours into the interview).
(5) Following up on a recommendation for further reading given at the end of Chapter 19 of Grus, consider the preliminary version of Deep Learning with Python, Third Edition (the final version of the third edition is estimated to appear September 2025).