This Week at The Palindrome
What I'm working on: epidemiological models and post-apocalyptic ecosystems
Hey! It’s been a hot minute since I shared what’s on my mind with you.
My son just turned eleven months old, and he is crawling around faster than a xenomorph facehugger in the vents of a Weyland-Yutani warship. Combine it with extreme curiosity and zero sense of fear, and you have a sweet little troublemaker who is at the top of a tall flight of stairs before I can say “what”.
So, I’m a full-time parent now, with a couple of work hours snuck in between 2:00 a.m. and 6:00 a.m.
But that’s alright: witnessing my little one grow every day is a blessing.
On the other hand, you are here for high-quality deep dives into mathematics and machine learning, and I have some exciting updates to share. I just finished with my latest video, talking about The Math You Missed Behind Gradient Descent. Check it out if you haven’t already. It’s a level up from all aspects: procedural animations in Manim and superior storytelling.
(Make sure to subscribe to The Palindrome YouTube channel. It would mean a great deal to me.)
Currently, I’m working on two new videos:
one about how I rebuilt Liberty City from the original Grand Theft Auto just to run epidemic simulations,
and one about simulating the entire post-apocalyptic ecosystem from the Metro 2033 universe.
Let’s see the details!
The Math You Missed Behind Gradient Descent
First, about my latest video. Differential equations and dynamical systems are two of my favorite topics by far; their beauty and usefulness are unparalleled.
Sadly, they are fully omitted from an average machine learning curriculum, as they require extensive familiarity with calculus. On the other hand, gradient descent feels like magic without looking through the lens of differential equations.
Think about it: why does going against the gradient of the loss landscape in parameter space get us to a local optimum?
Let me tell you: for the exact same reason a pendulum stops at the bottom. Mechanical systems converge to their equilibrium (if we are lucky), and gradient descent is a mechanical system in disguise.
I love this topic, so I decided to make a full video about it.
This video is a special one, as I created it with a new workflow. This time, I wrote an extremely detailed script in Manim (with voiceovers and scene descriptions), recorded the audio, then asked Codex to
transcribe it,
add timestamps,
and create a Manim animation as specified in the script.
Here’s a snippet from the script:
