Consuming content never felt as terrible as it feels now. Given the various “metas”1 of the past couple of years, that’s quite a statement: we’ve seen

• misogynistic red-pill shorts and Andrew Tate clips flooding every platform,

• superstars shilling shitcoins for money on every available outlet,

• or watched the chaos unfold during the first wave of COVID, with millions launching content creator careers from their quarantines.

(If I ever launch a punk band, I’ll name it Superstars Shilling Shitcoins.)

Today, even though it doesn’t feel as dramatic, I can’t help but constantly feel that something is off. Sure, we all knew that the President of the United States minting his own crypto was absurd; it was a Black Mirror episode happening in real life.

Right now, it’s more like Pleasantville, as my wife pointed out. She has been obsessing quite a lot lately over the model collapse phenomenon.

If you don’t remember Pleasantville, let me explain. I wouldn’t have remembered it either if it was not the very first movie I watched on HBO in the early 2000s. Growing up in a post-socialist country like Hungary, HBO was like a gateway to the Western world. So, this movie became a foundational memory for me.

In Pleasantville, everything is pleasant. (Surprise.) People are nice. They politely greet each other when they meet in the mornings, noons, evenings. The handsome husbands go home straight after work, just in time for dinner perfectly served by their gorgeous wives. (In the always spotless kitchen.) The high school basketball players never miss shots, fires are impossible to start, and the firemen’s only job is to rescue cats stuck in trees. Dates always end well, with lovers holding hands, staring at the stars in the night sky. Pleasantville is the fictional version of LinkedIn. Oh, and everything is black and white. No pain, no pleasure, no lows, no highs. It’s all predictable. All average.

Why are we talking about this? Because reading AI-generated content feels like spending time in Pleasantville.

Sentences are well-formed. The grammar is spotless. The voice is polite. The content is designed to avoid challenging your beliefs, because that would perhaps make you stop reading. No swearing. No sarcasm. No controversial opinions.

Really, it’s the nature of AI-generated content. Large language models generate responses token by token, selecting the one that has the highest probability given the previous output tokens. In other words, at their hearts, LLMs are statistical bullshit generators.

They are trained to say what you want to hear. Not what you need to hear.

(Yeah, I know, there’s reasoning, tool calling, memory, and other neat tricks. Still, fundamentally, generative models are “stochastic parrots”.)

For functional purposes like building makeshift apps, they are unbeatable, though. I don’t want my Python scripts to challenge my beliefs; I want them to work. I’m using them to automate tasks and build things. If I want a short animation illustrating gradient descent, I don’t care how neat the Manim script is. I only care if it does what I ask it to do, and that’s it.

LLMs are also perfect for collecting and processing information. Whenever I want to quickly learn about a technical subject, such as video game architectures or epidemiological models, nothing beats chatting up Claude and going from zero to one in a couple of minutes. Hell, there are Substacks with LLM-written posts that provide a ton of valuable information, ranging from science to parenting through engineering and education. I’m even using LLMs to fix the grammar mistakes in this very post you are reading.

But that’s not what I want to read when I subscribe to you, and I’m guessing that this is not what you, who are reading The Palindrome, want from me either.

I’m reading Substack to find something that I cannot find in a discussion with a chatbot. I want you, the author, to make me uncomfortable. I want to glimpse into your beautiful mind, your unhinged thoughts, your limitless creativity. To question my beliefs, to see old things from new perspectives, to expand my horizons.

To update my weights.

Lately, it’s hard to do. I open my timeline(s), and it’s full of bland, uninspired, fortune-cookie bullshit. Every note and post is formulaic. Everyone is desperate for engagement, but no one wants to engage. I stopped reading Notes. I stopped writing them. I reached a point where I’m reluctant to read books released after 2022.

As a content creator, I’m facing a choice now. One: I automate the writing and publishing process with AI to keep up the pace. I’ll feed it all I ever wrote (a couple thousand pages already), make it learn my voice (Eastern European mathematician, loves death metal2 and gardening), give it a couple of topic ideas (machine learning, mathematics, growing up in a post-socialist country right after the collapse of the Soviet Union), and have it churn out post after post, feeding the content machine.

Two: I make something that’s better than 99% of what’s out there. Learn to tell great stories. Bring my words to life via beautiful animations. Take the time to execute grand ideas instead of publishing mediocre ones twice per week. To improve my skills, and use AI to augment them instead of replacing me. To be the signal, not the noise.

To reach for quality, instead of rushing for quantity.

If you have been following me for a while, you already know which path I’m taking. This year, I put growth on hold and spent all my work hours increasing the value of my content from every aspect.

I moved from static illustrations to rich animations.

I started to build interactive visualizations that you can explore along with the posts.

I learned the foundations of audio engineering, invested in a high-quality recording setup, and took a couple of voice coaching classes to improve my voiceovers.

When I’m not creating, I’m reading about writing, storytelling, and animation.

I’m becoming an artist. Not a content creator, not a technical writer, not a Substack author.

For me, nothing can replace the joy of creation. I’m writing these lines, rewriting them, picking them apart and putting them back together, second-third-hundredth-guessing every sentence, changing the pace, playing around with its rhythm, converging toward a chunk of text that has me within the lines.

For me, writing is thinking, a process where I zero in on the truth. (See my Coding on Paper essay, where I talk about this.)

I won’t automate my existence away. In the future, you’ll get more of my thoughts, instead of just more words, shaped by an algorithm.

Thanks for reading! In a regular newsletter issue, this would be the place for a Call To Action, a message to convince you to subscribe, engage, etc. Instead of shoving you down a marketing funnel, I’m encouraging you to do some thoughtful introspection.

If you are a content creator: do you use AI responsibly, or is your channel just the Substack equivalent of dropshipping?

If you are a reader: if your body is what you eat, then your mind is what you read. What do you read?

My son just turned eleven months old, and he is crawling around faster than a xenomorph facehugger in the vents of a Weyland-Yutani warship. Combine it with extreme curiosity and zero sense of fear, and you have a sweet little troublemaker who is at the top of a tall flight of stairs before I can say “what”.

So, I’m a full-time parent now, with a couple of work hours snuck in between 2:00 a.m. and 6:00 a.m.

But that’s alright: witnessing my little one grow every day is a blessing.

On the other hand, you are here for high-quality deep dives into mathematics and machine learning, and I have some exciting updates to share. I just finished with my latest video, talking about The Math You Missed Behind Gradient Descent. Check it out if you haven’t already. It’s a level up from all aspects: procedural animations in Manim and superior storytelling.

(Make sure to subscribe to The Palindrome YouTube channel. It would mean a great deal to me.)

Currently, I’m working on two new videos:

• one about how I rebuilt Liberty City from the original Grand Theft Auto just to run epidemic simulations,

• and one about simulating the entire post-apocalyptic ecosystem from the Metro 2033 universe.

Let’s see the details!

The Math You Missed Behind Gradient Descent

First, about my latest video. Differential equations and dynamical systems are two of my favorite topics by far; their beauty and usefulness are unparalleled.

Sadly, they are fully omitted from an average machine learning curriculum, as they require extensive familiarity with calculus. On the other hand, gradient descent feels like magic without looking through the lens of differential equations.

Think about it: why does going against the gradient of the loss landscape in parameter space get us to a local optimum?

Let me tell you: for the exact same reason a pendulum stops at the bottom. Mechanical systems converge to their equilibrium (if we are lucky), and gradient descent is a mechanical system in disguise.

I love this topic, so I decided to make a full video about it.

This video is a special one, as I created it with a new workflow. This time, I wrote an extremely detailed script in Manim (with voiceovers and scene descriptions), recorded the audio, then asked Codex to

1. transcribe it,

2. add timestamps,

3. and create a Manim animation as specified in the script.

Here’s a snippet from the script:

Read more

Even your state-of-the-art LLMs are trained with a beefed-up version of this ancient algorithm, generally described through a mountain-climbing analogy. Look around your current position in the loss landscape, find the direction of the steepest descent, then take a step toward it. Yet no one ever tells you what’s really happening behind the scenes.

Here’s the secret math behind gradient descent:

If you are still here, I have a small favor to ask of you. My YouTube channel is still small, and if you enjoy my content, you can support me by

1. watching the video,

2. subscribing to my channel,

3. and leaving a comment on YouTube with your questions and thoughts.

Over the last few months, I’ve been focused on making my deep dives into the mathematics of machine learning better and clearer. You already love my drawings, and moving from still images to animated visualizations is a light-year leap in quality. This is the future of The Palindrome.

It only takes a couple of clicks from you, but it means the world to me. With your support, I can continue being a full-time content creator, delivering bangers like the video above for you.

In any case, thanks for being here!

Cheers,
Tivadar

It’s Tivadar from The Palindrome. The next post is already in the final stages (it’s going to be one of my best ones), but before that, I’d love to ask you a quick favor.

I’m running a short reader survey to better understand who’s reading, what you’re working on, and what you’d like to see more of.

I’ll use the results to improve the newsletter and make sure any future sponsors are actually relevant to you, and future issues will stay relevant to you. Individual responses won’t be shared, and your contact information will only be used if you explicitly opt in.

It only takes a few minutes:

Let's see the survey!

Thanks!

Cheers,
Tivadar

Read more

Your support makes it possible for me to write these high value, high signal posts every week.

Subscribe now

Read more

Your support makes it possible for me to write these high value, high signal posts every week.

Subscribe now

Hey!

It’s Tivadar from The Palindrome. Back when I wrote the Mathematics of Machine Learning book, I realized

1. what a great language Python is,

2. and that object-oriented Python should be the first thing taught to every machine learning engineer.

So, I’ve been thinking about publishing a series, but my Python skills are not exactly top-of-the-line; I just hack and slash until things work. Fortunately, I found the best person who could do that!

It’s my pleasure to introduce Stephen Gruppetta, author of The Python Coding Stack, and my longtime online friend from back when X was called Twitter.

If you ever wanted to become a power user of Python and take advantage of all the heavy machinery provided by classes, operator overloading, inheritance, composition, etc., this is the article for you.

(It’s the second post in the object-oriented Python miniseries. Check the first one here for the foundations of OOP!)

Dig in!

Cheers,
Tivadar

A class is a template you can use to create several objects that share the same characteristics. All the objects created from a class will have the same structure and can perform the same actions.

However, sometimes you need to create objects that don’t have the same attributes but that are still similar to each other. You may need objects that share some characteristics but are sufficiently different from each other, so you can’t create them using the same class.

You want to avoid writing a brand new class from scratch. Instead, you’d like to reuse some of the code from the original class.

There are several options for handling overlap between classes. Inheritance is a common tool to link classes. However, inheritance is not always the right solution. Composition is an alternative technique to use a class as part of another class. In this article, you’ll learn about inheritance and composition. You’ll learn how to use each technique, and just as importantly, when to use them.

The Velocity Class

You learned about classes in the first article in this series. You wrote a Vector class that you can use to create vector instances.

You’ll make some changes to this class as you progress through this article. However, you’ll primarily work on a new class: the Velocity class.

You’ll consider two different routes to create the Velocity class. Velocity is a vector: it has a magnitude and a direction. Therefore, you can start from the Vector class you defined in the first article and use it as a starting point to build the Velocity class. This is the inheritance route.

However, you can also create a class that has a Vector instance as one of its attributes. You’ll also explore this composition route in this article.

Read more

The primary goal of this post series is to teach you the Representational Similarity Analysis (RSA), which is a machine-learning analysis that is used to compare distributed representations in different systems.

If you haven’t already read Part 1 in this series, please do so! It provides necessary background about how the RSA score is calculated and interpreted.

As a brief reminder, an RSA (representational similarity analysis) works by comparing cosine similarity matrices across different embeddings spaces (layers, blocks, models, etc.). The idea is that different embeddings spaces may have distinct coordinate systems and even different dimensionalities, but if their internal representational structures are similar, the relative similarities should be strongly correlated even if the vectors are distinct.

The additional goals of this second post are (1) to learn more about RSA and category specificity, and (2) to learn how to dissect the “hid…

Read more

It’s Tivadar from The Palindrome. Back when I wrote the Mathematics of Machine Learning book, I realized

1. what a great language Python is,

2. and that object-oriented Python should be the first thing taught to every machine learning engineer.

So, I’ve been thinking about publishing a series, but my Python skills are not exactly top-of-the-line; I just hack and slash until things work. Fortunately, I found the best person who could do that, and even better, he agreed to write a special article for you!

It’s my pleasure to introduce , author of , and my longtime online friend from back when X was called Twitter.

If you ever wanted to become a power user of Python and take advantage of all the heavy machinery provided by classes, operator overloading, inheritance, composition, etc., this is the article for you.

Dig in!

Cheers,
Tivadar

“A computer program stores data and does stuff with the data.”

This is not the most technical definition of a computer program you’ll see. But it’s a valid one. When you learn to code, you learn about data structures to store different types of data. And you also learn about tools needed to manipulate and transform the data. You often define functions containing code to “do stuff” with the data.

Your code will contain data structures and functions, and you pass those data structures to the functions when needed.

Object-oriented programming (OOP) brings these two aspects together into a single unit. This unit is the object, which contains the data and the tools needed to manipulate the data. This may not sound like much, but it enables you to think about the problem you’re trying to solve differently. You can visualize your problem in a way closer to how humans see the world.

Let’s look at some examples, starting with a concrete one. Consider a country. There’s plenty of data relevant to a country: its name, population size, geographical area, capital city, and more. All countries have these attributes. Therefore, you can create a template that includes these attributes that you can use each time you want to represent a different country. Countries also include people, so these can be included in the data for each country, too.

But countries also perform actions. They issue passports, they collect taxes, they create laws, and so on. OOP urges you to think of a single unit to represent a country, which includes all the data and tools needed for the country to perform the actions required. You’d create a class called Country–this is the template you need to create lots of countries. The class doesn’t represent a specific country but the idea of a country. Once you define the class, you can create as many instances of the class as you need. Each instance represents a specific country.

Classes and Instances • The Vector Class

But let’s work on a different example in this article. Let’s consider a vector. One way to view vectors is as entities with both magnitude and direction. In three-dimensional (3D) Euclidean geometry, a vector is represented by three numbers.

Let’s put on our OOP hat. We need a unit in our program to represent a vector. It needs to represent its data and its functionality. Let’s start by creating a class called Vector:

Admittedly, this class doesn’t do much for now. The ellipsis (...) is just a placeholder that’s valid Python syntax. You’ll add more to this class soon. In the previous section, I mentioned that a class is a template for creating many objects modelled from the same blueprint. Let’s create a few instances of this class:

There’s only one Vector class. But now you have two instances of this class. You create an instance of the class when you add parentheses after the class name. The objects referenced by v1 and v2 are separate objects that occupy different areas of your computer’s memory. You can confirm that they’re different objects by showing their identity using Python’s id() function. The two instances of the Vector class have different identities:

You’ll get different values from the ones shown here when you run this code on your computer. But what matters is that the two numbers you get are different from each other. You can also confirm that v1 and v2 represent different objects by using v1 is v2, which returns False.

Note that the terms object and instance are both commonly used to refer to the unit created by a class. They refer to the same thing.

However, these are “blank” objects. They don’t have anything beyond the bare minimum a Python object needs. Let’s add some data.

Read more

returns to The Palindrome! You know I’m a big fan of his work, and if you are into machine learning, you should be too. His posts always strike the perfect balance between educational, practical, and entertaining.

He recently published the book 50 ML Projects to Understand LLMs, and his upcoming two-part series on exploring word representations is taken directly from the book. If you want to understand how Large Language Models work under the hood, don’t miss the post below.

Enjoy!

Cheers,
Tivadar

Read more

“How to get started in machine learning?” is one of the most common questions I get. I have a couple of default answers, but they are based more on my personal experience than on science.

Inspired by this, I’ve mapped out the knowledge graph of machine learning, building a hierarchy of concepts that can guide you from the foundations to the state of the art.

A couple of fascinating patterns have emerged from my journey: the thin spine of mathematics that holds up the entire knowledge graph, the central concepts like gradient descent that enable modern machine learning as we know it, and more.

Here’s the video where I talk about my findings:

Read more

I’m finishing up my video about the knowledge graph of machine learning, which will be released next week; I’ll do a live premiere right here on Substack Live, with a discussion after the viewing. (You can join here.)

This video will mark a milestone for me: instead of relying on hand-crafted slides and a presentation-style exposition, I built a scripting engine on top of the knowledge graph explorer that turns a JSON script like

[{
  "at": 0.0,
  "action": "autoRotate",
  "axis": "y",
  "speed": 0.01,
  "duration": 9.3,
  "windDown": 2.0,
  "easing": "linear"
},
{
  "at": 9.3,
  "action": "selectNode",
  "nodeId": "gpt",
  "showPrerequisites": true,
  "showDependents": false,
  "duration": 2.0
}]

into a beautifully rendered video.

Here’s a sneak peek:

The machine learning knowledge graph project could serve as a template for my future content. From now on, I’ll go full multimodal, meaning that I’ll

• build interactive visualizations (such as the knowledge graph explorer),

• then write posts and record videos, aided by the interactive tool.

Now that the current video-in-progress is about to be finished, what’s next?

Read on.

Read more

This week, it’s my pleasure to introduce , a fellow scholar working at the forefront of physics, mathematics, and computer science.

One of the main reasons behind the success of modern science is mathematical modeling, the process of translating complex real-life observations into a language that allows us to generalize, understand, and predict.

If you have enjoyed this post, subscribe to his newsletter , a space dedicated to translating the complexity of the empirical world, from your cells to entire societies, into language that is as simple as possible, though not necessarily simpler.

Read more

Read more

Let’s try something new. In recent months, I started to feel that the weekly publishing schedule takes its toll on the quality of my posts. I want to take more time per post to give you some high-quality technical content. I’m inspired by amazing writers such as Sebastian Raschka () or Cameron R. Wolfe (), who publish once a month, but each post is a work of art.

On the other hand, I miss you. A monthly schedule feels too long, and I have a lot to share with you. Ever since I got my ChatGPT Max subscription with access to Codex, my creativity is out of bounds.

So, here’s a new format. Each week, I’m sending you my unfiltered stream of consciousness, all the projects that I’m currently working on. Think of it as joining me for a coffee, where we talk about all the exciting/revolutionary/insane ideas we have in our minds.

This week, there are two things on my mind: knowledge graphs and Minecraft.

Let’s start with knowledge graphs.

The Complete Map of Machine Learning

If you are a regular reader, you know that one of the most common questions I get is “which part of mathematics do I need to study machine learning?” My default answer, based on my decade of experience, is: a ton of linear algebra, a decent amount of calculus, and a snippet of probability theory.

To be honest, I’m not completely satisfied with my reply. So, I dug deep with my newly found agentic AI-fueled superpower to find what is scientifically backed.

Without further ado: here’s the full knowledge graph of mathematics and machine learning. 2081 nodes, 5149 edges.

(The images are screenshots from the interactive graph explorer I’m building, which will be open source and publicly available.)

Let’s unravel this.

Read more

Yesterday we concluded our first monthly workshop! As promised, here is the full recording, exclusive to paid subscribers.

You can access the Jupyter Notebook lecture notes here: https://github.com/the-palindrome/mathematics-of-machine-learning-workshop

The next monthly workshop is already in the works; it’s going to be the next iteration of the Neural Networks From Scratch course. The tentative date is April 18th, 15:00–19:00 CET, but stay tuned for the announcement, as this date might change.

Thanks again so much for attending!

Cheers,
Tivadar

Read more

I’m finally working on my upcoming book Machine Learning From Zero, the sequel to Mathematics of Machine Learning.

The first chapter has just been finished, which sets the foundations for the neat stuff, like implementing neural networks from scratch. (Which is the core of the book.)

Without further ado, here’s an exclusive preview.

Enjoy!

P.S. I’m writing this book in Jupyter Notebooks, and I turn them into Substack posts with NotebookPress, a tool I’m building to bring technical writing on Substack to the next level. If you write math-and-code-heavy content, you should check it out.

Machine learning is training predictive models from data.

Sure, we can be academic about it and refine the definition of machine learning by looking at the countless of nuances, but that’s not what we are here to do. We are here to understand the core fundamentals of machine learning — the fundamentals that will take you further than anything else.

I believe the only way to aquire deep knowledge of any technical subject is to take it apart and put it back together again. This is what we are here to do.

The fundamental machine learning setup consists of:

1. a dataset, usually coming in the form of input and target variables,

2. a parametric function that models the relation between the input and the target variables,

3. and a loss function that measures the model’s fit to the data.

Let’s start with the data.

To look behind the curtain of machine learning algorithms, we have to precisely formulate the problems that we deal with. Three important parameters determine a machine learning paradigm: the input, the output, and the training data.

All machine learning tasks boil down to finding a model that provides additional insight into the data, i.e., a function f that transforms the input x into the useful representation y. This can be a prediction, an action to take, a high-level feature representation, and many more. We’ll learn about all of them.

Mathematically speaking, the basic machine learning setup consists of:

1. a dataset 𝒟,

2. a function f that describes the true relation between the input and the output,

3. and a parametric model h — also called a hypothesis — that serves as our estimation of f.

Remark. (Common abuses of machine learning notation.)

Note that although the function f only depends on the input x, the parametric model h also depends on the parameters and the training dataset.

Thus, it is customary to write h(x) as h(x; w, 𝒟), where w represents the parameters, and 𝒟 is our training dataset.

This dependence is often omitted, but keep in mind that it’s always there.

We make no restrictions about how the model f̂ is constructed. It can be a deterministic function like h(x) = −13.2 x² + 0.92 x + 3.0 or a probability distribution h(x) = P(Y = y ∣ X = x). Models have all kinds of families like generative, discriminative, and more. We’ll talk about them in detail; in fact, models will be the focal points of the majority of the chapters.

First, let’s focus on the paradigms themselves. There are four major ones:

• supervised learning,

• unsupervised learning,

• semi-supervised learning,

• and reinforcement learning.

What are these?

Supervised learning

The most common paradigm is supervised learning. There, we have inputs 𝐱ᵢ ∈ ℝᵐ and ground truth labels yᵢ that form our training dataset

Although the labels can be anything like numbers or text, they are all available for us. The goal is to construct a function that models the relationship between the input x and the target variable y.

Let’s saddle up and see a couple of examples.

Read more

They are the perfect medium for math-and-code-heavy technical content: they support LaTeX snippets, code execution, and, to top it all, enable interactive exploration. Every time I’m reading a hands-on tutorial about some fancy new framework, I cannot resist the urge to jump into edit mode and break the code in ways no author can think of.

Unfortunately, if you choose to write in Jupyter Notebooks, you either abandon content distribution by platforms such as Substack, LinkedIn, or X (because they don’t support the format) or manually convert the notebooks to satisfy every possible whim of every possible editor.

So, I built a tool that enables writers to publish Jupyter Notebooks on Substack (and other platforms) with a couple of clicks. It’s called NotebookPress, and it solves four major pain points:

• LaTeX rendering,

• code snippets,

• user interactivity,

• and cross-platform compatibility.

Let me give you a t…

Read more

Please welcome , author of the brilliant Substack. I’ve been following his work for years, and I’m honored to have him here for a guest post.

By the way, he is hosting a workshop on February 28th titled “Machine Learning and Generative AI System Design,” and he has kindly offered a 35% discount for readers of The Palindrome.

The code TIVADAR35 is valid until February 24th.”

Reserve Your Seat

Now, I’ll pass the mic to Sairam.

Enjoy!

Cheers,
Tivadar

Read more

Yes, I know that it is February. It took me a bit longer to write this post.

I also know that your inbox is (was) overloaded with triumphant 2025 reviews and grand plans for 2026. To respect your time, here's a no-bullshit summary of what you'll get from The Palindrome this year, and if you want the details, just read on.

All subscribers:

• I'm finishing my Machine Learning From Zero book this year, where we'll implement all the fundamental algorithms from scratch. This'll be the topic for my technical posts.

• I'll do more explainer-style videos, like this one from the Matrices and Graphs post.

Paid subscribers:

• Monthly live workshops, streamed live right here on Substack.

• First workshop: Mathematics of Machine Learning, March 7th, 15:00 - 20:00 CET.

• Second workshop: Neural Networks from Scratch, date TBD.

Read more