“How to get started in machine learning?” is one of the most common questions I get. I have a couple of default answers, but they are based more on my personal experience than on science.

Inspired by this, I’ve mapped out the knowledge graph of machine learning, building a hierarchy of concepts that can guide you from the foundations to the state of the art.

A couple of fascinating patterns have emerged from my journey: the thin spine of mathematics that holds up the entire knowledge graph, the central concepts like gradient descent that enable modern machine learning as we know it, and more.

Here’s the video where I talk about my findings:

Before you go: if you find value in my work, consider upgrading to a paid subscription. Your support is what makes it possible for me to keep creating high-quality educational content like this.

Right now is the best time to join—there’s a 33% discount until the end of April, and you can lock in that price for good.

Upgrade to Paid (33% off)

With a premium subscription, you’ll get access to

• the monthly live workshops,

• the subscriber chat,

• and the archive of 150+ deep dives into mathematics and machine learning.

In any case, I’m grateful you’re here. Your time and attention mean a lot, and The Palindrome wouldn’t exist without you.

Cheers,
Tivadar

I’m finishing up my video about the knowledge graph of machine learning, which will be released next week; I’ll do a live premiere right here on Substack Live, with a discussion after the viewing. (You can join here.)

This video will mark a milestone for me: instead of relying on hand-crafted slides and a presentation-style exposition, I built a scripting engine on top of the knowledge graph explorer that turns a JSON script like

[{
  "at": 0.0,
  "action": "autoRotate",
  "axis": "y",
  "speed": 0.01,
  "duration": 9.3,
  "windDown": 2.0,
  "easing": "linear"
},
{
  "at": 9.3,
  "action": "selectNode",
  "nodeId": "gpt",
  "showPrerequisites": true,
  "showDependents": false,
  "duration": 2.0
}]

into a beautifully rendered video.

Here’s a sneak peek:

The machine learning knowledge graph project could serve as a template for my future content. From now on, I’ll go full multimodal, meaning that I’ll

• build interactive visualizations (such as the knowledge graph explorer),

• then write posts and record videos, aided by the interactive tool.

Now that the current video-in-progress is about to be finished, what’s next?

Read on.

Read more

This week, it’s my pleasure to introduce , a fellow scholar working at the forefront of physics, mathematics, and computer science.

One of the main reasons behind the success of modern science is mathematical modeling, the process of translating complex real-life observations into a language that allows us to generalize, understand, and predict.

If you have enjoyed this post, subscribe to his newsletter , a space dedicated to translating the complexity of the empirical world, from your cells to entire societies, into language that is as simple as possible, though not necessarily simpler.

In 2000, a computer worm named ILOVEYOU infected millions of machines in a single day.

Two years later, a post-apocalyptic horror movie directed by Danny Boyle — 28 days later — was released. A story about how humans turned into zombies after getting infected by a virus.

Despite their stacking differences, both follow the same simple logic: interactions lead to contact, contact allows transmission, and transmission triggers transformation.

The surprising fact? Mathematics can capture that logic with striking simplicity and remarkable explanatory power.

Today we will see this in action with a twofold purpose: by focusing on these two seemingly different contagion processes, we will show how the same math can be fundamental to describe what’s going on — and, more broadly, why modeling matters.

Act 0 – What’s a model?

Mathematical models are not decorations on science: they are thinking tools. They allow us to distill messy realities into a few essential gears, showing how motion in one part turns another. This becomes immediately useful when the world feels too complex to grasp: a model offers a translation from tangled stories to relationships we can reason about, test, and sometimes even control.

However, a good model is not meant to reproduce every detail of reality: its purpose is to be useful, helping us identify what can drive a phenomenon, what can be ignored and where small interventions create large effects. The art of modeling lies in choosing the right simplifications: keeping what matters, discarding what doesn’t.

Compartments: how to turn “stories” into equations

In the vast universe of mathematical models, we will focus on contagion because its complexity can be captured in an elegant way.

Which complexity? Well, we should consider that there are people that take the bus or the metro, go to the cinema or supermarket, kids going to the school, people meeting in companies, at any hour of the day, in different part of a cities, or an entire country. These events actually corresponds to some form of interaction, each one able to trigger the reaction chain: contact → transmission → transformation.

In the animation below, that I have built from the amazing ComplexityExplorables, it is possible to appreciate a computer simulation of this process in a very simplified digital world without buses, metros, supermarkets, schools, just four groups (or communities). Initially, the majority of people are healthy and susceptible (white) to the virus brought by sick and infectious individuals (red). Because of random contacts, the virus can spread among individuals, and their movement can spread the virus from one community to another. If the disease is like a seasonal flu, red individuals turn into recovered (grey) after a while: they get healthy again. And after more time, those ones who recovered can become sick again, and so forth and so on.

However, if once get infected one cannot recover, all individuals will eventually turn red, as towards the end of the simulation, where I have intentionally changed the virus.

Obviously, this complexity is cumbersome: uncountable and often untrackable interactions, different immunological response (e.g., from a few symptoms to high fever) from different people, different behavioral response (e.g., from staying at home to compulsory job presence), just to mention some.

However, when we model epidemics we do not track every individual. We often make the rather simplifying assumption that they are like some labeled/colored particles, and we group them by state (i.e., the color) into compartments.
A compartment is simply a box collecting things that behave the same way: uninfected computers, infected ones, or those that can no longer be infected (like the recovered). Therefore, the story of a contagion then becomes a story of flows between those boxes: one box loses what another gains, exactly as in the illustration below.

Mathematically, we can use a minimal map like this:

where each arrow represents a rate indicating how often a change of state happens. Once the compartments and flows are clearly defined, the equations almost write themselves. The rest is interpretation.

Act I — The cyber epidemic

In May 2000, the ILOVEYOU worm spread through email attachments. One infected computer sent the same message to every contact in its address book; each new victim did the same. It’s a perfect digital analogue of infection.

Since there’s no natural recovery until someone patches (with an antivirus) or isolates the system, the right compartment model for this process is SI: the story is

where S denote uninfected computers, I the infected machines and β is the rate of successful transmission. Furthermore, let’s also forget about the specific interaction patterns of the machines: instead of caring about which computer is exactly connected to which other computer, let’s work under the “well-mixed” assumption that any infected can contact any susceptible. We are now ready to transform the above story into an equation, relating the rate at which the number of infected individuals increase with the other variables:

At the start, almost everyone is uninfected, so S ~ N, where N is the total number of machines in the world:

That single line explains the worm’s explosive global reach: exponential growth until the susceptible pool runs out (or, equivalently, until the S box becomes empty). Control, in this view, is mechanical: reduce β (slower transmission) or reduce S (fewer vulnerable systems).

What happens next? When the number of infected machines increases, we can no longer approximate and we must write S = N - I. The equation turns into:

The last step, obtained after elementary algebra, provides us with an operational way to estimate the fraction of infected machines over time, according to the model:

That’s pretty cool, isn’t it? Overall, because the shape of the two curve is very similar to the observed ones, with the difference that computer scientists responded back by writing an antivirus able to stop the virus. We could model that by adding the R compartment after some time.

Act II — The Anatomy of a Zombie Apocalypse (S–I–Z–R)

Now let’s make it cinematic, but still mathematical. Do you remember the movie “I am legend” starring Will Smith?

Suppose we have four compartments: S (survivors, susceptible humans), I (infected but not yet zombies), Z (zombies, fully infectious) and R (recovered/removed because immune, deceased, or isolated). The zombie flowchart is given by:

a bit more complicated, although very similar in spirit, to our previous example, where each arrow translates into a rate term. The system dynamics reads:

with N = S + I + Z + R. Early on S ~ N (nearly the whole population consists of susceptible individuals), so the basic reproduction number is

If it is larger than 1, the outbreak grows and zombies spread faster than they’re neutralized. Conversely, if it is smaller than 1, it dies out and humanity survives!

Act III — From well-mixed to networked worlds

So far, we’ve imagined a “well-mixed” world, as if everyone could meet everyone. However, real contagion, digital or biological, spreads on networks: a few hubs connect widely, most nodes connect locally. This structure changes everything:

• Ignition: infections that start near hubs explode faster.

• Control: targeting those hubs (email relays, hospitals, influencers) is far more effective than random protection, although they are often harder to locate than one could imagine.

The header image of this post shows the air-transport network: airports connected by flight routes, a primary pathway for moving pathogens between countries. Likewise, social networks provide the contact structure for transmitting disease, and the Internet provides the connectivity that enables the spread of computer viruses.

The same logic explains why ILOVEYOU spread via central mail servers and why social media virality behaves like an epidemic of ideas. In both cases, patching or isolating/immunizing the hubs can dramatically changes the system’s trajectory.

Modeling as “compressed understanding”

Mathematical models are compressed stories about cause and effect, how small interactions scale into global patterns. They do not predict the future with certainty, but they make the invisible visible: how rates, connections and structure shape collective outcomes.

Whether the contagion is viral code, zombies or ideas, the same framework applies (compartments, rates, networks). Once you draw the boxes and trace the arrows, you see where and how to act. What’s the take-home message? This one:

If you have enjoyed this post, subscribe to , a space dedicated to translating the complexity of the empirical world, from your cells to entire societies, into language that is as simple as possible, though not necessarily simpler.

Hey!

I spent my last week building the knowledge graph of machine learning, mapping out every concept and its relations.

With this, you can visualize subfields, the roots of machine learning (surprise: mathematics), the central concepts, and a bunch of other cool stuff, like tracing the prerequisites (yellow edges) and dependents (blue edges) of gradient descent.

Now you can explore them yourself with a little app I built Codex and Claude built. The app is publicly hosted on GitHub pages at https://the-palindrome.github.io/ml-knowledge-graph/, but you can check the source at https://github.com/the-palindrome/ml-knowledge-graph as well.

If this work is valuable to you, consider becoming a paid subscriber. Your support lets me keep building and sharing high-quality content like this.

Subscribe now

You’ll get full access to all paid posts, monthly workshops, and a growing archive of 150+ deep dives into mathematics and machine learning.

Enjoy the graph explorer!

Cheers,
Tivadar

Let’s try something new. In recent months, I started to feel that the weekly publishing schedule takes its toll on the quality of my posts. I want to take more time per post to give you some high-quality technical content. I’m inspired by amazing writers such as Sebastian Raschka () or Cameron R. Wolfe (), who publish once a month, but each post is a work of art.

On the other hand, I miss you. A monthly schedule feels too long, and I have a lot to share with you. Ever since I got my ChatGPT Max subscription with access to Codex, my creativity is out of bounds.

So, here’s a new format. Each week, I’m sending you my unfiltered stream of consciousness, all the projects that I’m currently working on. Think of it as joining me for a coffee, where we talk about all the exciting/revolutionary/insane ideas we have in our minds.

This week, there are two things on my mind: knowledge graphs and Minecraft.

Let’s start with knowledge graphs.

The Complete Map of Machine Learning

If you are a regular reader, you know that one of the most common questions I get is “which part of mathematics do I need to study machine learning?” My default answer, based on my decade of experience, is: a ton of linear algebra, a decent amount of calculus, and a snippet of probability theory.

To be honest, I’m not completely satisfied with my reply. So, I dug deep with my newly found agentic AI-fueled superpower to find what is scientifically backed.

Without further ado: here’s the full knowledge graph of mathematics and machine learning. 2081 nodes, 5149 edges.

(The images are screenshots from the interactive graph explorer I’m building, which will be open source and publicly available.)

Let’s unravel this.

Read more

Yesterday we concluded our first monthly workshop! As promised, here is the full recording, exclusive to paid subscribers.

You can access the Jupyter Notebook lecture notes here: https://github.com/the-palindrome/mathematics-of-machine-learning-workshop

The next monthly workshop is already in the works; it’s going to be the next iteration of the Neural Networks From Scratch course. The tentative date is April 18th, 15:00–19:00 CET, but stay tuned for the announcement, as this date might change.

Thanks again so much for attending!

Cheers,
Tivadar

Read more

I’m finally working on my upcoming book Machine Learning From Zero, the sequel to Mathematics of Machine Learning.

The first chapter has just been finished, which sets the foundations for the neat stuff, like implementing neural networks from scratch. (Which is the core of the book.)

Without further ado, here’s an exclusive preview.

Enjoy!

P.S. I’m writing this book in Jupyter Notebooks, and I turn them into Substack posts with NotebookPress, a tool I’m building to bring technical writing on Substack to the next level. If you write math-and-code-heavy content, you should check it out.

Machine learning is training predictive models from data.

Sure, we can be academic about it and refine the definition of machine learning by looking at the countless of nuances, but that’s not what we are here to do. We are here to understand the core fundamentals of machine learning — the fundamentals that will take you further than anything else.

I believe the only way to aquire deep knowledge of any technical subject is to take it apart and put it back together again. This is what we are here to do.

The fundamental machine learning setup consists of:

1. a dataset, usually coming in the form of input and target variables,

2. a parametric function that models the relation between the input and the target variables,

3. and a loss function that measures the model’s fit to the data.

Let’s start with the data.

To look behind the curtain of machine learning algorithms, we have to precisely formulate the problems that we deal with. Three important parameters determine a machine learning paradigm: the input, the output, and the training data.

All machine learning tasks boil down to finding a model that provides additional insight into the data, i.e., a function f that transforms the input x into the useful representation y. This can be a prediction, an action to take, a high-level feature representation, and many more. We’ll learn about all of them.

Mathematically speaking, the basic machine learning setup consists of:

1. a dataset 𝒟,

2. a function f that describes the true relation between the input and the output,

3. and a parametric model h — also called a hypothesis — that serves as our estimation of f.

Remark. (Common abuses of machine learning notation.)

Note that although the function f only depends on the input x, the parametric model h also depends on the parameters and the training dataset.

Thus, it is customary to write h(x) as h(x; w, 𝒟), where w represents the parameters, and 𝒟 is our training dataset.

This dependence is often omitted, but keep in mind that it’s always there.

We make no restrictions about how the model f̂ is constructed. It can be a deterministic function like h(x) = −13.2 x² + 0.92 x + 3.0 or a probability distribution h(x) = P(Y = y ∣ X = x). Models have all kinds of families like generative, discriminative, and more. We’ll talk about them in detail; in fact, models will be the focal points of the majority of the chapters.

First, let’s focus on the paradigms themselves. There are four major ones:

• supervised learning,

• unsupervised learning,

• semi-supervised learning,

• and reinforcement learning.

What are these?

Supervised learning

The most common paradigm is supervised learning. There, we have inputs 𝐱ᵢ ∈ ℝᵐ and ground truth labels yᵢ that form our training dataset

Although the labels can be anything like numbers or text, they are all available for us. The goal is to construct a function that models the relationship between the input x and the target variable y.

Let’s saddle up and see a couple of examples.

Read more

They are the perfect medium for math-and-code-heavy technical content: they support LaTeX snippets, code execution, and, to top it all, enable interactive exploration. Every time I’m reading a hands-on tutorial about some fancy new framework, I cannot resist the urge to jump into edit mode and break the code in ways no author can think of.

Unfortunately, if you choose to write in Jupyter Notebooks, you either abandon content distribution by platforms such as Substack, LinkedIn, or X (because they don’t support the format) or manually convert the notebooks to satisfy every possible whim of every possible editor.

So, I built a tool that enables writers to publish Jupyter Notebooks on Substack (and other platforms) with a couple of clicks. It’s called NotebookPress, and it solves four major pain points:

• LaTeX rendering,

• code snippets,

• user interactivity,

• and cross-platform compatibility.

Let me give you a t…

Read more

Please welcome , author of the brilliant Substack. I’ve been following his work for years, and I’m honored to have him here for a guest post.

By the way, he is hosting a workshop on February 28th titled “Machine Learning and Generative AI System Design,” and he has kindly offered a 35% discount for readers of The Palindrome.

The code TIVADAR35 is valid until February 24th.”

Reserve Your Seat

Now, I’ll pass the mic to Sairam.

Enjoy!

Cheers,
Tivadar

Read more

Yes, I know that it is February. It took me a bit longer to write this post.

I also know that your inbox is (was) overloaded with triumphant 2025 reviews and grand plans for 2026. To respect your time, here's a no-bullshit summary of what you'll get from The Palindrome this year, and if you want the details, just read on.

All subscribers:

• I'm finishing my Machine Learning From Zero book this year, where we'll implement all the fundamental algorithms from scratch. This'll be the topic for my technical posts.

• I'll do more explainer-style videos, like this one from the Matrices and Graphs post.

Paid subscribers:

• Monthly live workshops, streamed live right here on Substack.

• First workshop: Mathematics of Machine Learning, March 7th, 15:00 - 20:00 CET.

• Second workshop: Neural Networks from Scratch, date TBD.

Read more

Today’s post is a very special one, written by my friend Miklós, whom I met during our PhD years. (Which was more than ten years ago. I feel old.) He is one of the smartest people I know, and he’s been doing impressive research projects since then.

One of his latest projects made the news recently, because the data collection took place on the International Space Station (ISS). This is interesting in itself, but what you rarely see is the “backend” side of science, the stuff that don’t make the news, but makes or breaks a research project of this scale.

What follows is a deep-dive report on the entire lifecycle of a space-bound project:

• grant proposal (moving from a whiteboard to orbit),

• agile problem solving (like jumping hoops to meet Apple Store regulations),

• stakeholder coordination (managing the logistics between international space agencies),

• project management (handling “no second chance” execution under the pressure of shifting launch windo…

Read more

Because of its recursive nature, computing the Fibonacci sequence via brute force is computationally expensive.

However, the Fibonacci numbers have a simple and beautiful closed-form expression written in terms of the golden ratio (φ) and the conjugate golden ratio (ψ).

Read more

How did they compute it?

Let me show you:

Read more

For the longest time, people have told me that I should start sharing my educational content in video format.

I’ve gotten compliments like the fact that my illustrations resemble those of 3B1B, for which I’m extremely honored.

So, after a few years of second-guessing whether I should dive into video content, I’ve finally decided to give it a try…

…and, in the short time I’ve been doing it, the results have amazed me.

Introducing The Palindrome YouTube channel

I’ve had a YouTube account since 2011, but just very recently I started shifting towards the creator side instead of being just a plain consumer.

My goal for the next couple of weeks is simple: every fan-favorite post from The Palindrome newsletter and from my social media posts will be turned into video.

So far, there are three videos uploaded to the channel. These are sort of the greatest hits of my educational content. Those that, no matter how many times I republish them, they always get a lot of engagement and I keep gett…

Read more

I am dead serious. Deep down, this tiny rule is the cog in many methods. Ones that you use every day.

Here is what it is, how it works, and why it is essential:

Read more

Hope that you enjoyed the Holidays! This time, I have a brand new video post for you. I'm really into video making now, so you'll see many more like this one in the future.

Enjoy!

Read more

Here’s a quick compilation of the most popular posts we published on The Palindrome in 2025.

Take a chance to read some of them, in case you missed them, or revisit your favorites.

Here’s the full list:

The 10 most popular posts of 2025

1. The Roadmap of Mathematics for Machine Learning

2. Machine Learning is not just Statistics

3. Your Machine Learning Library

4. The Competitive Programmer’s Introduction to Graph Theory

5. Coding on Paper

6. Representing Graphs

7. The Camel Principle

8. Introduction to Algorithmic Analysis

9. Comparing Algorithms

10. The Most Important Skill You Were Never Taught

I hope you enjoy reading them as much as I enjoyed making them for you.

And, if you have a favorite that didn’t make the list, reply to this email and let me know! I reply to all messages from you.

Read more

In a single variable, the chain rule says that

holds for any two univariate differentiable functions f and g. It’s simple, compact, and user-friendly.

However, the math quickly becomes complicated when we introduce multiple variables. And we have to, because there are billions of parameters to tune.

In the general case, we are interested in the composition of the vector-vector function g: ℝⁿ → ℝᵐ with the vector-scalar function f: ℝᵐ → ℝ. For instance, if we have

• a machine learning model with n parameters x = (x₁, ..., xₙ),

• a training dataset of m samples d₁, ..., dₘ,

• predictions g(x) = (g₁(x, d₁), ..., gₘ(w, dₘ)),

• and a loss function f,

then fitting the model is equivalent to finding the maximum of the composite function h(x) = f(g(x)). (The dependence on the data in g(x) is omitted for simplicity.)

To compute the gradient, that is,

(where ∂ᵢh denotes the partial derivative of the function h with respect to its i-th variable), the chain rule says that

This post is about what’s behind this formula.

Read more

It's not a secret; every mathematician is intimately familiar with the idea. They call it “trivial”, move on to study virtuoso mathematical feats, and spend no time pondering the staggering consequences of such a short computation. It's hidden in plain sight, obscured by simplicity. Here it is:

If you want to hit your target, take more shots.

Thinking back on my life, a disproportionate amount of my most significant “level up” moments are underpinned by a single mathematical formula:

As Goethe said,

“Mathematicians are like Frenchmen: whatever you say to them they translate into their own language and forthwith it is something entirely different.”

This time, we’ll do the opposite: translate a simple mathematical result into a profound life lesson.

I don’t want to be the stereotypical mathematician, so instead of bamboozling you with definitions, theorems, and proofs, let me invite you to an experiment: pick up a six-sided dice, and roll it until you get a 6. (You can use this online dice roll simulator if there are no dice at hand.)

It took me nine attempts, but I got it. You might have gotten lucky in your first attempt, or you had to sit around for fifteen minutes to get one, perhaps cursing the author who convinced you to undertake such a pointless task.

Sorry for that. The lesson is worth the price: it might take a lot of throws, but eventually, you’ll succeed.

I learned this on my skin, not by rolling dice, but by playing Texas Hold ‘em during my spring break, motivated by completing the “Introduction to Probability Theory” course for junior mathematicians, like me at the time. Soon, I saw this idea everywhere.

The more poker hands I played, the more times I was dealt pocket aces.

The more research problems I attempted (and quickly failed), the more I eventually solved.

The more business ideas I tried (and again, quickly failed), the more financial success I had.

The more I posted, the more I went viral.

The more women I swiped right on Tinder, the more matches I got. More matches led to more conversations, which led to more dates, which led to more sex. (I hate to break it to you, but online dating for men works more like e-commerce, instead of a chain of genuine human interactions. Thanks, Tinder.)

The more shots I took, the more I hit the target.

It took me a while to notice the pattern, but it finally clicked. It's what the formula

has been telling me all along.

But what do all of these mean?

Read more

It’s Tivadar from The Palindrome. I’ve been thinking about moving towards video for a while now. This is my first one, which is a video version of a previous post.

Hope you’ll like it!

Cheers,
Tivadar

P.S. Don’t forget that we have a special Holiday discount:

For a limited time, you can get 30% off from the yearly subscription to The Palindrome.

This is not like your typical “holiday sale.” Every new paid member helps us hit specific milestones that I’ll reinvest straight back into The Palindrome.

Here’s the plan for 2026:

• 600 paid subscribers → “Mathematics of Machine Learning” exclusive deep-dive mini course, included in your sub.

• 700 paid subscribers → Another exclusive mini-course (Q4 2026)

  A second mini-course with a different expert/angle. We are building up an extensive “course library” for members.

• 800 paid subscribers → Dedicated Manim animator

  We’ll bring on a Manimator so more articles come with clean, high-quality animations that make the concepts instantly click for you.

• 1,000 paid subscribers → Full-time writer

  This is the big one. With a full-time writer on the team, we can ship more deep dives, more courses, and more structured learning tracks throughout 2026.

Currently, we are at 559 paid members (~56% of our goal).

■■■■■▢▢▢▢▢ 56%

Subscribe now