The Palindrome

The Palindrome

The Idea That Makes Derivatives Click

Derivatives are approximations

Tivadar Danka's avatar
Tivadar Danka
Nov 17, 2026
∙ Paid

Hey! It’s Tivadar from The Palindrome.

If you are a regular reader, you know that derivatives are 1) close to my heart, and most importantly, 2) play an essential role in machine learning. We can’t talk about them enough.

There’s an idea that’s like a cheat code for understanding what derivatives are really doing, and it’s almost never taught. So, I wrote a post about it.

As usual, the video version is upcoming as well, premiering on November 20th. From now on, the written versions (published a couple of days before the video premiere) will be exclusive to paid subscribers, and the video will be free for everyone.

Paid subscribers also get:

  • a special thanks and mention in the posts and videos,

  • and the full archive of 200+ math-and-machine-learning-packed posts, also available as a beautifully rendered 1000+ page book.

If you enjoy my content, a paid subscription is the best way to support me. I do this full time, and your support enables me to keep democratizing mathematics and machine learning, one post at a time.

Enjoy the post, and see you soon!

Cheers,
Tivadar


The Idea That Makes Derivatives Click

You probably first met derivatives as limits of difference quotients, or as the slopes of tangent lines. This is even reflected by the df / dx notation.

Difference quotient and tangent slope

But if we zoom in where the tangent line and the function meet, we can truly see what the derivative really is. Near that point, the function and its tangent line line up almost perfectly. If we zoom in enough, it’s almost like we can substitute one for the other.

This is not an accident. The derivative gives the best local linear approximation of the function. This perspective makes the chain rule feel natural, carries over to higher dimensions, and leads naturally to Taylor approximations.

Derivatives are approximations; this is the idea that makes derivatives click.

What is a derivative?

Let’s start at the beginning. What is a derivative? Forget about the function for now; let’s talk about a simple dot-like object, moving along a line, bouncing up and down within its single degree of freedom.

We can visualize its motion in a time-distance plot, where the x axis represents time, and the y axis represents the position.

Its time-position graph

To calculate the average speed of our moving object on, say, the time interval [0, 2], we can simply approximate the trajectory with constant velocity motion, then compute the ratio of the distance travelled (denoted by delta f) and the time it took (denoted by delta t). This is also the slope of the line that is the graph of our approximating constant velocity motion.

Average speed as a secant slope

But how do we calculate the exact speed at t = 0? Let’s shift our perspective a bit and get rid of the y-axis for clarity.

If the trajectory is smooth, then as the interval Δ t shrinks to zero, the ratio Δ f / Δ t converges to the speed of the moving dot at t = 0.

This is the definition of the derivative. By playing around with Δ t, we can express the same quantity in a couple of different ways, which is useful on certain occasions, but none of them express the idea I talked about earlier: that derivatives are approximations.

User's avatar

Continue reading this post for free, courtesy of Tivadar Danka.

Or purchase a paid subscription.
© 2026 Tivadar Danka · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for writing, podcasting, and media founders.