The Palindrome

The Palindrome

Share this post

The Palindrome
The Palindrome
Computational Graphs and the Forward Pass
Copy link
Facebook
Email
Notes
More

Computational Graphs and the Forward Pass

Reverse-engineering the computational graph interface

Tivadar Danka's avatar
Tivadar Danka
May 23, 2025
∙ Paid
17

Share this post

The Palindrome
The Palindrome
Computational Graphs and the Forward Pass
Copy link
Facebook
Email
Notes
More
1
Share

Now that we understand how to work with computational graphs, it's time to pull back the curtain and see how they work on the inside. After all, this is what we're here for. (At least, it's why I'm writing this.)

A quick recap: a computational graph is the graph representation of a mathematical computation. By considering the order of operations, we can represent any expression in a graph form. For instance, here's the graph representation of ax + b.

It’s not just linear regression. Any neural network, no matter how large, can be described with a computational graph. Our goal is to implement them from scratch to gain a deep understanding of how they work.

The plan:

  • understand how to work with computational graphs,

  • and reverse-engineer the implementation.

For this purpose, I have built mlfz, an educational machine learning library, where computational graphs are represented by the Scalar class. You can simply define graphs by using expressions such as y = a * x + b, and compute the gradient

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Tivadar Danka
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More