Coding on Paper
Coding as a process, not a product
When I was a young mathematics student, one of my first university courses was “Introduction to Programming,” where the computer scientists in our faculty threw us, innocent and naive mathematicians, straight into the deep water.
In the Eastern school system, we believe in immediately applying pressure so intense that it weeds out the unworthy faster than they can chug their welcome drinks at the regular semester launch parties. So, fifteen minutes into the first lecture, we were dereferencing pointers left and right, because everything else was trivial. At least, according to our professor. (By the way, this was the same in our math courses. The first sentence of our linear algebra class was “let F be a field, then V is a vector space over F if…,” leaving our heads spinning.)
📌 The Palindrome breaks down advanced math and machine learning concepts with visuals that make everything click.
Join the premium tier to get access to the live courses on Neural Networks from Scratch and Mathematics of Machine Learning.
I remember having classmates so technologically inept that they could hardly manage to navigate in Windows XP (the most popular OS at the time), let alone compile C code. So, the “Introduction” to Programming class felt like a punishment for most of us.
The pinnacles of our gauntlet were the coding on paper exercises, where we had to write functioning C programs — with pen and paper.
Coding on paper is antithetical to how we code in practice. Whenever I’m working on a project, I never progress linearly. I constantly change implementations, refactor, bugfix.
Nowadays, I write most of my code in Jupyter Notebooks, where
I take a shot at implementing the function,
test it on a couple of examples,
and fix all the dumb mistakes I’ve made, perhaps even prompting an LLM or two meanwhile. (Or Googling when I feel old school.)
None of that is available on paper. No autocomplete, no syntax checks, no code formatter.
With pen and paper,
you can’t run the script,
you don’t have autocomplete,
and you can’t change the “code” you’ve written.
It’s linear, uninteractive, and tedious. Coding on paper felt like a punishment.
Is there a point?
I used to believe there wasn’t; now I know there is.
The revelation finally came by drawing analogies, my Number One Tool for observing patterns. Patterns that might be hard to observe in our objects of study, but obvious when looking at analogies.
(Sometimes, we are emotionally charged, like I am about coding on paper, which I used to hate. However, taking another perspective through analogies can help us see clearly.)
How about having an LLM write your essay for you?
Even when generative AI is allowed, you cheat yourself by outsourcing cognition to a language model. The purpose of writing lies not only in the final outcome, ready to transfer your thoughts to other minds, but in forming those thoughts as well!
What is writing? As said by Donald M. Murray in his famous essay “Teach Writing as a Process Not Product”, writing is
“…the process of discovery through language. It is the process of exploration of what we know and what we feel about what we know through language. It is the process of using language to learn about our world, to evaluate what we learn about our world, to communicate what we learn about our world.” — Donald M. Murray
Coding — that is, engineering and building systems — is also a process, not just a product.
Here's the catch: you can only get better at delivering products if you master the process. There are no shortcuts.
“But Tivadar, how to master the process?”, you might ask.
By practice. Lots of practice. And by far the best way to practice is to restrict your tools. To make it as hard as possible.
Have you ever seen an athlete exercise with a mask that limits airflow? Something like this one:
That's just to make things hard, to help them perform even better when the restriction is lifted.
When I was practicing Kung Fu, our trainer instructed us to practice complex moves with our non-dominant hands. This technique is magic: after stumbling around with a wooden stick in my left hand for a couple of minutes, I could effortlessly perform the flashy movements with my right hand. (I'm right-handed.)
Another example: a couple of weeks ago, I wrote about the life lessons that Elden Ring, a particularly difficult video game, taught me. One comment was talking about the air-restriction-training analogue for Elden Ring: “putting constraints on your gameplay is actually a hidden trick to really try a lot of mechanics and equipment.”
By purposefully denying yourself, say, wearing armor, you are forced to get good at reading attack patterns and dodging at the perfect moments. When any boss can one-shot you, there’s no room for error.
Another example (the last one): I want to move towards video content and live lectures, so I started live-streaming a lot on Substack. (This very post was drafted live.)
Going live puts a metric ton of pressure on me, and I already feel my discipline snapping into place. I’m focused on work, prepared for the lectures, and my spoken English is already better.
That's when I realized that coding on paper fits the above. It took me a decade and a half, but I'm finally there.
Coding on paper forces us
to pay attention to details (instead of letting autocomplete do it for you),
to be deliberate (instead of brute forcing our way to a solution through trial and error),
to figure things out on our own (instead of outsourcing our cognitive abilities to an LLM).
Whenever you are alone with a pen and a piece of paper, there’s room for your thoughts.
Now, I know that by restricting our tools, we can build the muscles whose work we offload to tools.
Muscles that will carry all the cognitive load for you later.
Hahaha exactly, faced same problem during my BSc years as a computer Science student. The lecturer would make us write Java code on paper and it was so boring and annoying, but that made us stick to every syntax and when we got the opportunity to write code on the computer we faced less syntax errors.
Hmm, you talking about limiting tools reminds me of that interview last week by Dwarkesh Patel with Richard Sutton. Richard was talking about learning from experience when you have no knowledge of the world. No training or education. Just be an animal start some task and get better at it by making mistakes and improving. The philosophy of reinforcement learning is really soothing when you keep it in your mind when you're making mistakes on a new skill you're developing.
I was learning plastering over the weekend I was making mistakes, particularly losing a lot of plaster onto the floor. It was frustrating but I did remember back to Richard, and it changed my perspective as I was making mistakes. I was seeing each error as an incremental improvement in acquiring a new skill, not another failure in a series of failures.