Vectorization in theory
Making expressions do more, faster
There’s a pattern in machine learning that blows my mind every time, even though I’ve seen it more than I can count.
Look at this expression:
Yes, I know. You are more than familiar with linear regression; we are not here to discuss that. I want to share a wonderful mathematical principle with you, learning through the example of linear regression.
Depending on what we understand by a, x, b, +, and ·, the expression “ax + b” can either be the very first machine learning model a student encounters or the main component of a powerful neural network.
Its evolution from basic to state-of-the-art was shaped by the two great forces of mathematics:
generalizing the meaning of simple symbols such as + and · to make them do more,
and then abstracting the complex symbols into a-s, b-s, and x-es to keep them simple.
This dance of generalization and abstraction is the essence of mathematics; it’s why we can treat functions as vectors, use matrices as exponents, build the foundations of mathematics by dr…
