How to grow a decision tree

Beating neural networks one if-else condition at a time

In the year 2024 (when I am writing this post), two classes of algorithms rule the supervised learning world. You have probably guessed that one is neural networks, the driving force behind powerful tools such as ChatGPT or other generative models such as Stable Diffusion. Neural networks are intricate, mysterious, often enormous in size, and require a bunch of data and computational resources. Moreover, we really can't tell why a neural network prefers one representation or the other; it's a black box.

On the top of machine learning hall of fame, the other entry is the family of decision trees. Opposed to neural nets, they are

• simple as 1-2-3,

• fast as lightning,

• require only a small amount of data,

• and we can precisely tell how they make decisions.

Let's make them a permanent tool under our belt.