eriklindernoren/ML-From-Scratch

What it covers

ML-From-Scratch collects bare-bones Python implementations of classic machine learning models and algorithms, written with NumPy rather than a deep learning framework. The goal is transparency, not speed: the code is meant to expose what each algorithm actually computes, so it is a poor choice as a production library and a good one for study and experimentation. The scope runs from linear, ridge, lasso and polynomial regression through decision trees, random forests, gradient boosting, SVM, naive Bayes and k-nearest neighbors, to unsupervised methods (k-means, DBSCAN, PCA, Gaussian mixtures, autoencoders, GANs), a deep Q-network, and a small deep learning stack with dense, convolutional, pooling, batch normalization, dropout and RNN layers. See the full implementations index and the deep learning layer list.

The repository briefly explains this design intent in its About section. It suits students and developers who already know some ML and want to see the mechanics, rather than users looking for tuned, GPU-accelerated training.

Getting started

Installation is a standard setuptools flow from the repository root, and requires a Python environment with a C compiler toolchain available if the package build needs it:

git clone https://github.com/eriklindernoren/ML-From-Scratch
cd ML-From-Scratch
python setup.py install

After installation, each example is a standalone script run from the repository root. A short, low-cost run to confirm the setup is the polynomial regression demo:

python mlfromscratch/examples/polynomial_regression.py

It fits a regularized polynomial regression model to temperature data and renders training progress. Other examples follow the same pattern, e.g. python mlfromscratch/examples/dbscan.py for clustering and python mlfromscratch/examples/convolutional_neural_network.py for digit classification with a CNN. Some demos are heavier or have external dependencies: the deep Q-network example trains on the CartPole-v1 environment from OpenAI gym, and the CNN example reports roughly two minutes of training in the README's captured run.

Usage constraints

The code prioritizes readability over numerical or computational efficiency, so do not expect optimized training loops, GPU support or production-grade robustness. Implementations use NumPy and, for the reinforcement learning demo, OpenAI gym; other examples may require additional libraries that the README does not enumerate per script. A 2017 discussion on the project noted that computing a matrix inverse to solve a linear system is numerically inferior to QR/SVD or a least-squares solver; the author acknowledged the point at the time. Treat the code as reference material and check numerical details before reusing it in anything sensitive. The project is MIT licensed.

HotRepo's take

The project drew a large early Hacker News discussion in February 2017 (694 points, 63 comments). In that thread, readers asked for method-level documentation and explanations alongside the working code, and the author responded that more documentation and references would follow. One commenter also pointed out the matrix-inverse issue above as a concrete numerical improvement. These are useful signals about what the repository is and is not: a widely read teaching artifact whose value lies in transparency, with known rough edges that the author himself acknowledged. On HotRepo, the only recorded daily Python-list snapshot is from 2026-11-01 at 36 stars gained and position 16; there is no all-language daily rank on record, so that single snapshot should not be read as a trend.