Handwritten digit recognition with weighted KNN
A KNN classifier for handwritten digits, plus a k-value optimizer that collapses repeated distance calculations into a single pass.
- Type
- School
- Year
- 2020
- Built with
- Python · KNN · Optimization
Classify a handwritten digit from a 28×28 grid of pixels, each an integer
between 0 and 255. The KNN part is textbook. The part I actually cared about was
the cost of choosing k.
The naive approach re-runs the whole distance calculation for every candidate
value of k. I wrote an optimizer that reduces this to one instance of the
distance computation per unclassified object, then evaluates every candidate k
against those already-computed neighbours.
Try it
Draw a digit below and a weighted KNN guesses it as you go. It votes with the
k training digits closest to yours, each weighted by how close it is, and
shows you which ones they were. Slide k to see the same tradeoff the
optimizer was built to search: a small k trusts one lookalike, a large k
lets more distant digits outvote it.
Draw a digit
Draw with a mouse, finger or pen. With the keyboard: arrow keys move the pen, Space puts it down or lifts it, Delete clears.
Prediction
Waiting for ink
Share of the weighted vote
Votes
- 0
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
Nearest training digits
The closest matches from the training set appear here, closest first.
The training set here is a balanced 750-digit slice of MNIST (75 of each
digit), stored at 4 bits per pixel so the whole thing is about 300 KB. On
held-out MNIST test digits it scores around 86% at k = 1 to 5. Your own
handwriting will do a little worse: a mouse stroke isn’t a pen on paper, and
750 examples is a small fraction of MNIST’s 60,000 training digits.
The full write-up, including the accuracy comparison across k values, is in the
research report.