Build Your Own ML in JS: a Model that Spots Bad UI

Even Opus, loaded with skills, commands and design-system rules, doesn’t implement a mockup pixel-perfect. The root of the problem is that a screenshot test only tells you “the images don’t match”. What exactly is different remains unclear.

We’ll use this problem to step into machine learning, even if you’ve never touched it before. In JavaScript, we’ll build a small neural network that looks at the reference and the rendered component and names which CSS properties differ: color, font weight, letter spacing, border. No formulas: with stencils, light bulbs and fishing, we’ll figure out how a network sees an image, how it learns and where its data comes from. We’ll train it with TensorFlow.js right inside headless Chromium and plug it into a Playwright test. Most importantly, we’ll learn not to fool ourselves with numbers: I’ll show which metrics we need, how to read them and how to move them.

You’ll leave understanding how neural network training works, with a ready-made recipe for applying it to your own frontend.