AI-Driven Microscopic Defect Detection System
An AI-driven defect detection platform I rebuilt for SanDisk with a five-person team, mentored by Fei Zhou. It uses transformer-based segmentation to automatically find and measure microscopic defects in electron-microscope images of semiconductor products, replacing a manual inspection workflow that had become the bottleneck between research and production. The stack is Python, PyTorch, Flask, and Pillow on a Qt desktop client, with AI inference offloaded to a dedicated server so the tool feels responsive even on the largest images.
The problem
SanDisk researchers analyze electron-microscope images of flash memory and semiconductor products to catch microscopic defects before they hit production. The images are dense, each one packed with structures that need to be measured, segmented, and cross-referenced. Doing that inspection by hand was slow, easy to get wrong, and lagged well behind what the fab was actually running.
There was already an internal tool for this. It sort of worked. But bugs piled up, the UI made routine tasks harder than they needed to be, and the AI pieces were bolted on in a way that couldn't scale to newer models. What started as 'fix a few things in the tool' turned into a real workflow problem — no amount of patching was going to make the existing app the right shape for how researchers actually used it.
What good looked like
A few things had to hold true for the rebuild to be worth it:
- Defect detection had to run on transformer-based segmentation, generalizable enough to handle defects, contours, and pattern recognition without a separate pipeline per task.
- Researchers had to be able to zoom, upscale, and annotate directly in the tool without leaving for another app.
- AI computation had to move off the researcher's laptop so a heavy image didn't stall the rest of their work.
- The next team to touch it had to be able to swap out models, tweak the UI, or extend features without one change breaking the other.
Why rebuild instead of patch
The existing tool was close enough to what SanDisk needed that 'just keep fixing it' was a tempting path. But the UI and the AI logic were tangled together in a way that made every fix in one half risk breaking the other, and every new model meant another round of surgery on the frontend. It was slower to keep going than to redraw the seams.
Rebuilding let us settle the parts that were going to keep changing — the models, the annotation tools, and the display filters. It also let us do the boring but important thing of splitting the UI from the logic, so the frontend could ship independently of the inference pipeline.
Conditions and constraints
The people using it are researchers, not engineers, so the tool had to feel like an image viewer first and an AI platform second. The primary loop is: open an image, look at it, run detection, mark up what the model missed, save the result. Anything that got in the way of that loop was overhead.
The images themselves are large and detailed, which drove two of the bigger decisions. Zoom and upscale had to be part of the client experience — researchers need to pull a single well or a snippet of chip-surface text like '4-01' up to 4x without opening a separate tool — and the transformer inference couldn't live on the researcher's machine. Running it on a dedicated server meant a heavy image never froze the UI or blocked the next task.
The stack was chosen for the same reason the tool exists: it's what SanDisk's internal team already uses and can maintain long after handoff.