Every year, doctors examine thousands of breast lumps to decide whether they are malignant (cancer) or benign (not cancer). Getting it right matters: catching cancer early saves lives, but a missed diagnosis can be fatal.
This 10-day research project asks a simple question: can machine learning tell the difference, and could we trust it if it did?
Using real data from 569 patients, I explored how cancer cells differ from healthy ones, trained and compared four AI models, tested where they fail, and built CellScan, an interactive app that lets anyone explore the AI.
Educational project only. Not a medical diagnostic tool.
The Data
The data comes from the Wisconsin Diagnostic Breast Cancer dataset, collected at the University of Wisconsin Hospitals in the early 1990s and published by the UCI Machine Learning Repository.
Doctors took cells from each patient's lump using a thin needle (a fine-needle aspirate), photographed them under a microscope, and a computer measured the nucleus of each cell.
- 569 real patients: 357 benign, 212 malignant
- 30 measurements per patient: size, texture, dents, smoothness, symmetry and more
- Each measurement recorded for the average cell, the variation between cells, and the worst cells
The patients were split into 455 for training (for the AI to learn from) and 114 for testing (patients the AI had never seen), to check whether it had really learned rather than memorised.
First Discoveries
Before using any AI, I explored the data with graphs.
Cancer nuclei were on average much bigger (average radius 17.5 vs 12.1) and around 3.5× more dented than benign ones.
But there was an overlap zone: some cancers had small cells, and some harmless lumps had large ones. No single measurement could separate them perfectly, but combining size and dents separated them much better.
Human vs Machine
My rule: "If the nucleus radius is over 15, it's cancer." It scored 86.8% accuracy.
The AI (Logistic Regression): trained on 455 patients using all 30 measurements. It scored 93.9% accuracy.
The AI won because it combined many clues at once. It caught cancers with small but irregularly shaped cells that my one-measurement rule missed.
Why Accuracy Isn't Enough
93.9% sounds impressive, but looking closer, the AI had missed 6 cancers, telling people who had cancer that they were fine. It caught only 85.7% of cancers.
| Mistake | What it means | How serious? |
|---|---|---|
| Missed cancer | Someone with cancer is told they're fine | Very dangerous: the cancer can spread |
| False alarm | A healthy person is told they might have cancer | Stressful, but further tests would clear them |
From this point on, the project focused on reducing missed cancers, not just raising accuracy.
The Model Competition
Four AI models were trained and compared:
- Logistic Regression: a points system, where each measurement adds or subtracts points towards cancer
- Decision Tree: a chain of yes/no questions
- k-Nearest Neighbours: copies the diagnosis of the most similar past patients
- Random Forest: 100 decision trees that vote
On a single test, Random Forest looked best. But a single test can be lucky, so all four were tested fairly using cross-validation (5 different groups of test patients, averaged).
| Model | Accuracy | Cancers caught | AUC |
|---|---|---|---|
| Logistic Regression | 98.1% | 96.2% | 0.995 |
| k-Nearest Neighbours | 96.5% | 93.4% | 0.992 |
| Random Forest | 95.6% | 93.0% | 0.991 |
| Decision Tree | 91.7% | 91.1% | 0.916 |
AUC measures overall performance: 1.0 is perfect, 0.5 is random guessing.
The Caution Line
The AI gives each patient a % chance of cancer. Normally, anything over 50% is labelled cancer. But because missing a cancer is so dangerous, the line can be lowered to make the AI more cautious.
| Caution line | Missed cancers | False alarms |
|---|---|---|
| 50% | 3 | 1 |
| 30% | 1 | 1 |
| 10% | 1 | 5 |
At 30%, the best model missed just 1 cancer with 1 false alarm out of 114 patients. The "best" setting depends on what matters most in the real world.
What the AI Is Looking At
The models relied most on the size, dents and texture of the worst cells, and each matches real cancer biology:
| Feature | Why it differs in cancer |
|---|---|
| Size | Cancer cells often have extra chromosomes (aneuploidy), so the nucleus grows |
| Dents | Faulty lamins, the nucleus's internal scaffolding, make its outline lumpy |
| Texture | DNA (chromatin) clumps unevenly, making the nucleus look patchy |
The worst cells mattered most because a few cancer cells can be hidden in an average.
Important ≠ causes. Large, dented nuclei are a sign of cancer, not its cause, like smoke from a fire.
Trying to Break the AI
A good scientist tries to prove their own model wrong. These experiments revealed serious weaknesses:
- Luck: across 10 random test groups, missed cancers ranged from 1 to 8
- Borderline cancers: every missed cancer had cells only slightly abnormal, possibly early cancers, where catching them matters most
- Confidently wrong: one real cancer patient was given only a 10% chance of cancer
- A different microscope: making measurements just 10% smaller tripled the missed cancers (3 → 10)
Not Ready for Hospitals
Despite strong results, this AI should not be used on real patients:
- Trained on only 569 patients, from one hospital, over 30 years ago
- Never tested at another hospital
- Small differences in equipment could make it miss many more cancers
- Real doctors also use age, family history, scans and symptoms
What would be needed next:
- Much more data from many hospitals and countries
- Testing on completely new patients (external validation)
- Making the model robust to different equipment
- AI that analyses the microscope images directly
CellScan
CellScan brings the whole project together in one interactive app:
- Measurements: set a patient's cell measurements, or load a real patient
- Caution line: choose how cautious the AI should be
- Trade-off and 2nd opinion: see missed cancers vs false alarms, and compare with a second AI
- Previous patients: browse 114 real patients and see where the AI got it right or wrong
- Why?: see which measurements pushed each decision
- Challenge: try to beat the AI
Conclusion
Can machine learning tell malignant from benign tumours? On this dataset, yes, very well. The best model caught about 96% of cancers, using features that make real biological sense.
But the most important lesson is that building an AI is the easy part. Proving it can be trusted is much harder. A high accuracy score can hide dangerous mistakes, results can depend on luck, and a model that works in one hospital might fail in another.
Medical AI has huge potential, but only when it is tested carefully, honestly and on the right data.
Data: Wisconsin Diagnostic Breast Cancer dataset (Wolberg, Street & Mangasarian), UCI Machine Learning Repository. Built with Python, scikit-learn and Gradio.