Loading...
Fraud detection · MLOps
6.4 million mobile-money transactions, processed with PySpark into strictly point-in-time behavioural features, then scored by XGBoost. The decision threshold is chosen by what fraud actually costs — not by 0.5 — and a scheduled job watches for population drift. The model below runs in your browser.
A held-out slice of transactions the model never trained on, replayed in order and scored as they arrive. Drag the cost slider to change what a manual review is worth, and the threshold — and everything it catches or misses — moves with it.
PaySim zeroes out an account once it has been drained, so the difference between the balances before and after a transaction almost gives the answer away. Models trained on those columns score near-perfectly and learn nothing transferable. This project trains both ways and reports both: the headline is the behavioural model, which never sees the simulator's bookkeeping.
| Feature set | PR-AUC | ROC-AUC |
|---|---|---|
| Behavioural only | — | — |
| Everything | — | — |
Behavioural features only, trained on the first three weeks and scored on the last. PR-AUC is the headline because at a 0.13% fraud rate ROC-AUC flatters models that are useless in practice.
| Model | PR-AUC | ROC-AUC |
|---|---|---|
| Majority-class guess | — | — |
| Logistic regression | — | — |
| Random forest | — | — |
| XGBoost | — | — |
PaySim: 6.36M simulated mobile-money transactions over 31 days, of which roughly one in eight hundred is fraud. Read straight from CSV into Spark, never committed to the repo.
Velocity, amount anomaly and counterparty history, each computed over a window that ends one row before the transaction being scored. A test asserts that appending later transactions never changes an earlier row.
Train on the first ~22 days, tune on the next 4, test on the last 5. Cross-validation expands forward in time rather than shuffling, because shuffling would hand the model the future.
Class weighting via scale_pos_weight, and PR-AUC as the metric that actually moves when a rare class is handled better.
Chosen by expected cost — money lost to missed fraud plus the cost of reviewing what gets flagged — on the validation window, then reported on the test window.
FastAPI in Docker on a free tier, with per-feature attribution from XGBoost pred_contribs. A scheduled job recomputes PSI against the training baseline and publishes it to /drift.