Back to projects

PIT-Scorecard: Point-in-Time Credit Risk Scoring System

0.30 AUC of a standard credit-scorecard model turned out to be leakage, not signal.

SWE (AI/MLOps)Credit RiskTabular MLLightGBMFastAPIDuckDBDocker

Wraps the honest scorecard in a full risk-analytics and serving layer: a DuckDB feature mart standing in for a production warehouse, PSI/CSI drift monitoring with a documented retrain trigger, and a cost-sensitive 3-tier approve/review/decline policy derived from the dataset's own historical unit economics (3.32:1 cost ratio), not an assumed rule of thumb. A FastAPI service returns a probability, points score, decision tier, and plain-English reason codes per applicant, backing both a Swagger /docs page and a Gradio demo that call the identical scoring function. Packaged in Docker and deployed live to a Hugging Face Space, surviving five distinct production issues along the way, including a cross-Python-version scikit-learn pickle break.

The leakage gap: honest scorecard and LightGBM both land near 0.69-0.70 AUC; restoring 38 leakage columns fakes a 0.9999 AUC
Risk-band separation: Band 1 is 93.6% Fully Paid, Band 10 is 42.8% Charged Off, with the observed bad rate overlaid
PSI/CSI drift monitoring: score-level PSI stays under the 0.25 retrain trigger, while int_rate and fico_range_low drift individually in 2018

What I built

  • Built a cost-sensitive 3-tier approve/review/decline policy from the dataset's own historical profit-per-good-loan vs. loss-per-bad-loan numbers (3.32:1 FN:FP cost ratio), not an assumed industry rule of thumb.
  • Set up PSI/CSI drift monitoring with a documented 0.25 retrain trigger; score-level PSI stayed under 0.05 throughout the 2016-2018 test window, while per-feature CSI correctly caught int_rate and fico_range_low drifting in 2018, consistent with real Lending Club policy changes.
  • Served the scorecard through a FastAPI /score endpoint returning a probability, points score, decision tier, and plain-English reason codes per applicant, with a Gradio demo calling the exact same scoring function so the API and the UI can never disagree.
  • Built a DuckDB SQL feature mart as a stand-in for a production warehouse, and a 54-check pytest suite spanning all four pipeline phases plus exact-DPI figure regression tests.
  • Debugged five distinct production issues on the live Hugging Face Space: a Gradio/Uvicorn port race, an SSR Node.js proxy crash, a ZeroGPU hardware requirement surfaced only in a deep API error field, a cross-Python-version scikit-learn pickle break requiring a full retrain, and a missing Swagger /docs route, each verified fixed against the live URL, not just localhost.
  • Packaged the pipeline for Docker self-hosting while adapting the actual deployment to Hugging Face's Gradio-SDK-on-ZeroGPU tier after a pricing change made the originally planned Docker Space tier paid-only mid-project.

Tech stack

SWE (AI/MLOps)Credit RiskTabular MLLightGBMFastAPIDuckDBDocker

Key results

+0.30

Leakage AUC inflation

0.6922

Scorecard OOT AUC

3.32:1

Cost ratio (FN:FP)

2.26M

Loans analyzed

Back to all projects