01 / The problem
Predict cancellation without leaking the future.
Insurance records arrive over time, so a random train-test split can make performance look better than it will be in practice.
The project predicts policy cancellation status from historical policy records. The central design choice was to preserve chronology, fit preprocessing only on training data, and enrich policies with external context that would have been knowable at the prediction date.
Built for the 2026 New England Statistics Symposium Statathon, with reusable preprocessing, model comparison, and tuning code rather than a single notebook-only submission.
02 / Data design
Policy records in geographic context.
Customer and property
Age, premium, tenure, residence length, household composition, coverage, dwelling type, and claims.
Rural–urban context
ZIP-level commuting-area codes and derived urban/rural groupings.
Hazard and expected loss
County-level flood, hurricane, wildfire, wind, winter-weather, and earthquake measures.
Crime context
Prior-year county burglary, theft, vehicle theft, property crime, and arson rates.
| Source | Join level | Examples | Timing rule |
|---|---|---|---|
| RUCA 2020 | ZIP | Primary RUCA, urban/rural class | Static context |
| FEMA National Risk Index | County/FIPS | Risk, expected loss, hazard fields | Version recorded per run |
| NIBRS | County-year | Property and theft rates | Prefer policy year − 1 |
03 / Methodology
A fitted transformer, not ad hoc cleaning.
Handle missingness
Learn numeric medians on training data, add missingness indicators, and map unseen categories to “Other.”
Stabilize the schema
Persist feature names and ordering so validation and test matrices exactly match training.
Control cardinality
Keep state directly, group rare state–city pairs, and expose post-transform feature dropping.
Compare models
Run LightGBM and XGBoost helpers with validation metrics, saved seeds, feature toggles, and submission generation.
04 / What was delivered
The durable result is the modeling pipeline.
The repository contains a fit/transform preprocessor, LightGBM and XGBoost training helpers, Optuna tuning, county-crime feature construction, detailed preprocessing documentation, and lightweight automated syntax checks.
Medians, vocabularies, and schemas never learn from validation or test
RUCA, FEMA, and NIBRS enrichment
Fitted preprocessing can be serialized and reused for scoring
05 / What is public
No performance numbers are invented here.
Competition data and outputs are not committed.
The public repository intentionally excludes raw competition files, trained models, logs, submissions, and generated artifacts. Because final evaluation results are not versioned publicly, this page documents the methodology and implementation without claiming an accuracy, rank, or business lift.