Technical specification
The methodology, stated plainly enough to be checked.
Every claim on this page is reproducible from the public repository with three commands and no credentials. Where the system is weak, this page says so.
Architecture
Four modules, one direction of flow.
SOURCES
ICES DATRAS
ECMWF ERA5
Copernicus Marine
EMODnet
INGEST
datras.py
era5.py
copernicus.py
Streamed parsing, resumable, quota-aware
FEATURES
build.py
True-zero join, leakage quarantine, cyclical season terms
MODELS
baseline.py
XGBoost classifier + regressor, forward-in-time scoring
Leakage control
The most tempting features are the ones we throw away.
Each haul record carries shipboard measurements taken at the moment the net went in: surface and bottom temperature, surface and bottom salinity. They correlate strongly with catch and would lift every metric on this site.
They are excluded. A skipper asking for a forecast is still in port; that reading does not exist yet. A model trained on it would score beautifully in validation and fail at sea. The columns are kept aside for a different job — checking modelled fields against ground truth.
ADMITTED — 7 FEATURES
QUARANTINED — MEASURED AT HAUL TIME
QUEUED — CMEMS INTEGRATION
tests/test_features.py::
test_in_situ_measurements_are_never_features
Validation protocol
Forward in time, or it does not count.
Two hauls taken minutes apart on the same survey leg carry almost the same information. Split them randomly between train and test and the model copies rather than predicts. Splitting on time removes that path entirely.
Distribution shift is shown, not hidden
Sprat prevalence rises from 69.9% to 85.9% between the two periods as the stock grew. A time split exposes that; a random split would have averaged it away.
Baselines are published alongside
Every R² is reported next to the mean-predictor score on the same test set (−0.006 to −0.161), so the reader can see what the model is beating.
Undefined is reported as undefined
When a lift ratio's denominator approaches zero the figure is suppressed rather than printed. A "6000×" headline says nothing except that the divisor was zero.
Stack
Deliberately unexotic.
| Language | Python 3.11+ |
| Data | pandas, NumPy |
| Models | XGBoost, scikit-learn metrics |
| Geospatial | xarray, Zarr / NetCDF — ingest only |
| Tests | pytest — 9 invariants of the modelling table |
| Dependencies | 5 direct, all mainstream |
Gradient boosting on tabular features outperforms deep architectures at this sample size. A more complex model enters the product only when it demonstrably scores better — not because it sounds better.
Reproducibility
Three commands, no credentials.
# 1 — labelled catch records $ python scripts/download_survey_data.py # 2 — meteorological fields $ python scripts/download_weather.py # 3 — fit and score $ python scripts/train_baseline.py → reports/baseline_baltic.json
No raw data is versioned — the scripts rebuild every dataset from source.
Downloads resume after a quota interruption instead of restarting.
The metrics report records the features actually consumed, not those requested.
Known limits
What this system cannot do yet.
A specification that lists only strengths is marketing. These are the constraints a technical evaluator would find, stated first.
No oceanographic signal in production yet
Temperature, salinity, chlorophyll and current fields are not yet integrated into the fitted models. Published metrics are a floor.
No temporal memory
Each prediction uses conditions at a single instant. How long a front has persisted — often the thing that concentrates fish — is not represented.
Survey seasons, not the full year
Baltic surveys run in quarters 1 and 4. Labels outside those windows are sparse, so summer forecasts carry wider uncertainty.
Gridded inference not shipped
Models score haul positions today. Producing a continuous basin-wide surface requires the bathymetry layer and is in progress.