Initial PuGa toolkit: data layer, econ, depth-aware scan, state sync, plan push

- puga/: cached FIO + PRUNplanner clients, market view with order-book walk,
  econ formulas ported from PRUNplanner (tested against its suite and live FIO),
  saturation model v1 (reviewed by Opus)
- tools/: scan (depth-aware), price, book, chain, state sync, plan_push
  (dry run default, [PuGa]-prefixed plans only), legacy prun_scan/prun_cxarb
- docs/: mechanics (PRUNplanner is source of truth), roadmap, decisions,
  saturation design, archived handoff
- secrets stay in .env (gitignored); ref/ holds PRUNplanner source (ignored)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-18 23:00:21 +02:00
co-authored by Claude Sonnet 5
commit 7a538cb300
35 changed files with 1692 additions and 0 deletions
+50
View File
@@ -0,0 +1,50 @@
# Saturation model design (v1 as implemented; v0 draft below superseded where marked)
Problem: PRUNplanner's ROI Overview (and our old scan) rank recipes as if the market absorbs unlimited output at top-of-book prices. Result: recipes whose market fits 1 to 2 buildings rank first with absurd ROI (e.g. 0.25 day). We need, per recipe and per CX (or universe), a number of buildings N the market can realistically support, and profit computed at prices you would actually get at that N.
## Inputs per material and CX (from `puga/market.py`)
- Book totals: ask, bid, supply (units on sell side), demand (units on buy side).
- Flow: `traded7`, `traded30` = average units traded per day; `vwap7`, `vwap30`.
- Live order book (per order price, quantity) via `market.walk`.
- MM (market maker) prices `mm_buy` (floor: MM buys at this) and `mm_sell` (cap: MM sells at this), when present. Treat as unlimited depth at that price.
## Per-building daily quantities
q_out[m], q_in[m] per building per day at efficiency e (from `econ.production_io`), including wages/upkeep costs per building.
## Capacity: how many buildings can each side support
1. **Flow cap** (steady state): a producer cannot take more than a share s of the market's traded flow without moving price. `N_flow_out[m] = s * traded_ref[m] / q_out[m]`, with traded_ref = min(traded7, traded30) (conservative), s default 0.25 (parameter). Inputs identical: `N_flow_in[m] = s * traded_ref[m] / q_in[m]`.
2. **Stock cap** (short horizon): standing book absorbs a one-time batch: bids for outputs, asks for inputs. Over hold horizon H days (default 7), `N_stock = book_units / (q * H)`. Stock refills through flow, so the effective cap per side is `max(N_flow, N_stock)`? To be decided (flow is the steady-state truth; stock only helps ramp-up). Proposal: cap = N_flow; report N_stock separately as ramp-up buffer.
3. **MM override**: if the output has `mm_buy` (floor with unlimited depth) then N_out is unbounded but priced at mm_buy; if an input has `mm_sell` then its price is capped at mm_sell, unbounded depth.
4. **N* = floor(min over all outputs and inputs of the capacity)**. If N* < 1: flag "thin", still report ROI at N=1 but exclude from default ranking.
## Price at N buildings (price impact)
- Selling: two modes. `instant`: walk bids for the quantity N*q_out*H, avg price. `patient`: price = vwap7 with a discount d(N) that grows with our share of flow: `d = k * (N*q_out/traded_ref)` (k calibrated from book slope, default so that at share s the discount equals the walk-price deficit). Report both; default ranking uses patient.
- Buying inputs: mirror (asks, or vwap7 + premium).
- Net per building n(N) = revenue(N) - inputs(N) - wages - amortised capex (optional). Profit curve for N in 1, 2, 4, ... N*. Report N_opt = argmax total profit N * n(N) with n(N) >= a minimum ROI.
## Output columns
recipe, building, N*, limiting material (which side caps), ROI/day at N=1, ROI/day at N_opt, total profit/day at N_opt, capex at N_opt, price used (mode), thin flag.
## Known weaknesses / questions for review
1. Is flow-share s the right primitive? traded volume is noisy and includes our own competitors' equilibrium; what better estimate of "room for a new producer"?
2. Existing producers: a market with big standing supply relative to flow signals saturation on the producer side (price pressure). Should supply/flow ratio adjust the price or cap?
3. Stock vs flow double counting; horizon H arbitrary.
4. Input side and output side are coupled through prices (our buying raises input prices): first-order only.
5. Patient price discount model is hand-wavy; alternatives.
6. Universe scope: with `--cross` across CX, shipping costs ignored; how to bound.
7. Intermediate goods we produce ourselves (chains) are not modelled here (see `tools/chain.py`).
## Review outcome (Opus, 2026-09-18) and what was implemented in `puga/saturation.py` + `tools/scan.py`
Verdict: flow-share alone is half right. The standing sell queue (supply / flow, in days) is the dominant signal; order-book walking is a red herring for saturation (selling one day of output at N* barely moves the price); what costs money is the bid vs vwap7 haircut. Implemented:
- `tref = min(traded7, traded30)`, with a Poisson lower bound `traded30*(1 - 1.96/sqrt(30*traded30))` when traded30 < 20.
- `N_out = (s*tref/q_out) * min(1, T_q*tref/supply) * min(1, demand/(T_d*tref))`, s=0.25, T_q=T_d=7 days. Limiting output reported.
- Thin flag: `tref < 3*q_out or demand < 7*q_out` (hard exclude; `--show-thin` overrides). NOTE: borderline. HWP BHP 7.2h variant (13.3/day vs tref 39.2) is flagged thin at exactly this threshold while the 8.4h variant passes; the 3x rule is arbitrary and should be revisited.
- Inputs: no flow cap; priced by ask-walk of N*q_in (1 day) and profit falls out negative if the book is shallow.
- Output price: `p_patient` = highest ask level whose units-ahead <= T_w*(tref - produced), clamped to [bid, min(vwap7, vwap30, ask)], T_w = 3 days; falls to the bid when we out-produce flow.
- MM override only when `mm_buy >= 0.9*bid`: unlimited depth at mm_buy.
- Bug fixed: `market.walk` dropped MM orders (`ItemCount: null` means unlimited).
Later refinements (not built): supply-feedback fixpoint (our N raises supply), daily snapshot cache and d(supply)/dt, seller-count crowding (distinct `CompanyCode` per side), use `NarrowPriceBand*`/`WwidePriceBand*` to bound prices, 149/370 AI1 materials have traded7 < 5/day so confidence handling matters, cross-CX with freight, chains. MM-priced materials (SP, AIR, CCD, CBS, RED...) trade AT mm_buy so their volume is the MM absorbing, not player demand.
## Addendum: effective supply (implemented)
Stage 1 of `tools/scan.py` uses flow + demand only; stage 2 (shortlist) fetches the ask book and counts only standing sell orders priced <= 1.25 x vwap7 (`saturation.effective_supply`). Reason: DEC at AI1 showed 2,013 units 'queue' of which 1,501 sat at 50,000 (1.5x vwap), which made the raw queue penalty call the market saturated (N=0.38) when the effective queue gives about 1.7.