dodox/PuGa
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Docs: register today's fixes, prune completed roadmap items

Browse Source 
CLAUDE.md/docs/library.md updated to match the sell.py/parkcash.py/
scan.py changes in the preceding commits. docs/decisions.md gets the
full reasoning trail: the Black-Scholes closed form marked superseded,
the actual backtest entry (BHP/AL touch-probability table) that both
sell.py and library.md already cited but was never written, the
discretization-bias caveat on that backtest, the price-impact-cap and
demolition-value decisions. docs/roadmap.md pruned back to open items
only - finished work is discoverable from CLAUDE.md/library.md/
decisions.md instead of staying listed as a done backlog entry.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
dodox
2026-10-10 23:07:02 +02:00
co-authored by Claude Sonnet 5
parent 3ee26e5a85
commit 12d26c5d68
4 changed files with 210 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+4 -3
View File
@@ -43,12 +43,12 @@ Run as `puga <tool> ...` (e.g. `puga scan --min-n 3`, `puga plan pull <uuid>`);
- `tools/state.py sync|show` refreshes `empire/state/company.yaml` from live FIO (cash, permits, buildings, real production efficiency, storage, ships).
- `tools/plan_push.py` builds/validates PRUNplanner plans from YAML specs (`plans/examples/`, `empire/plans/`). Dry run by default; `--apply` only after the player says yes; only `[PuGa]`-named plans are created/updated/deleted (`delete <uuid>`, only when asked); `list` shows the account's plans, `pull <uuid> -o file.yaml` reads a plan back into a YAML spec including edits made in the PRUNplanner UI. Workflow: push a plan, the player refines it in the UI and saves, then `pull` / `simulate --uuid` read it back and you continue from there.
- `tools/simulate.py <spec.yaml | --uuid U> [--off EXT,SME] [--basis real|uni30|vwap30|ask|...]` replica of PRUNplanner's simulator (efficiency, workforce, material I/O, profit). Flows and efficiency verified exact vs screenshots; profit within ~2%. The NEW CAPEX line = planned minus already built (from state), the honest payback; ignore the plan-level ROI (PRUNplanner always adds a core module). `--uuid` reads the SAVED plan from the account (UI edits must be saved first).
- `tools/scan.py` DEPTH-AWARE recipe scan (use this): buildings the market could absorb, ROI for our own size, patient prices, ask-walked inputs. Fully staffed AND understaffed variants by default (`--staffing`), freight netted out (`--trip-cost`, `--cargo`), HQ/COGC/experts/faction, `--planet ID` adds that planet's extraction, fertility and COGC (a planet not in state is a new base: no HQ, permits+1), `--deprec 60` prices demolish-later. `--min-n` is ONLY a noise filter on market capacity (use 3, ideally 10); ROI is always for our own size (`--own`, default 1 building), never at the filtered scale. Below 1 lets sub-building junk in. `--json rows.json` feeds `persistence.py`. Library: `puga/saturation.py`.
- `tools/scan.py` DEPTH-AWARE recipe scan (use this): buildings the market could absorb, ROI for our own size, patient prices, ask-walked inputs. Fully staffed AND understaffed variants by default (`--staffing`), freight netted out (`--trip-cost`, `--cargo`), HQ/COGC/experts/faction, `--planet ID` adds that planet's extraction, fertility and COGC (a planet not in state is a new base: no HQ, permits+1), `--deprec 60 [--reclaim-frac 0.65]` prices demolish-later: amortizes the UNRECLAIMED capex (not full capex - real FIO reclaim data shows 50-86% comes back even at near-100% condition, see `docs/decisions.md`) over the holding period. `--min-n` is ONLY a noise filter on market capacity (use 3, ideally 10); ROI is always for our own size (`--own`, default 1 building), never at the filtered scale. Below 1 lets sub-building junk in. `--json rows.json` feeds `persistence.py`. `--sort area` ranks by profit/day per tile of BUILDING area (FIO `AreaCost`, not hab area) - once an existing base's area cap binds, this is the real allocation criterion, not %-yield/payback (a higher-%% recipe needing much more area can lose to a lower-%% one that fits more densely). `--max-impact 10` (stage 2 only) flags `[exit-thin]` when the LIVE bid book can't absorb even 1 day of 1 building's output within that %% (`puga.saturation.max_units_at_impact`) - an INSTANT-DUMP/panic-exit liquidity check, not production price impact (that's what `market`/`lim` already measure via real 30-day flow); a single snapshot (orders change constantly), so it's a WARNING, not folded into `market`/`lim`. Library: `puga/saturation.py`.
- `tools/history.py TICKER` monthly margin history of the recipe producing TICKER; `tools/persistence.py rows.json` re-prices scan rows over history (mean ROI 7/14/30/90/180d, payback, net gain over 7 and 14 days, % days profitable). ALWAYS check persistence before recommending: current margins are often a spike.
- `tools/buy.py <spec.yaml|--uuid U> --days N` cash-aware shopping list: construction gap (plan vs empire state's built buildings) plus N days of the plan's NET operating stock (via `simulate()`, so self-produced inputs like AL net against consumption), priced at real fill cost, checked against cash/reserve. Pass a plan covering the WHOLE base for restocking, not just a new addition (see `docs/library.md`).
- `tools/sell.py TICKER QTY [--tight]` where to post an ask, expected hours to clear, and a self-contained tranche split: aggressive (undercut) tranche capped at the median 30d-volume quantile (or 80th pct with `--tight`, for when a payment is imminent and stockout risk outweighs a few points of margin), patient tranche priced just under the next competitor tier. `--show-bid` for the instant-bid comparison (off by default; usually worse).
- `tools/sell.py TICKER QTY [--tight]` where to post an ask, expected hours to clear, and a self-contained tranche split: aggressive (undercut) tranche capped at the median 30d-volume quantile (or 80th pct with `--tight`, for when a payment is imminent and stockout risk outweighs a few points of margin), patient tranche priced just under the next competitor tier. `--show-bid` for the instant-bid comparison (off by default; usually worse). `--tiers [--rate R]` answers "is it ever worth pricing above the current market": combines queue-depth days (real resting quantity, within the traded range) with an EMPIRICAL first-passage estimate from the ticker's own multi-year price history (no random-walk assumption - a driftless-walk closed form was tried first and backtested wrong, see `docs/decisions.md`'s "Optimal ask price" entry), via max (the binding constraint). `rate` should be the player's REALIZED capital growth rate (read off their equity-growth chart, default 0.05), not a theoretical build ROI. Library: `puga/history.py::empirical_touch()`.
- `tools/network.py <chain.yaml>` models a MULTI-BASE chain (e.g. mine LST at Nike, ship it, make BSE at Deimos): simulates each base, nets a transferred material's producer-surplus against consumer-need, charges real freight only on what's moved (`puga/network.py`). Spec and worked example in `plans/chains/`. `cm_free: true` per base for a new founding covered by a Core Module Kit. profit/day per base is the WHOLE plan, not the marginal addition - diff against the base's plan without the addition for a true marginal ROI.
- `tools/whatif.py <before.yaml> [--add BUILDING:AMOUNT[:RECIPE]] [--infra HB1:+1] [--after after.yaml]` marginal payback of adding to a base you already run (whole-plan diff, same rule as buy/runway - pass the WHOLE base, not just the addition). `--add` mutates a copy of `before` in-memory (reuses an existing building's recipe if omitted, e.g. `--add POL:1`); `--after` takes a second full spec instead. Flags any workforce tier the addition newly overloads (vs. a pre-existing one, which it won't flag) - this is what caught the PP2/pioneer-housing and 2nd-CHP/pioneer-housing surprises by hand before this tool existed. Charges marginal freight too (real fuel model, CX-to-planet, via `puga.shipping`): only the EXTRA round trips this addition needs beyond the base's existing traffic, so a small addition riding along on already-necessary hauls correctly costs ~0, while one that meaningfully adds to shipped tonnage doesn't get a free pass - this is what would catch a recipe whose ROI looks great only because its freight cost was left out.
- `tools/whatif.py <before.yaml> [--add BUILDING:AMOUNT[:RECIPE]] [--infra HB1:+1] [--after after.yaml]` marginal payback of adding to a base you already run (whole-plan diff, same rule as buy/runway - pass the WHOLE base, not just the addition). `--add` mutates a copy of `before` in-memory (reuses an existing building's recipe if omitted, e.g. `--add POL:1`); `--after` takes a second full spec instead. Flags any workforce tier the addition newly overloads (vs. a pre-existing one, which it won't flag) - this is what caught the PP2/pioneer-housing and 2nd-CHP/pioneer-housing surprises by hand before this tool existed. Charges marginal freight too (real fuel model, CX-to-planet, via `puga.shipping`): a CONTINUOUS amortized rate (extra tonnage/cargo-hold x round-trip cost), scaling with the addition's own tonnage, not a step function - a small addition adds a small continuous freight cost, not exactly zero, and a recipe whose ROI looks great only because its freight cost was left out still gets caught.
- `tools/runway.py <spec.yaml|--uuid U> [--no-inbound]` days of runway per material before a base stalls: real NET consumption via `simulate()` (self-produced inputs excluded), base storage, PLUS (by default) any ship currently flying to that planet, auto-detected from live flight data (`tools/state.py` tags each ship with its flight origin/destination). Pass the WHOLE base plan (see `docs/library.md`).
- `tools/undervalued.py [--cx AI1] [--min-vol 5] [--top 20]` manufactured goods currently priced below implied production cost - buy-and-hold-for-reversion candidates, not build candidates. **Expected ROI is much lower than production builds** (~0.10-0.24%/day backtested, vs 14-29%/day for the builds this toolkit recommends) - only worth it for excess idle capital once better-yielding builds are exhausted. Library: `puga/undervalued.py`.
- `tools/chain.py TICKER` make-vs-buy cost tree plus sourcing depth of inputs. `--consumers` flips it: what recipes EAT this ticker as an input, with each output's market depth - for "is there a deeper downstream product worth chaining into" questions (e.g. checking BOR/BSE/BBH for further processing).
@@ -56,6 +56,7 @@ Run as `puga <tool> ...` (e.g. `puga scan --min-n 3`, `puga plan pull <uuid>`);
- `tools/route.py <from planet/system> <to planet/system>` jump-distance route (not a PRUNplanner feature - built from FIO's `/systemstars` graph, Dijkstra by straight-line ly since that's what FTL fuel/time scale with, jump count reported alongside). Accepts planet natural IDs, system natural IDs, or names. Library: `puga/route.py`.
- `tools/shiptrip.py SHIP FROM TO [--v0 X --v1 Y]` the simple ship-trip tool: give a ship registration + start + stop, it works out jumps/charges/landing fuel itself via `tools/route.py`'s jump graph and reports fuel cost (defaults to max sliders). Direct-jump routes only - an unconnected pair means gateway-only, and it tells you exactly what to read off the live APEX Ship Flight Control screen (the LOCK fee - genuinely not available via FIO/PRUNplanner, confirmed by inspecting both) and what to run instead (`tools/shipcost.py`). Warns if the given ship's live specs (FIO `/ship/ships/{user}`) don't match the calibrated hull/reactor.
- `tools/shipcost.py [--jumps N --charges N] [--v0 X --v1 Y] [--gateways N --lock-fee AMOUNT:CURRENCY] [--to PLANET]` the explicit/manual version: fuel/AIC cost of a ship trip from physics parameters you already know (jump count, slider values, gateway LOCK fees read off the live game) - use `tools/shiptrip.py` instead if you just want "ship X from A to B". Landing fuel is a measured-planet lookup via `--to`, independent of whether the route used jumps or gateways. Both tools scoped to this account's one hull/reactor (blueprint BP-STRT-0000, shared by both ships AVI-07JFB/AVI-07JFC); open caveats (v0 relative to current fuel load, landing fuel confounded with gravity/distance, no second hull to cross-check) are in `puga/shipping.py`'s module docstring - don't trust magnitudes beyond what's there. Raw calibration data (APEX DOM reads, 2026-10-09): `empire/data/shipping_calibration/`. Also wired into `tools/network.py`'s freight cost (direct-jump transfers use the real model; gateway transfers need an explicit `trip_cost` in the chain YAML, else a 9250 AIC fallback). Library: `puga/shipping.py` (`trip_fuel()`, `round_trip_cost()`).
- `tools/parkcash.py [--min-spread X] [--min-trade-days N] [--rate R] [--top N]` where to park idle cash as a buy-low/sell-high position (not a build) while ships are busy or cash has no better use right now - scans every ticker for a big bid/ask spread AND real trading liquidity (trade-days/gap over a calendar window - a huge spread with few trading days is a stale-quote trap, confirmed by hand: AST showed a 130% spread but its resting ask had gone stale after a 60% real price crash). Round-trip wait/value is EMPIRICAL (`puga.history.empirical_touch`, same model as `tools/sell.py --tiers`), not an assumed random walk - a driftless-walk version was tried and backtested wrong (overstated reachability 2-4x vs 3 years of real data, see `docs/decisions.md`). This dropped the whole universe's top pick from ~94%/day to ~1%/day with real waits of 19-194+ days - treat any number from the old model as stale. `--rate` defaults to 0.0 (matches the idle-cash premise; pass your realized growth rate if this cash IS earmarked) and shows up as a separate `discounted_pct` column (margin% x real odds both legs ever happen) alongside the simple non-compounding `daily_pct` used for ranking. Library: `puga/parkcash.py`.
- `tools/lmscan.py PLANET [--direction source|destination] [--min-margin X]` live Local Market hauling-ad scan (FIO, public/no-auth - real player-posted freight jobs, not the CONTD player-contract system), ranked by margin after real fuel cost (`puga.shipping.leg_cost`). One-way cost only - margin assumes the next job is picked up near the destination, not an empty flight home. A small measured gateway-fee table covers ANT<->HRT/BEN; other gateway routes can't be priced (LOCK fee isn't available via any API). Library: `puga/contracts.py`.
- `tools/price.py` prices across exchanges with VWAP, volume, fill price for a quantity; `tools/book.py` order-book ladder.
- `tools/prun_scan.py`, `tools/prun_cxarb.py` legacy top-of-book scans; overstate thin markets. Baseline only. `tools/arb.py` replacement is on the roadmap.
docs/decisions.md
+138
View File
@@ -17,6 +17,144 @@
- If a bigger-model spawn is refused, tell the player and ask them to run the review.
- Routine tool building is not reviewed by a bigger model.
## 2026-10-10 Optimal ask price: first-passage-time (Black-Scholes), not order-book depth
**SUPERSEDED same day - see "Black-Scholes model backtested wrong" below. The closed form here
(`daily_log_vol()`, a driftless-random-walk assumption) is no longer what `tools/sell.py --tiers`
uses; kept for the reasoning trail, not as current guidance.**
- The naive model ("units ahead of me in the book ÷ historical daily volume = days to clear")
is WRONG: it assumes the book is static (no one else reprices, cancels, or adds orders while
you wait) and that historical trade volume at normal prices still applies at a resting ask far
outside the recently-traded range. Tried first, broke immediately: fed a fat-finger 1,000,000
AIC ask the same reachability as the real book, "confirming" it as optimal.
- Correct model: whether the market ever pays price P is governed by price DIFFUSION, not queue
depth. Model ln(price) as a driftless random walk with the ticker's own measured daily
log-return volatility sigma (deliberately no drift - same "don't model trend" rule as
`puga/parkcash.py`), and ask for the discounted value of the first time it diffuses up to P.
Closed form (standard Brownian-motion hitting-time Laplace transform):
y = ln(P/s0), rho = ln(1 + rate), D(P) = P * exp(-(y/sigma) * sqrt(2*rho)) for P > s0
D(P) = P for P <= s0
`rate` is the player's own REALIZED capital growth rate (opportunity cost of the delay - see
below), not a theoretical marginal-build ROI.
- ln(D) is LINEAR in y, so there is no interior optimum: D is strictly decreasing past the
current price (sell near top of book / at the last traded price wins) whenever
`sigma < sqrt(2*rho)`, and rises without bound (a degenerate corner case, not real free money)
otherwise. Checked against this account's real tickers (AL 2.5%/day, BHP 8.1%/day, DEC
10.8%/day vol, all against a 5%/day rate -> threshold ~31%/day): every one is comfortably in
the "sell now dominates" regime. Treat a result where climbing pays as a signal to double
check sigma/rate inputs, not as license to post a speculative high ask.
- Implementation: `puga.history.daily_log_vol()` (pooled variance estimator, scales each
consecutive trading-day gap's squared log-return by its elapsed days so sparse tickers don't
understate how far price can move between trades) + `tools/sell.py --tiers --rate R`.
- This also resolved the earlier "what discount rate" question: use the player's REALIZED equity
growth rate (read off their own in-game equity-growth chart, ~5%/day as of 2026-10-10), not the
scan/build tools' marginal per-building ROI figures (14-29%/day) - those are a ceiling on one
marginal building at current scale (capped by permits, workforce-arrival delay, and market-
depth saturation the moment you scale), not what a freed-up dollar actually compounds at. The
realized rate already nets out all of that friction, which is exactly what a discount rate for
"should I wait for a better price" needs to reflect.
## 2026-10-10 Black-Scholes closed form backtested wrong - empirical model instead
- Tested the driftless-random-walk assumption above against this account's real exchange history
(BHP: 802 trading days, sigma 7.39%/day; AL: 995 days, sigma 4.42%/day). Model predicts
P(touch level k*sigma*sqrt(T)) = 2*(1-Phi(k)), independent of T - the scale-invariance signature
of a true random walk. Measured the real empirical touch frequency for the same (k, T) pairs:
| k | model pred | BHP empirical (T=7/14/30/60d) | AL empirical (T=7/14/30/60d) |
|-----|-----------:|----------------------------------|-----------------------------------|
| 0.5 | 61.7% | 16.6% / 18.0% / 17.5% / 20.1% | 32.6% / 33.9% / 32.4% / 40.1% |
| 1.0 | 31.7% | 8.8% / 8.0% / 8.2% / 7.9% | 9.2% / 10.2% / 10.8% / 16.7% |
| 1.5 | 13.4% | 4.1% / 3.8% / 0.1% / 0.0% | 3.9% / 3.9% / 4.4% / 2.7% |
| 2.0 | 4.6% | 2.0% / 1.2% / 0.0% / 0.0% | 1.9% / 1.5% / 2.0% / 0.0% |
| 3.0 | 0.3% | 0.5% / 0.0% / 0.0% / 0.0% | 0.7% / 0.5% / 0.0% / 0.0% |
Model overstates touch probability by roughly 2-4x at low-moderate k, consistently across both
tickers. Scale-invariance partially holds at low k (flat across T, locally random-walk-like
short-term) but breaks down at higher k/longer T (empirical rate COLLAPSES faster than the model
past k=1.5, instead of staying flat) - the signature of mean reversion: real prices are anchored
to production cost/supply-demand equilibrium, not free-floating, so they wander less far over
longer horizons than a pure random walk would.
- The 2-4x figure overstates how much of this is mean reversion specifically: touches were
measured on daily VWAPs, a discrete sample of a continuous path, and the model's prediction
assumes continuous monitoring. Discrete sampling can only miss touches (an intraday spike that
reverts before the next VWAP looks like "no touch"), biasing the measured gap upward especially
at short horizons (k=0.5, T=7 - standard discrete-barrier corrections put this at roughly
0.58*sigma*sqrt(step) per sampling interval, shrinking as the horizon grows). The k>=1.5/T>=30
COLLAPSE is not explained by discretization (a roughly constant/shrinking effect, not a
progressive one) - that part is real mean reversion.
- Consequence: the model's absolute numbers are upper bounds, not honest estimates - directionally
conservative (it already said "don't climb the book," and the true probability of ever
profitably climbing is even lower than it claimed) but not quantitatively trustworthy.
- Replaced with `puga.history.empirical_touch()`: a non-parametric estimator measuring first-
passage directly from the ticker's own full multi-year history (every historical start day,
first time the price actually moved by the target amount, right-censored samples contribute 0
rather than being excluded) - no assumed distribution, so whatever mean-reversion/fat-tails/etc.
the real data has shows up directly. Inherits the same daily-VWAP discretization conservatism as
the backtest above (a resting ask can fill on an intraday high the VWAP series never shows) - so
it also leans a little more toward "sell now" than fully warranted, not corrected for. Used by
`tools/sell.py --tiers` (combined with queue-depth as a deterministic shift of the hitting time,
not a max - ranking by the resulting hit_factor, not a point estimate of median days) and
`puga/parkcash.py` (two sequential legs: buy-dip,
sell-recovery).
- Refined further: the unconditional version above still averages over EVERY historical start day,
which answers "how often does this ticker move by y from an arbitrary day" - but mean reversion
specifically means the answer depends on where price is NOW relative to its own recent trend.
Conditioned on state instead (`empirical_touch(..., condition_on_state=True)`, the default):
only start days whose z-score against a trailing 60-day mean is within +/-0.5 of TODAY's are
used, falls back to every start day if there's not enough history to compute today's z-score or
the match band is empty. Stays non-parametric (no fitted mean-reversion speed/strength). Real
effect confirmed on DEC (`tools/parkcash.py`): round-trip wait dropped from 19 days
(unconditional) to 3 days once conditioned on DEC's actual current state - a big, real
difference, not a rounding adjustment.
## 2026-10-10 Price-impact cap: real book depth, not a flat 25%-of-flow guess
- `puga/saturation.py`'s `S_SHARE=0.25` capped planned output at a flat 25% of a ticker's traded
flow, regardless of how deep or thin that specific market's standing bid book actually is -
never estimated how much price actually falls as supply rises, flagged by review.
- Tried first: a day-over-day regression of d(log price) on d(log volume) per ticker, the
textbook first-order elasticity estimate. Real data killed it: BHP -0.009, AL +0.004,
DEC +0.038, KV +0.007 - near zero and inconsistently signed. Cause: daily volume spikes in this
market are mostly DEMAND-driven (more buyers -> more volume AND flat/higher price), which
swamps and often reverses the sign of the SUPPLY-side effect we actually want. Shipping this
would have been worse than the flat guess (could tell you adding supply raises price).
- Fix: `puga.saturation.max_units_at_impact()` walks the LIVE bid book (same mechanics as
`puga.market.walk()`) and returns how many units you could sell before the realized price
degrades more than `--max-impact`%% (default 10) below vwap7/bid - a real, ticker-specific
measure, no regression, no confounding.
- Caveat, caught immediately when first wired as a hard `min()` cap alongside the flow-based
check: a live order book is a SINGLE SNAPSHOT - standing bids get added/withdrawn constantly,
same class of problem already found and fixed twice this session (`sell.py`'s queue model,
`parkcash.py`'s driftless walk), and we have no historical book-snapshot log to tell a
structurally-thin book from an unlucky instant. Using it as a hard cap let one noisy snapshot
silently zero out `n_lim` and hide a recipe entirely (happened to KV: flow-based n_lim 10.6 ->
0.0 purely from the snapshot). The 30-day flow figure (`tref_precise`) is the more reliable
signal precisely because it aggregates many real days, not one instant.
- Final design: `n_lim`/filtering stays flow-only (unchanged, reliable). `max_units_at_impact()`
surfaces as an `[exit-thin]` WARNING flag instead (can the live book absorb an instant dump of 1
day of 1 building's output right now, if cash were needed immediately?) - a panic-exit liquidity
check, NOT a production price-impact measure (that's `market`/`lim`, which price via patient
asks, the real selling mechanism) - visible, not silently filtering. Caught real signal on
several recipes (KV, SC, HER, TRN, NL, RG all flagged) without hiding any of them.
## 2026-10-10 Demolition value: flat reclaim fraction, not a time-decay curve
- `tools/scan.py --deprec` assumed building value decays LINEARLY to 0 over ~60 days (a guess,
never checked against real data). Checked it: pulled this account's live FIO reclaimables
(`/rain/userplanetbuildingreclaimables/{user}`) for all 8 owned building types, normalized by
built-count, and compared to real construction cost (FIO `BuildingCosts` at live ask prices).
- Result: reclaim fraction is 50-86% (CHP 68.7, EXT 86.5, HB1 58.3, HB2 50.9, HWP 73.1, POL 78.0,
SME 74.6; CM a low outlier at 7.4%, likely a core-module-specific rule) - but EVERY one of these
buildings is at ~99.7-99.9% live condition (this account's base is young). If reclaim really
decayed from ~100% toward 0 over 60 days, a near-perfect-condition building should reclaim close
to 100%, not 50-86%. This contradicts the decay-curve premise and is more consistent with a
roughly FLAT, material-dependent reclaim fraction that doesn't move much with condition in the
normal operating range.
- Can't fully settle the shape - no aged/low-condition building exists in this account to check
the other end of the curve, so this is suggestive, not conclusive (single base, narrow condition
range, small sample).
- Fix: `scan.py --deprec` now amortizes `capex * (1 - --reclaim-frac)` (the UNRECLAIMED portion)
over the holding period, not full capex. `--reclaim-frac` defaults to 0.65 (the mean of the 7
non-CM observations) - still a rough placeholder, not a calibrated curve; re-check once an aged
building exists to compare against.
## 2026-10-09 Drop "ROI/day %" terminology
- `profit/day ÷ new_capex` (what tools were calling "ROI/day X%") is just 1/payback_days rewritten as a percentage. It is NOT a compoundable rate of return - there's no reinvestment mechanism, and every opportunity here is capacity/market-limited (saturation), so it can't be extrapolated or annualized like an interest rate.
- Report **payback in days** as the primary number everywhere. A "%/day" figure, if shown at all, must be labelled as simple/non-compounding and never compared to a real yield/interest rate.
docs/library.md
+28 -3
View File
@@ -39,9 +39,15 @@ EOF
per-tier need table), `HAB_CAP`/`HAB_AREA`, `TIERS`. This is what you use for "what if I
understaff this building" or "what's the real wage bill for N heads" — the CLI tools only
expose fixed slices of this.
- **`puga.saturation`** — `tref`, `n_out`, `p_patient`, `effective_supply`, `is_thin`: the
depth/persistence primitives behind `scan.py`. Reuse these instead of re-deriving a "how many
buildings can the market take" estimate by hand.
- **`puga.saturation`** — `tref` (cheap, Poisson-bounded, for stage-1 screening), `tref_precise`
(real daily-volume variance, for a bounded candidate set), `n_out` (flow x queue, NO demand
term - thin resting-bid books don't disqualify a market sold into via patient asks), `p_patient`,
`effective_supply`, `is_thin` (demand still checked here, independently), `max_units_at_impact`
(real live bid-book depth cap - how many units before realized price degrades past a threshold;
replaced a flat 25%-of-flow guess after a naive price-vs-volume elasticity regression on real
data came back near-zero/wrong-signed, see `docs/decisions.md`): the depth/persistence
primitives behind `scan.py`. Reuse these instead of re-deriving a "how many buildings can the
market take" estimate by hand.
- **`puga.simulate`** — `simulate(plan, recipes, buildings, resources, fertility, price, ...)`:
the PRUNplanner-simulator replica. Reuse this for anything plan-shaped rather than reimplementing
efficiency/workforce/flows again.
@@ -63,6 +69,25 @@ EOF
(ship+from+to, auto-computes everything via `puga.route`), `tools/shipcost.py` (explicit
jumps/sliders/gateway-fees, for when you already know the breakdown). Raw calibration data:
`empire/data/shipping_calibration/`.
- **`puga.parkcash`** — `scan(cx="AI1", min_spread_pct=20.0, days=30, rate=0.05)`: every ticker
with a live quote and a big bid/ask spread, scored on real trading liquidity
(`puga.history.trading_days()`/`trade_gaps_days()`) and an EMPIRICAL round-trip estimate
(`puga.history.empirical_touch()`, two sequential legs: dip to bid, recover to ask) - no
random-walk assumption. CLI: `tools/parkcash.py`. See the module docstring for open gaps (no fee
data; the two legs are treated as independent, an approximation).
- **`puga.history`** — `daily_prices(tk, cx)`, `trading_days(tk, cx, days)` (real calendar-window
trading days, not "last N rows" - a sparse ticker needs this distinction), `trade_gaps_days(dates)`,
`full_series(tk, cx)` (all available history, ascending), `empirical_touch(series, y, rate)`
(non-parametric first-passage stats for a target log-move `y` - hit_factor/touched_frac/
median_days, estimated directly from how often/fast the ticker's own real history has made
moves of that size, by default only drawing on historical days in a similar state to today
(trailing-60d z-score within +/-0.5, `condition_on_state=True`) since mean reversion's effect
depends on where price is now, not just the move size. Optional `horizon` (days, off by
default): bounds both eligibility and the search window to a fixed number of days so every
start-day sample gets the same fair window, instead of recent start days being biased by
running out of data to search. Replaced an earlier driftless-random-walk formula that
backtested wrong, see `docs/decisions.md`). Shared by `tools/history.py`, `tools/sell.py`, `tools/undervalued.py`,
`puga.parkcash`.
- **`puga.contracts`** — `scan(planet, direction, sf_price, ff_price, v0=1.0, v1=1.0)`: live
Local Market hauling ads (FIO `/localmarket/shipping/{source,destination}`), each enriched with
real one-way fuel cost (`puga.shipping.leg_cost`) and margin. CLI: `tools/lmscan.py`.
docs/roadmap.md
+40 -28
View File
@@ -1,34 +1,46 @@
# Roadmap
Legend: [ ] todo, [x] done. Build order matters; each step is usable on its own.
Open work only - what's already built/fixed is discoverable from `CLAUDE.md`'s Tools list,
`docs/library.md`, `docs/decisions.md`, and module docstrings, not repeated here.
1. [x] Repo skeleton, git (master), venv, CLAUDE.md, mechanics from PRUNplanner.
2. [x] `puga/fio.py`: cached FIO client (TTL per endpoint, keys from `.env`), plus `puga/prunplanner.py` for `api.prunplanner.org/data/*`. Inspect what `exchanges/` adds (VWAP?) vs FIO `/exchange/all`.
3. [x] (price + book tools done; cxpc trend history not needed, PRUNplanner has vwap7/30) `puga/market.py` + `tools/price.py`, `tools/book.py`: prices across all CX, trends from cxpc, order-book walk for real fill price at quantity N.
4. [x] `puga/econ.py` (22 tests pass, expected values from PRUNplanner's suite): efficiency stack, workforce consumption, hab LP, extraction (port the formulas from `docs/mechanics.md`, with tests against PRUNplanner values).
5. [x] (v1 done; see saturation-design.md for refinements) `tools/scan.py` (replaces prun_scan.py): depth-aware ROI. Per recipe report: buildings the market can absorb (output demand + traded volume, input supply), ROI at fill price for N buildings, absorbable profit/day. Filter `--min-buildings`. Both CX-local and universe.
5b. [ ] `tools/arb.py` (replaces prun_cxarb.py): inter-CX arbitrage on `market.walk` at real fill size; capital required, fuel/shipping cost from state, profit per trip within cash. Reason: old script uses top-of-book spread x min(supply,demand); on 2 of 4 top hits it overstated profit 7 to 11x (WAI, SCN).
6. [x] `tools/chain.py`: full bill of materials, make-vs-buy per node.
7. [x] `tools/whatif.py`: marginal ROI of adding a building to a base you already run (whole-plan diff via `simulate()`, built dict from `state/company.yaml`; flags newly-caused workforce overloads).
8. [x] `tools/planet.py`: rank planets for a resource (FIO bulk `Resources`/`BuildRequirements` data), founding cost from real environment-extras materials, jump distance via `tools/route.py`. `tools/found.py` (full founding workflow/writeback) still todo.
13b. [x] `puga/network.py` + `tools/network.py`: multi-base chain modelling (per-base simulate, net transfers against real freight, `cm_free` for kits). Worked example: `plans/chains/nike_deimos_lst.yaml` (Nike LST -> Deimos PP2/BSE). Known gap: reports whole-plan profit per base, not the marginal addition; caller must diff manually.
9. [x] `tools/route.py`: jump graph (FIO `/systemstars`, Dijkstra by straight-line ly, jump count alongside). `tools/haul.py` (fuel cost, profit per trip) still todo.
9c. [x] `puga/shipping.py` + `tools/shipcost.py` (explicit params) + `tools/shiptrip.py` (just ship/from/to) + `puga.shipping.round_trip_cost()` wired into `tools/network.py`'s freight cost: real ship flight fuel/cost model (STL cruise + FTL jump/charge, linear in the fuel-usage/reactor-usage sliders, additive/no cross-term; gateway hops free-fuel + manual LOCK fee read off the live game, not lookupable via any FIO/PRUNplanner endpoint - checked; landing fuel a measured per-planet lookup). Calibrated 2026-10-09 on this account's one hull/reactor (blueprint BP-STRT-0000, shared by both ships AVI-07JFB/AVI-07JFC, confirmed identical specs via FIO) via a controlled 4x4-slider-grid sweep across 5 real destinations, read from the live APEX Ship Flight Control DOM. Raw data: `empire/data/shipping_calibration/`. Open gaps (see `puga/shipping.py` docstring): departure-side gravity dependence untested (every calibration DEP left the same origin), landing fuel vs gravity/distance confounded (n=3), no second hull/reactor to cross-check generalization. `tools/scan.py --trip-cost` still uses the flat 9250 AIC placeholder (not yet wired to the real model - needs a per-recipe destination planet, not just a CX code).
9d. [x] `puga/contracts.py` + `tools/lmscan.py`: live Local Market hauling-ad scan (FIO `/localmarket/shipping/{source,destination}`, public/no-auth - real player-posted freight jobs, not the CONTD player-contract system) ranked by margin after real fuel cost (`puga.shipping.leg_cost`, one-way, with a small measured gateway-fee table `KNOWN_GATEWAY_ROUTES` for ANT<->HRT/BEN). Ship cargo hold confirmed 500t/500m3 via FIO for both ships.
9b. [x] (dry-run default, --apply after user yes) `tools/plan_push.py`: create/update `[PuGa]` plans in PRUNplanner from YAML (Api-Key auth; see decisions.md for guardrails).
10. [x] `tools/state.py sync` (sites, production efficiency, storage, ships, cash, permits): inventory, production, ships into `state/`.
11. [ ] Docs to fill: `docs/apex.md`, `docs/fio.md`, `docs/playbook.md` (question type -> tool), `docs/decisions.md`.
## Todo
- [ ] `tools/arb.py`: inter-CX arbitrage at real fill size (replaces `prun_cxarb.py`, which
overstated profit 7-11x on 2/4 top hits).
- [ ] `tools/haul.py`: fuel cost / profit per trip (route.py has the jump graph).
- [ ] `tools/portfolio.py`: profit-maximizing set of 1-building opportunities under shared
housing/capex/permits. Ties into backlog #2 (profit/permit ranking, needs this tool to mean
anything - permits bind at the whole-base level, not per-recipe).
- [ ] Docs to fill: `docs/apex.md`, `docs/fio.md`, `docs/playbook.md`.
13. [x] `tools/buy.py`: cash-aware shopping list (construction gap vs empire state's built count, plus N days of net operating stock via simulate(), priced at real fill cost, checked against cash/reserve).
14. [x] `tools/sell.py`: ask-post-price + expected-clear-time advisor from 30d traded-volume percentiles, vs hitting the bids now.
15. [x] `tools/runway.py`: days-of-runway per material from simulate()'s net consumption, base storage plus auto-detected inbound ship cargo (tools/state.py now tags each ship with its live flight origin/destination, and each STORE with its planet, needed once a 2nd base existed).
12. [ ] `tools/portfolio.py`: choose the set of 1-building opportunities for one base that maximises total profit given shared whole-building housing (HB1/HB2/...), capex, permits and (for demolish-later) 60-day value decay. Persistence check exists (`tools/persistence.py`); wire it into the ranking.
14. [x] `tools/undervalued.py`: trading (not building) strategy - manufactured goods priced below implied production cost, candidates for a buy-and-hold-for-reversion bet. Backtested 2026-10-09 (326 output tickers, 287 deduplicated dip-onset episodes, 112 distinct tickers): +3mo forward return mean +21.3%/median +8.9%/73.2% hit rate at >=10 units/day volume - roughly 0.10-0.24%/day, far below the 14-29%/day marginal ROI on production builds found this session. Only worth excess idle capital, not a primary strategy.
## Backlog: review findings, still open (spot-checked against code)
1. [ ] `scan.py` payback ignores working capital (input stock/output inventory/consumables) - can
rival building capex for smelting-type recipes. `buy.py` already computes this number.
2. [ ] No profit/permit ranking (permits bind at the base level, not per-recipe) - needs
`tools/portfolio.py`'s cross-recipe/whole-base view.
3. [ ] `scan.py` doesn't charge degradation (capex/180/day) like `simulate.py` does - small but real
inconsistency between tools ranking the same thing.
4. [ ] No input-side flow cap (buying a big share of an input's flow should price-impact like
selling output does) - documented as absent, not modelled.
5. [ ] `persistence.py --lag 3` default is too low given `docs/mechanics.md`'s weekly workforce
distribution - real lag is 3-10 days.
6. [ ] Possible forward-fill bias if any tool reindexes `daily_prices()` onto a full calendar grid
(it only emits trade days) - would understate volatility for sparse tickers. Not yet confirmed
whether `persistence.py`/`undervalued.py` actually do this.
7. [ ] `undervalued.py`'s backtest (+21.3%/3mo) has no control group - meaningless without the
universe's unconditional return over the same window.
8. [ ] `persistence.py` measures past margin variance of a market we were NOT in - can't see the
dominant real risk (our own entry + the next player's compressing margin).
9. [ ] `undervalued.py::manufacturing_candidates()` keeps the first recipe seen per output, not
the cheapest (`seen.setdefault`).
10. [ ] `scan.py` hab cost ignores combined habs (HBB/HBC/HBM/HBL), which can be cheaper per head.
11. [ ] `scan.py --trip-cost` still defaults to the flat 9250 placeholder despite the real
`puga/shipping.py` model existing and already used by `whatif.py`.
12. [ ] `sell.py`'s `s0` is the day's VWAP, not its high - `--tiers` touch probability is slightly
pessimistic (same discretization conservatism as `empirical_touch()`, see `docs/decisions.md`).
13. [ ] `empirical_touch()`'s optional `horizon` (fixed-window right-censoring) exists but isn't
wired into any tool's default call yet.
## Open questions
- PRUNplanner auth is known (`Authorization: Api-Key <key>`). FIO REST and FIO API are separate services with separate keys (separate services with separate keys; an earlier "one API" reading was wrong). Keys are in `.env`. Verified 2026-09-18: FIO_REST_KEY works on rest.fnar.net as `Authorization: <key>` (own /sites, /storage etc. readable); FIO_API_KEY gives 401 there, host unknown. PRUNplanner key works (plans 0, empires 1, cx prefs 1 on his account). Still unknown: the OpenAPI spec at https://doc.fnar.net/api.json has no securitySchemes. Probe with read-only calls.
- (resolved) PRUNplanner `exchanges/` has vwap_daily/7d/30d and traded volume; merged into `puga/market.py`. Trend history via cxpc still todo.
- Extraction: concentration to daily rate formula.
- (resolved 2026-09-25) Real price-band filtering (`NarrowPriceBandLow/High`, `WidePriceBandLow/High`) implemented in `puga/saturation.py` (`in_band`, `effective_demand`, `effective_supply` band param) and wired into `tools/scan.py` stage 2. Was flagged as an unimplemented refinement in saturation-design.md since day one; the player noticed a stale out-of-band bid in a live screenshot and asked the right question.
- FIO REST and FIO API are separate services/keys (`.env`). FIO_REST_KEY works on rest.fnar.net;
FIO_API_KEY's host is unknown; PRUNplanner's Api-Key works. `doc.fnar.net/api.json` has no
securitySchemes.