Add the map-table vision pipeline
Reads a screenshot of the in-game map table and recovers where on the board it is looking. The scene is a flat table under a perspective camera, so a single homography describes grid<->screen exactly; that is fitted from line evidence (LSD segments, vanishing-point RANSAC, 1-D lattice fits). Line evidence alone cannot finish the job: the 1km cells and the 100m subgrid are locally identical, so it fixes neither scale, axis assignment, axis direction nor phase. Those come from the cell labels -- and the labels are never *detected*, they are correlated where the grid says they must be (9% in from a cell's left edge, 6% down from its top). Blob detection on aerial-photo terrain finds texture, not glyphs; correlating a known template at a known place has no such failure mode. The same score then ranks the geometry hypotheses, since a wrong lattice puts the crop where no label is, so one number resolves scale, axis assignment, direction, phase and anchor together. Markers are found by hostile/friendly colour plus an IoU test against an ideal inscribed diamond, which cut a red-lit shot from 43 false positives to 2 while preserving every hand-verified count. Unit-type classification is present but not yet reliable, and returns unknown rather than guessing. Measured over the fixture set: 7 of 10 solve, each with 100% of its ground-truth points in the correct cell (85/112 overall), residual spread 0.005-0.033 cells. The other three reject cleanly; none has ever produced a plausible-but-wrong grid. Across 122 typewriter screenshots solve() accepted none, which is what makes clipboard routing safe. Fixtures: 10 map shots with hand-transcribed ground truth, plus 10 typewriter shots spanning 262-5366px for routing and future OCR tests. Map shots wider than 2400px (the pipeline's own maximum working width) were downscaled with their coordinates rescaled to match; re-measured afterwards, the results are identical. Typewriter shots stay at native resolution because OCR needs the text legible. Drops docs/map_vision_plan.md and its WIP prototype: the design now lives in the module docstring, next to the code it describes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@@ -0,0 +1,36 @@
|
||||
# Test fixtures
|
||||
|
||||
Real screenshots of the game, used as regression data. They are the game
|
||||
author's work, not covered by this repo's MIT license (see `/LICENSE`).
|
||||
|
||||
## `map_shots/` + `map_shots_gt.json`
|
||||
|
||||
The evaluation set for the map-grid solver (`src/fenigma/map_vision.py`), scored
|
||||
by `tools/eval_map_vision.py`. The JSON holds hand-transcribed cell labels at
|
||||
native pixel positions; its own header comment explains the format and the
|
||||
9%/6% label-padding constant.
|
||||
|
||||
Shots wider than 2400px were downscaled to 2400px, and their ground-truth
|
||||
coordinates rescaled with them. 2400 is `map_vision.RETRY_WORK_W`, the widest
|
||||
the pipeline ever works at, so nothing the code can actually read was lost.
|
||||
Measured after the downscale: the same 7 of 10 solve, 100% of their points land
|
||||
in the correct cell, residual spread unchanged.
|
||||
|
||||
`too_hard/` holds shots that are permanent rejections; see its own README.
|
||||
|
||||
## `writer_shots/`
|
||||
|
||||
Typewriter/field-log screenshots. Two uses:
|
||||
|
||||
- Measuring false positives in the map-vs-text routing gate
|
||||
(`map_vision.looks_like_map` / `solve`). Over the full 122-shot set the cheap
|
||||
gate false-positived on 6% and `solve()` accepted **none**.
|
||||
- OCR regression material for `ocr.py`.
|
||||
|
||||
Kept at NATIVE resolution deliberately: the routing gate only ever sees 1500px,
|
||||
but OCR needs the text legible, so these must not be downscaled.
|
||||
|
||||
Ten shots are committed, chosen to span the capture-scale range (262px to
|
||||
5366px wide) since scale is what both the gate and OCR are sensitive to. The
|
||||
false-positive numbers above were measured on all 122; this subset is a
|
||||
regression guard, not the measurement.
|
||||
|
After Width: | Height: | Size: 3.1 MiB |
|
After Width: | Height: | Size: 1.5 MiB |
|
After Width: | Height: | Size: 4.8 MiB |
|
After Width: | Height: | Size: 3.5 MiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 4.5 MiB |
|
After Width: | Height: | Size: 4.3 MiB |
|
After Width: | Height: | Size: 3.2 MiB |
|
After Width: | Height: | Size: 3.4 MiB |
|
After Width: | Height: | Size: 2.8 MiB |
|
After Width: | Height: | Size: 1.1 MiB |
|
After Width: | Height: | Size: 501 KiB |
|
After Width: | Height: | Size: 4.5 MiB |
@@ -0,0 +1,14 @@
|
||||
# Screenshots we deliberately do not try to solve
|
||||
|
||||
Kept for the record, excluded from the evaluation set. These fail for reasons
|
||||
that are properties of the screenshot, not of the algorithm, so working around
|
||||
them would mean guessing:
|
||||
|
||||
- `06.png`, `14.png` — exactly ONE grid label visible. A single label cannot be
|
||||
cross-checked, so a misread would silently shift the whole board with nothing
|
||||
to contradict it. Two mutually consistent labels is the minimum safe anchor.
|
||||
- `12.png` — 710x594 native. Too few pixels per cell for the label glyphs to
|
||||
correlate; measured, the best label score stays ~0.44 at every working
|
||||
resolution, so it is not a tuning problem.
|
||||
|
||||
"Too zoomed in" and "too low resolution" are legitimate hard rejections.
|
||||
@@ -0,0 +1,640 @@
|
||||
{
|
||||
"_comment": [
|
||||
"Ground truth for the map-vision fixtures, transcribed by hand from the",
|
||||
"screenshots. Each entry is [cell_label, x, y] in NATIVE pixel coordinates",
|
||||
"of the corresponding file in map_shots/, where (x, y) is roughly the",
|
||||
"centre of the drawn cell label glyphs.",
|
||||
"",
|
||||
"The invariant being asserted is simply: the pixel (x, y) lies inside the",
|
||||
"map cell named by cell_label. That is enough to catch every failure mode",
|
||||
"seen so far (wrong lattice scale, wrong integer offset, badly misfitted",
|
||||
"homography) without needing sub-pixel corner annotation.",
|
||||
"",
|
||||
"Positions were read off a ruler overlay by eye, so treat them as accurate",
|
||||
"to roughly +/-15 native px. The cell IDENTITIES are exact.",
|
||||
"",
|
||||
"Game UI constant, useful for the estimator: a cell's label is drawn with",
|
||||
"about 9% of the cell size as padding from the cell's left edge and 6% from",
|
||||
"its top edge, so label_top_left - (0.09, 0.06) * cell_size lands on the",
|
||||
"cell's top-left corner."
|
||||
],
|
||||
"01.png": [
|
||||
[
|
||||
"G10",
|
||||
899,
|
||||
294
|
||||
],
|
||||
[
|
||||
"H10",
|
||||
996,
|
||||
294
|
||||
],
|
||||
[
|
||||
"I10",
|
||||
1083,
|
||||
294
|
||||
],
|
||||
[
|
||||
"J10",
|
||||
1168,
|
||||
294
|
||||
],
|
||||
[
|
||||
"K10",
|
||||
1270,
|
||||
294
|
||||
],
|
||||
[
|
||||
"L10",
|
||||
1355,
|
||||
294
|
||||
],
|
||||
[
|
||||
"M10",
|
||||
1446,
|
||||
294
|
||||
],
|
||||
[
|
||||
"N10",
|
||||
1537,
|
||||
294
|
||||
],
|
||||
[
|
||||
"O10",
|
||||
1628,
|
||||
294
|
||||
],
|
||||
[
|
||||
"P10",
|
||||
1716,
|
||||
294
|
||||
],
|
||||
[
|
||||
"G9",
|
||||
882,
|
||||
366
|
||||
],
|
||||
[
|
||||
"I9",
|
||||
1072,
|
||||
366
|
||||
],
|
||||
[
|
||||
"K9",
|
||||
1266,
|
||||
366
|
||||
],
|
||||
[
|
||||
"M9",
|
||||
1456,
|
||||
366
|
||||
],
|
||||
[
|
||||
"O9",
|
||||
1650,
|
||||
366
|
||||
],
|
||||
[
|
||||
"H8",
|
||||
965,
|
||||
445
|
||||
],
|
||||
[
|
||||
"J8",
|
||||
1168,
|
||||
445
|
||||
],
|
||||
[
|
||||
"L8",
|
||||
1370,
|
||||
445
|
||||
],
|
||||
[
|
||||
"N8",
|
||||
1572,
|
||||
445
|
||||
],
|
||||
[
|
||||
"H7",
|
||||
953,
|
||||
536
|
||||
],
|
||||
[
|
||||
"J7",
|
||||
1166,
|
||||
536
|
||||
],
|
||||
[
|
||||
"L7",
|
||||
1379,
|
||||
536
|
||||
],
|
||||
[
|
||||
"N7",
|
||||
1592,
|
||||
536
|
||||
],
|
||||
[
|
||||
"H6",
|
||||
935,
|
||||
637
|
||||
],
|
||||
[
|
||||
"J6",
|
||||
1166,
|
||||
637
|
||||
],
|
||||
[
|
||||
"L6",
|
||||
1391,
|
||||
637
|
||||
],
|
||||
[
|
||||
"N6",
|
||||
1619,
|
||||
637
|
||||
]
|
||||
],
|
||||
"02.png": [
|
||||
[
|
||||
"I9",
|
||||
54,
|
||||
28
|
||||
],
|
||||
[
|
||||
"J9",
|
||||
329,
|
||||
28
|
||||
],
|
||||
[
|
||||
"K9",
|
||||
608,
|
||||
30
|
||||
],
|
||||
[
|
||||
"L9",
|
||||
880,
|
||||
28
|
||||
],
|
||||
[
|
||||
"M9",
|
||||
1155,
|
||||
28
|
||||
],
|
||||
[
|
||||
"I8",
|
||||
37,
|
||||
257
|
||||
],
|
||||
[
|
||||
"J8",
|
||||
320,
|
||||
257
|
||||
],
|
||||
[
|
||||
"K8",
|
||||
620,
|
||||
257
|
||||
],
|
||||
[
|
||||
"L8",
|
||||
902,
|
||||
257
|
||||
],
|
||||
[
|
||||
"M8",
|
||||
1197,
|
||||
257
|
||||
],
|
||||
[
|
||||
"I7",
|
||||
11,
|
||||
517
|
||||
],
|
||||
[
|
||||
"J7",
|
||||
316,
|
||||
517
|
||||
],
|
||||
[
|
||||
"K7",
|
||||
626,
|
||||
517
|
||||
],
|
||||
[
|
||||
"L7",
|
||||
936,
|
||||
517
|
||||
],
|
||||
[
|
||||
"M7",
|
||||
1248,
|
||||
517
|
||||
],
|
||||
[
|
||||
"J6",
|
||||
313,
|
||||
805
|
||||
],
|
||||
[
|
||||
"K6",
|
||||
638,
|
||||
805
|
||||
],
|
||||
[
|
||||
"L6",
|
||||
968,
|
||||
805
|
||||
],
|
||||
[
|
||||
"M6",
|
||||
1295,
|
||||
805
|
||||
]
|
||||
],
|
||||
"03.png": [
|
||||
[
|
||||
"H8",
|
||||
224,
|
||||
229
|
||||
],
|
||||
[
|
||||
"I8",
|
||||
856,
|
||||
229
|
||||
],
|
||||
[
|
||||
"J8",
|
||||
1496,
|
||||
229
|
||||
],
|
||||
[
|
||||
"K8",
|
||||
2136,
|
||||
229
|
||||
],
|
||||
[
|
||||
"H7",
|
||||
216,
|
||||
853
|
||||
],
|
||||
[
|
||||
"I7",
|
||||
896,
|
||||
853
|
||||
],
|
||||
[
|
||||
"J7",
|
||||
1555,
|
||||
853
|
||||
],
|
||||
[
|
||||
"K7",
|
||||
2224,
|
||||
853
|
||||
]
|
||||
],
|
||||
"04.png": [
|
||||
[
|
||||
"N8",
|
||||
243,
|
||||
180
|
||||
],
|
||||
[
|
||||
"O8",
|
||||
1119,
|
||||
180
|
||||
],
|
||||
[
|
||||
"P8",
|
||||
1996,
|
||||
180
|
||||
],
|
||||
[
|
||||
"N7",
|
||||
232,
|
||||
1038
|
||||
],
|
||||
[
|
||||
"O7",
|
||||
1135,
|
||||
1038
|
||||
],
|
||||
[
|
||||
"P7",
|
||||
2051,
|
||||
1038
|
||||
]
|
||||
],
|
||||
"05.png": [
|
||||
[
|
||||
"M3",
|
||||
261,
|
||||
213
|
||||
],
|
||||
[
|
||||
"N3",
|
||||
1160,
|
||||
213
|
||||
]
|
||||
],
|
||||
"07.png": [
|
||||
[
|
||||
"J8",
|
||||
460,
|
||||
285
|
||||
],
|
||||
[
|
||||
"K8",
|
||||
1221,
|
||||
285
|
||||
],
|
||||
[
|
||||
"L8",
|
||||
1973,
|
||||
285
|
||||
],
|
||||
[
|
||||
"J7",
|
||||
456,
|
||||
1036
|
||||
],
|
||||
[
|
||||
"K7",
|
||||
1256,
|
||||
1036
|
||||
],
|
||||
[
|
||||
"L7",
|
||||
2053,
|
||||
1036
|
||||
]
|
||||
],
|
||||
"08.png": [
|
||||
[
|
||||
"L8",
|
||||
184,
|
||||
417
|
||||
],
|
||||
[
|
||||
"M8",
|
||||
1085,
|
||||
417
|
||||
],
|
||||
[
|
||||
"N8",
|
||||
1983,
|
||||
417
|
||||
],
|
||||
[
|
||||
"L7",
|
||||
176,
|
||||
1341
|
||||
],
|
||||
[
|
||||
"M7",
|
||||
1121,
|
||||
1341
|
||||
]
|
||||
],
|
||||
"09.png": [
|
||||
[
|
||||
"O9",
|
||||
1348,
|
||||
180
|
||||
],
|
||||
[
|
||||
"P9",
|
||||
1520,
|
||||
180
|
||||
],
|
||||
[
|
||||
"Q9",
|
||||
1692,
|
||||
180
|
||||
],
|
||||
[
|
||||
"M8",
|
||||
993,
|
||||
336
|
||||
],
|
||||
[
|
||||
"N8",
|
||||
1172,
|
||||
336
|
||||
],
|
||||
[
|
||||
"O8",
|
||||
1352,
|
||||
336
|
||||
],
|
||||
[
|
||||
"P8",
|
||||
1536,
|
||||
336
|
||||
],
|
||||
[
|
||||
"Q8",
|
||||
1718,
|
||||
336
|
||||
],
|
||||
[
|
||||
"M7",
|
||||
982,
|
||||
513
|
||||
],
|
||||
[
|
||||
"N7",
|
||||
1175,
|
||||
513
|
||||
],
|
||||
[
|
||||
"O7",
|
||||
1366,
|
||||
513
|
||||
],
|
||||
[
|
||||
"P7",
|
||||
1557,
|
||||
513
|
||||
],
|
||||
[
|
||||
"Q7",
|
||||
1748,
|
||||
513
|
||||
]
|
||||
],
|
||||
"11.png": [
|
||||
[
|
||||
"J9",
|
||||
174,
|
||||
133
|
||||
],
|
||||
[
|
||||
"K9",
|
||||
996,
|
||||
133
|
||||
],
|
||||
[
|
||||
"L9",
|
||||
1803,
|
||||
133
|
||||
],
|
||||
[
|
||||
"J8",
|
||||
159,
|
||||
956
|
||||
],
|
||||
[
|
||||
"K8",
|
||||
999,
|
||||
956
|
||||
],
|
||||
[
|
||||
"L8",
|
||||
1832,
|
||||
956
|
||||
]
|
||||
],
|
||||
"13.png": [
|
||||
[
|
||||
"L4",
|
||||
182,
|
||||
213
|
||||
],
|
||||
[
|
||||
"M4",
|
||||
509,
|
||||
197
|
||||
],
|
||||
[
|
||||
"O4",
|
||||
1129,
|
||||
157
|
||||
],
|
||||
[
|
||||
"P4",
|
||||
1425,
|
||||
133
|
||||
],
|
||||
[
|
||||
"Q4",
|
||||
1711,
|
||||
117
|
||||
],
|
||||
[
|
||||
"L3",
|
||||
146,
|
||||
441
|
||||
],
|
||||
[
|
||||
"M3",
|
||||
511,
|
||||
412
|
||||
],
|
||||
[
|
||||
"N3",
|
||||
863,
|
||||
388
|
||||
],
|
||||
[
|
||||
"O3",
|
||||
1209,
|
||||
359
|
||||
],
|
||||
[
|
||||
"P3",
|
||||
1538,
|
||||
329
|
||||
],
|
||||
[
|
||||
"L2",
|
||||
106,
|
||||
724
|
||||
],
|
||||
[
|
||||
"M2",
|
||||
518,
|
||||
687
|
||||
],
|
||||
[
|
||||
"N2",
|
||||
914,
|
||||
651
|
||||
],
|
||||
[
|
||||
"O2",
|
||||
1304,
|
||||
615
|
||||
],
|
||||
[
|
||||
"P2",
|
||||
1664,
|
||||
580
|
||||
],
|
||||
[
|
||||
"L1",
|
||||
37,
|
||||
1100
|
||||
],
|
||||
[
|
||||
"M1",
|
||||
521,
|
||||
1053
|
||||
],
|
||||
[
|
||||
"N1",
|
||||
985,
|
||||
1005
|
||||
],
|
||||
[
|
||||
"O1",
|
||||
1428,
|
||||
953
|
||||
],
|
||||
[
|
||||
"P1",
|
||||
1852,
|
||||
910
|
||||
]
|
||||
],
|
||||
"_excluded": {
|
||||
"note": "moved to map_shots/too_hard/, see its README",
|
||||
"06.png": [
|
||||
[
|
||||
"L5",
|
||||
542,
|
||||
616
|
||||
]
|
||||
],
|
||||
"12.png": [
|
||||
[
|
||||
"M3",
|
||||
151,
|
||||
160
|
||||
],
|
||||
[
|
||||
"N3",
|
||||
535,
|
||||
136
|
||||
],
|
||||
[
|
||||
"M2",
|
||||
163,
|
||||
469
|
||||
],
|
||||
[
|
||||
"N2",
|
||||
580,
|
||||
440
|
||||
]
|
||||
],
|
||||
"14.png": [
|
||||
[
|
||||
"J7",
|
||||
1557,
|
||||
1659
|
||||
]
|
||||
]
|
||||
}
|
||||
}
|
||||
|
After Width: | Height: | Size: 386 KiB |
|
After Width: | Height: | Size: 1008 KiB |
|
After Width: | Height: | Size: 1.6 MiB |
|
After Width: | Height: | Size: 2.1 MiB |
|
After Width: | Height: | Size: 7.3 MiB |
|
After Width: | Height: | Size: 637 KiB |
|
After Width: | Height: | Size: 3.7 MiB |
|
After Width: | Height: | Size: 37 KiB |
|
After Width: | Height: | Size: 858 KiB |
|
After Width: | Height: | Size: 120 KiB |