Drop PLAN.md (its content is now stale/superseded by the actual implementation)
This commit is contained in:
@@ -1,117 +0,0 @@
|
|||||||
# wgBill — plan
|
|
||||||
|
|
||||||
Webapp to replace the manual "upload to NC → share link → paste into Cospend
|
|
||||||
bill title" flow with: photo → extracted items → pick which ones (per
|
|
||||||
splitting group) → highlighted photo → uploaded + shared → Cospend bill(s)
|
|
||||||
created automatically.
|
|
||||||
|
|
||||||
## Confirmed via API research
|
|
||||||
|
|
||||||
- **WebDAV** (`PUT /remote.php/dav/files/<user>/<path>`) — uploads the
|
|
||||||
receipt/highlighted image. Auth via NC app password (Basic auth).
|
|
||||||
- **OCS Share API** (`POST /ocs/v2.php/apps/files_sharing/api/v1/shares`,
|
|
||||||
`shareType=3`) — creates the public link, response includes ready `url`.
|
|
||||||
- **Cospend API** — verified against the actual source (`appinfo/routes.php`
|
|
||||||
+ `lib/Controller/ApiController.php` + `lib/Db/{Project,Member}.php` in
|
|
||||||
`julien-nc/cospend-nc`), not just doc summaries:
|
|
||||||
- It's an OCS API, same family as the Share API — full paths are under
|
|
||||||
`/ocs/v2.php/apps/cospend/api/v1/...`, need `OCS-APIRequest: true`.
|
|
||||||
- `GET /projects` → `getLocalProjects` — lists the authenticated user's
|
|
||||||
projects (`{id, name, ...}`). No hardcoded project id needed; the app
|
|
||||||
lets you pick.
|
|
||||||
- `GET /projects/{id}/members` → `{id, name, ...}` (member `id` is
|
|
||||||
numeric).
|
|
||||||
- `POST /projects/{id}/bills` → `createBill(projectId, date, what, payer,
|
|
||||||
payedFor, amount, ..., comment, ...)` — **camelCase** params (not the
|
|
||||||
old IHateMoney-style snake_case an earlier doc summary suggested);
|
|
||||||
`payer`/`payedFor` are numeric member ids, `payedFor` is a
|
|
||||||
comma-separated string of them. Returns just the new bill's **id**
|
|
||||||
(int), not a bill object.
|
|
||||||
**No native attachment/link field on a bill.** Confirms the current manual
|
|
||||||
trick (put the link in the bill title) is the only integration point — the
|
|
||||||
app does the same. Cospend's UI only recognizes a bill as "has an
|
|
||||||
attachment" when the link is in `what` (the title), not `comment` — a link
|
|
||||||
in `comment` is just plain text. So: link goes in `what`, itemized
|
|
||||||
breakdown goes in `comment`.
|
|
||||||
- CORS ruled out doing this as a pure static frontend talking to NC directly
|
|
||||||
from a different origin (WebDAV PUT and Login Flow v2 both fight the
|
|
||||||
browser here) — decided to just build a small backend instead.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
[PWA frontend] --(HTTPS, JSON)--> [backend] --(WebDAV / OCS / Cospend API)--> [Nextcloud]
|
|
||||||
|
|
|
||||||
+--(OpenAI-compatible chat completions)--> [vision LLM]
|
|
||||||
```
|
|
||||||
|
|
||||||
- **Backend**: Python + Flask + SQLite. Holds NC app password and the
|
|
||||||
vision-provider API key as server config (env vars), never exposed to the
|
|
||||||
client. SQLite holds the per-receipt session (parsed items, image path,
|
|
||||||
group selections) — durable across a backend restart, unlike an in-memory
|
|
||||||
store, and trivial to inspect/debug directly.
|
|
||||||
- **Frontend**: Vite + React + TypeScript, PWA (installable, camera access
|
|
||||||
via `<input capture>` / `getUserMedia`).
|
|
||||||
- **Vision extraction**: any OpenAI-compatible endpoint. Config = `base_url`
|
|
||||||
+ `api_key` + `model`. Works unmodified for OpenAI; for Gemini point
|
|
||||||
`base_url` at `https://generativelanguage.googleapis.com/v1beta/openai/`.
|
|
||||||
Switch/compare providers via env var, no code change.
|
|
||||||
|
|
||||||
## Data flow
|
|
||||||
|
|
||||||
1. **Capture** — frontend takes/picks a photo, uploads to backend.
|
|
||||||
2. **Extract** — backend sends image to the vision LLM with a prompt asking
|
|
||||||
for strict JSON: `{ items: [{ id, label, price, bbox: [x,y,w,h] }] }`
|
|
||||||
(bbox in normalized 0–1 coords, for the highlight step later). Backend
|
|
||||||
stores the original image on disk and the parsed items in a SQLite
|
|
||||||
`receipts` table, keyed by a random session id; a periodic cleanup (or
|
|
||||||
just TTL-on-read) drops old sessions/images.
|
|
||||||
3. **Review** — frontend shows the parsed list, editable (OCR/LLM extraction
|
|
||||||
won't be perfect — fix a mis-read price, merge a split line, add a missed
|
|
||||||
item, adjust a bbox by dragging).
|
|
||||||
4. **Group & select** — user selects items into one or more groups; each
|
|
||||||
group gets its own "split with" member list (loaded from the relevant
|
|
||||||
Cospend project's members) and payer. Same UI, run N times per receipt.
|
|
||||||
5. **Highlight** — backend (or client canvas) draws boxes over selected
|
|
||||||
items' bboxes on the original image, one output image per group.
|
|
||||||
6. **Per group**, backend:
|
|
||||||
- `PUT` highlighted image via WebDAV to a configured folder.
|
|
||||||
- Creates a public share link via OCS Share API.
|
|
||||||
- `createBill` in Cospend: amount = sum of group's item prices, payer,
|
|
||||||
payedFor = group's members, title (`what`) includes the share link
|
|
||||||
(so Cospend's UI shows it as an attachment), comment = itemized list.
|
|
||||||
7. Frontend shows confirmation + links to the created bill(s).
|
|
||||||
|
|
||||||
## Config (env vars)
|
|
||||||
|
|
||||||
- `NC_BASE_URL`, `NC_USERNAME`, `NC_APP_PASSWORD`, `NC_UPLOAD_FOLDER`
|
|
||||||
- `COSPEND_PROJECT_ID` (or a picker if you use more than one project)
|
|
||||||
- `LLM_BASE_URL`, `LLM_API_KEY`, `LLM_MODEL`
|
|
||||||
- Single-user/household tool — one shared backend config, not per-user
|
|
||||||
login. If later you want it multi-tenant (each housemate logging in as
|
|
||||||
themselves), that's a bigger change (real NC OAuth/session per user) —
|
|
||||||
intentionally deferred, not needed for v1.
|
|
||||||
|
|
||||||
## Open items to confirm before/while building
|
|
||||||
|
|
||||||
- Cospend project/member selection is handled live via the API (project
|
|
||||||
picker + member list fetched per request), not hardcoded — resolved.
|
|
||||||
- Where the backend actually runs — decided: no Docker for now, plain
|
|
||||||
`install.sh`/`run.sh` (venv + npm, tmux session with a backend/frontend
|
|
||||||
window each), matching the pattern used in ~/Projects/gain.
|
|
||||||
- Bbox-based highlighting depends on the vision model returning usable
|
|
||||||
coordinates — some models are much better at this than others; may need a
|
|
||||||
fallback (e.g. user manually drags a highlight box) if a given provider's
|
|
||||||
bboxes are unreliable. Worth testing with your candidate receipts early.
|
|
||||||
|
|
||||||
## Build order
|
|
||||||
|
|
||||||
1. Scaffold backend (Flask + SQLite) with the three NC/Cospend calls wired
|
|
||||||
to a real test project — verify the WebDAV→share→bill chain works end to
|
|
||||||
end with a hardcoded fake bill first, before touching OCR.
|
|
||||||
2. Add the LLM extraction endpoint, test against a few real receipt photos,
|
|
||||||
compare providers.
|
|
||||||
3. Scaffold frontend (Vite/React/PWA), wire capture → review → group →
|
|
||||||
submit against the backend.
|
|
||||||
4. Highlighting (canvas draw on selected bboxes).
|
|
||||||
5. Polish: error handling for OCR misses, mobile camera UX, PWA install.
|
|
||||||
Reference in New Issue
Block a user