Drop PLAN.md (its content is now stale/superseded by the actual implementation)
This commit is contained in:
@@ -1,117 +0,0 @@
|
||||
# wgBill — plan
|
||||
|
||||
Webapp to replace the manual "upload to NC → share link → paste into Cospend
|
||||
bill title" flow with: photo → extracted items → pick which ones (per
|
||||
splitting group) → highlighted photo → uploaded + shared → Cospend bill(s)
|
||||
created automatically.
|
||||
|
||||
## Confirmed via API research
|
||||
|
||||
- **WebDAV** (`PUT /remote.php/dav/files/<user>/<path>`) — uploads the
|
||||
receipt/highlighted image. Auth via NC app password (Basic auth).
|
||||
- **OCS Share API** (`POST /ocs/v2.php/apps/files_sharing/api/v1/shares`,
|
||||
`shareType=3`) — creates the public link, response includes ready `url`.
|
||||
- **Cospend API** — verified against the actual source (`appinfo/routes.php`
|
||||
+ `lib/Controller/ApiController.php` + `lib/Db/{Project,Member}.php` in
|
||||
`julien-nc/cospend-nc`), not just doc summaries:
|
||||
- It's an OCS API, same family as the Share API — full paths are under
|
||||
`/ocs/v2.php/apps/cospend/api/v1/...`, need `OCS-APIRequest: true`.
|
||||
- `GET /projects` → `getLocalProjects` — lists the authenticated user's
|
||||
projects (`{id, name, ...}`). No hardcoded project id needed; the app
|
||||
lets you pick.
|
||||
- `GET /projects/{id}/members` → `{id, name, ...}` (member `id` is
|
||||
numeric).
|
||||
- `POST /projects/{id}/bills` → `createBill(projectId, date, what, payer,
|
||||
payedFor, amount, ..., comment, ...)` — **camelCase** params (not the
|
||||
old IHateMoney-style snake_case an earlier doc summary suggested);
|
||||
`payer`/`payedFor` are numeric member ids, `payedFor` is a
|
||||
comma-separated string of them. Returns just the new bill's **id**
|
||||
(int), not a bill object.
|
||||
**No native attachment/link field on a bill.** Confirms the current manual
|
||||
trick (put the link in the bill title) is the only integration point — the
|
||||
app does the same. Cospend's UI only recognizes a bill as "has an
|
||||
attachment" when the link is in `what` (the title), not `comment` — a link
|
||||
in `comment` is just plain text. So: link goes in `what`, itemized
|
||||
breakdown goes in `comment`.
|
||||
- CORS ruled out doing this as a pure static frontend talking to NC directly
|
||||
from a different origin (WebDAV PUT and Login Flow v2 both fight the
|
||||
browser here) — decided to just build a small backend instead.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
[PWA frontend] --(HTTPS, JSON)--> [backend] --(WebDAV / OCS / Cospend API)--> [Nextcloud]
|
||||
|
|
||||
+--(OpenAI-compatible chat completions)--> [vision LLM]
|
||||
```
|
||||
|
||||
- **Backend**: Python + Flask + SQLite. Holds NC app password and the
|
||||
vision-provider API key as server config (env vars), never exposed to the
|
||||
client. SQLite holds the per-receipt session (parsed items, image path,
|
||||
group selections) — durable across a backend restart, unlike an in-memory
|
||||
store, and trivial to inspect/debug directly.
|
||||
- **Frontend**: Vite + React + TypeScript, PWA (installable, camera access
|
||||
via `<input capture>` / `getUserMedia`).
|
||||
- **Vision extraction**: any OpenAI-compatible endpoint. Config = `base_url`
|
||||
+ `api_key` + `model`. Works unmodified for OpenAI; for Gemini point
|
||||
`base_url` at `https://generativelanguage.googleapis.com/v1beta/openai/`.
|
||||
Switch/compare providers via env var, no code change.
|
||||
|
||||
## Data flow
|
||||
|
||||
1. **Capture** — frontend takes/picks a photo, uploads to backend.
|
||||
2. **Extract** — backend sends image to the vision LLM with a prompt asking
|
||||
for strict JSON: `{ items: [{ id, label, price, bbox: [x,y,w,h] }] }`
|
||||
(bbox in normalized 0–1 coords, for the highlight step later). Backend
|
||||
stores the original image on disk and the parsed items in a SQLite
|
||||
`receipts` table, keyed by a random session id; a periodic cleanup (or
|
||||
just TTL-on-read) drops old sessions/images.
|
||||
3. **Review** — frontend shows the parsed list, editable (OCR/LLM extraction
|
||||
won't be perfect — fix a mis-read price, merge a split line, add a missed
|
||||
item, adjust a bbox by dragging).
|
||||
4. **Group & select** — user selects items into one or more groups; each
|
||||
group gets its own "split with" member list (loaded from the relevant
|
||||
Cospend project's members) and payer. Same UI, run N times per receipt.
|
||||
5. **Highlight** — backend (or client canvas) draws boxes over selected
|
||||
items' bboxes on the original image, one output image per group.
|
||||
6. **Per group**, backend:
|
||||
- `PUT` highlighted image via WebDAV to a configured folder.
|
||||
- Creates a public share link via OCS Share API.
|
||||
- `createBill` in Cospend: amount = sum of group's item prices, payer,
|
||||
payedFor = group's members, title (`what`) includes the share link
|
||||
(so Cospend's UI shows it as an attachment), comment = itemized list.
|
||||
7. Frontend shows confirmation + links to the created bill(s).
|
||||
|
||||
## Config (env vars)
|
||||
|
||||
- `NC_BASE_URL`, `NC_USERNAME`, `NC_APP_PASSWORD`, `NC_UPLOAD_FOLDER`
|
||||
- `COSPEND_PROJECT_ID` (or a picker if you use more than one project)
|
||||
- `LLM_BASE_URL`, `LLM_API_KEY`, `LLM_MODEL`
|
||||
- Single-user/household tool — one shared backend config, not per-user
|
||||
login. If later you want it multi-tenant (each housemate logging in as
|
||||
themselves), that's a bigger change (real NC OAuth/session per user) —
|
||||
intentionally deferred, not needed for v1.
|
||||
|
||||
## Open items to confirm before/while building
|
||||
|
||||
- Cospend project/member selection is handled live via the API (project
|
||||
picker + member list fetched per request), not hardcoded — resolved.
|
||||
- Where the backend actually runs — decided: no Docker for now, plain
|
||||
`install.sh`/`run.sh` (venv + npm, tmux session with a backend/frontend
|
||||
window each), matching the pattern used in ~/Projects/gain.
|
||||
- Bbox-based highlighting depends on the vision model returning usable
|
||||
coordinates — some models are much better at this than others; may need a
|
||||
fallback (e.g. user manually drags a highlight box) if a given provider's
|
||||
bboxes are unreliable. Worth testing with your candidate receipts early.
|
||||
|
||||
## Build order
|
||||
|
||||
1. Scaffold backend (Flask + SQLite) with the three NC/Cospend calls wired
|
||||
to a real test project — verify the WebDAV→share→bill chain works end to
|
||||
end with a hardcoded fake bill first, before touching OCR.
|
||||
2. Add the LLM extraction endpoint, test against a few real receipt photos,
|
||||
compare providers.
|
||||
3. Scaffold frontend (Vite/React/PWA), wire capture → review → group →
|
||||
submit against the backend.
|
||||
4. Highlighting (canvas draw on selected bboxes).
|
||||
5. Polish: error handling for OCR misses, mobile camera UX, PWA install.
|
||||
Reference in New Issue
Block a user