feat: SQLAlchemy ORM models, filing cache incremental fetch, yfinance price cache

- Replace db/schema.sql + raw sqlite3 with SQLAlchemy ORM (db/models.py)
  - Filing, Signal, PriceCache models with proper indexes
  - db/db.py uses SQLAlchemy sessions throughout; no raw SQL strings
- Add PriceCache table: stores daily close prices per ticker
  - backtest._fetch_prices checks DB first; skips yfinance for completed ranges
  - New data persisted via upsert_prices()
  - get_cached_prices() / upsert_prices() added to db.py
- EDGAR poller incremental fetch: get_latest_filed_date() returns newest
  filed_date in DB; fetch_and_store_new_filings skips entries older than
  that cutoff before even checking accession_exists
- Add get_signals_for_backtest() to db.py; backtest no longer opens its
  own sqlite3 connection
- requirements.txt: add sqlalchemy>=2.0.0

Co-authored-by: dodox <dodox@users.noreply.local>
This commit is contained in:
2026-05-04 17:21:23 +00:00
co-authored by dodox
parent 2e2be3e9c7
commit b119b9abae
7 changed files with 368 additions and 231 deletions
+7 -1
View File
@@ -7,7 +7,7 @@ from lxml import etree, html
import config
from ingestion.form4_parser import parse_form4
from db.db import insert_filing, accession_exists
from db.db import accession_exists, get_latest_filed_date, insert_filing
logger = logging.getLogger(__name__)
@@ -82,7 +82,13 @@ def fetch_and_store_new_filings() -> list[dict]:
logger.error(f"Failed to fetch EDGAR index: {e}")
return new_filings
latest_in_db = get_latest_filed_date()
for _index_url, accession, filed_date in entries:
if latest_in_db and filed_date and filed_date < latest_in_db:
logger.debug(f"Skipping {accession}: filed_date {filed_date} older than latest in DB {latest_in_db}")
continue
if accession_exists(accession):
continue