nngo/lottosight
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
25534062543ab7bd6f7508405f086de5d070db39
lottosight/CLAUDE.md
T
nngo 2553406254 05/23 Phase 18,19
2026-05-23 16:31:55 -04:00

510 lines
24 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# LottoSight — Project Documentation
## App Overview
**Name:** LottoSight
**Type:** Desktop Application
**Framework:** Python + Tkinter
**Database:** SQLite (local, zero-config)
**Purpose:** Analyze historical lottery drawing data and predict next winning numbers using multiple statistical strategies.
---
## Tech Stack
| Component | Technology |
|---------------|-----------------------------------|
| UI | Tkinter + ttk |
| Charts | Matplotlib (embedded in Tkinter) |
| Database | SQLite via sqlite3 |
| Data Fetch | requests + BeautifulSoup |
| Analysis | collections, statistics, numpy |
| Scheduler | APScheduler |
| Export | openpyxl |
| Packaging | PyInstaller |
---
## Supported Games
| Game | Main Balls | Pool | Bonus Ball | Pool |
|---------------|------------|-------|------------|-------|
| Powerball | 5 | 1–69 | 1 | 1–26 |
| Mega Millions | 5 | 1–70 | 1 | 1–25 |
| Custom | config | config| optional | config|
---
## Data Sources
| Source | Game | Coverage | Method |
|-------------------------|---------------|------------------|--------------|
| NY Open Data API | Powerball | 2010–present | REST API |
| NY Open Data API | Mega Millions | 2002–present | REST API |
| Texas Lottery CSV | Mega Millions | 2003–present | CSV download |
### API Endpoints
- **Powerball (NY):** `https://data.ny.gov/resource/d6yy-54nr.json`
- **Mega Millions (NY):** `https://data.ny.gov/resource/5xaw-6ayf.json`
- **Mega Millions (TX):** `https://www.texaslottery.com/export/sites/lottery/Games/Mega_Millions/Winning_Numbers/download.html`
---
## File Structure
```
lottosight/
├── main.py # Entry point, app window, toolbar
├── db/
│ ├── database.py # SQLite setup, schema, migrations
│ └── models.py # CRUD operations, duplicate check
├── core/
│ ├── analyzer.py # All analysis logic
│ ├── predictor.py # Prediction strategies (5 total)
│ └── fetcher.py # Fetch logic — auto + manual, all 3 sources
├── ui/
│ ├── dashboard.py # Home/summary screen
│ ├── history.py # Draw history browser
│ ├── analysis.py # Charts and stats screen
│ ├── predictor_ui.py # Prediction generator screen
│ ├── settings.py # Settings screen + manual fetch button
│ └── statusbar.py # Bottom status bar component
├── assets/
│ └── icon.png
├── exports/ # Excel/CSV exports output folder
├── requirements.txt
└── CLAUDE.md # This file
```
---
## Database Schema
### Table: `games`
| Column | Type | Description |
|--------------|---------|------------------------------------|
| id | INTEGER | Primary key |
| name | TEXT | Game name (e.g. Powerball) |
| main_count | INTEGER | Number of main balls |
| main_max | INTEGER | Max value for main balls |
| bonus_count | INTEGER | Number of bonus balls (0 if none) |
| bonus_max | INTEGER | Max value for bonus ball |
| active | INTEGER | 1 = active, 0 = disabled |
### Table: `draws`
| Column | Type | Description |
|--------------|---------|------------------------------------|
| id | INTEGER | Primary key |
| game_id | INTEGER | Foreign key → games.id |
| draw_date | TEXT | ISO date string (YYYY-MM-DD) |
| numbers | TEXT | Comma-separated main numbers |
| bonus | TEXT | Bonus ball number(s) |
| multiplier | TEXT | Power Play / Megaplier (nullable) |
| source | TEXT | Data source identifier |
| created_at | TEXT | Record insert timestamp |
### Table: `predictions`
| Column | Type | Description |
|--------------|---------|------------------------------------|
| id | INTEGER | Primary key |
| game_id | INTEGER | Foreign key → games.id |
| strategy | TEXT | Strategy name used |
| numbers | TEXT | Predicted main numbers |
| bonus | TEXT | Predicted bonus number |
| created_at | TEXT | Prediction timestamp |
### Table: `fetch_log`
| Column | Type | Description |
|--------------|---------|------------------------------------|
| id | INTEGER | Primary key |
| source | TEXT | Source name |
| fetched_at | TEXT | Timestamp of fetch |
| added | INTEGER | New records inserted |
| skipped | INTEGER | Duplicate records skipped |
| status | TEXT | success / error |
| message | TEXT | Error message or notes |
---
## Fetch System
### Auto-Fetch
- Triggers on **app launch** (background thread)
- Repeats every **24 hours** via APScheduler
- Runs in background — does not block UI
### Manual Fetch
- Button in **Toolbar** (always visible, all screens)
- Button in **Settings screen**
- Shows "Fetching…" state while running
### Duplicate Handling
- Compare incoming records by `draw_date` + `game_id`
- Skip if already exists in DB
- Report result in **status bar**: `X added, Y skipped`
### Status Bar Format
```
Last fetch: Powerball — 3 added, 2 skipped | Mega Millions — 5 added, 0 skipped | 2026-05-23 08:42 AM
```
---
## Analysis Features
| Feature | Description |
|-----------------------|------------------------------------------------------|
| Frequency analysis | Hot/cold numbers by total draw count |
| Gap/skip analysis | How many draws since each number last appeared |
| Positional frequency | Which numbers appear most in each draw position |
| Pair/triplet patterns | Number combinations that appear together often |
| Odd/even ratio | Ratio of odd vs even numbers per draw |
| Sum range analysis | Distribution of total sums across all draws |
| Delta patterns | Differences between consecutive numbers in a draw |
---
## Prediction Strategies
| # | Strategy | Description |
|---|--------------------|-----------------------------------------------------------|
| 1 | Hot Numbers | Top N most frequent numbers in last X draws |
| 2 | Due Numbers | Numbers overdue based on expected frequency gap |
| 3 | Weighted Random | numpy random choice weighted by historical frequency |
| 4 | Monte Carlo | 10,000 simulations, pick most common result |
| 5 | Positional | Most frequent number per draw position |
---
## UI Screens
| Screen | Description |
|--------------|----------------------------------------------------------|
| Dashboard | Summary stats, last draw result, quick actions |
| History | Searchable/sortable draw history table |
| Analysis | Charts — frequency bar, heatmap, gap chart |
| Predictor | Pick strategy → generate ticket numbers |
| Settings | Game config, data sources, fetch interval, manual fetch |
---
## Logging
All actions are logged to console and optionally to a log file:
- `[FETCH]` — auto/manual fetch events
- `[DB]` — insert, skip, error events
- `[PREDICT]` — prediction generated
- `[EXPORT]` — export actions
- `[ERROR]` — any exception with traceback
---
## Build Phases & To-Do
### ✅ Phase 0 — Planning
- [x] Define app scope and features
- [x] Choose tech stack
- [x] Design database schema
- [x] Design fetch system
- [x] Define data sources and API endpoints
- [x] Create CLAUDE.md
---
### ✅ Phase 1 — Database Setup
- [x] Create `lottosight/` project folder structure
- [x] Write `db/database.py` — SQLite init, create all tables
- [x] Write `db/models.py` — CRUD: insert draw, get draws, check duplicate, insert prediction, insert fetch log
- [x] Seed default game configs (Powerball, Mega Millions)
- [x] Write `requirements.txt`
- [x] Test DB creation and seed on fresh run
- [x] Write `tests/conftest.py` — shared tmp_db fixture
- [x] Write `tests/test_database.py` — 8 tests for database.py (47/47 pass)
- [x] Write `tests/test_models.py` — 39 tests for models.py (47/47 pass)
- [x] Write `.gitea/workflows/ci.yml` — CI on every push (syntax check + pytest)
---
### ✅ Phase 2 — Fetch System
- [x] Write `core/fetcher.py`
- [x] `fetch_powerball_ny()` — NY Open Data API
- [x] `fetch_megamillions_ny()` — NY Open Data API
- [x] `fetch_megamillions_tx()` — Texas Lottery CSV
- [x] `fetch_all()` — calls all 3, aggregates results
- [x] Duplicate detection logic
- [x] Return added/skipped counts per source
- [x] Write `ui/statusbar.py` — bottom status bar widget
- [x] Wire auto-fetch on app launch (background thread)
- [x] Wire 24hr scheduled fetch (APScheduler)
- [x] Write `main.py` — app window, toolbar with manual fetch button
- [x] Wire manual fetch button → `fetch_all()` → update status bar
- [x] Test: fresh DB → fetch → verify records inserted (67/67 pass)
- [x] Test: second fetch → verify duplicates skipped, counts correct
- [x] Test: cross-source dedup (same date, different source → skipped)
---
### ✅ Phase 3 — History Browser
- [x] Write `ui/history.py`
- [x] Treeview table with columns: Game, Date, Numbers, Bonus, Mult., Source
- [x] Filter by game (dropdown)
- [x] Search by date range (From / To entries)
- [x] Sort by column headers (▲/▼ indicators, numeric sort for bonus/multiplier)
- [x] Row count display
- [x] Connect history screen to DB reads via `get_draws_with_game()`
- [x] Auto-refresh after fetch completes
- [x] 13 tests (80/80 total passing)
---
### ✅ Phase 4 — Analysis Engine + Charts
- [x] Write `core/analyzer.py`
- [x] `frequency_analysis(game_id, last_n)`
- [x] `gap_analysis(game_id)`
- [x] `positional_frequency(game_id)`
- [x] `pair_analysis(game_id)`
- [x] `odd_even_ratio(game_id)`
- [x] `sum_range_analysis(game_id)`
- [x] `delta_analysis(game_id)`
- [x] Write `ui/analysis.py`
- [x] Frequency bar chart (Matplotlib + FigureCanvasTkAgg)
- [x] Positional frequency heatmap (imshow, YlOrRd colormap)
- [x] Gap chart (color-coded: blue=recent, red=due)
- [x] Game dropdown + frequency window selector
- [x] NavigationToolbar on each tab for zoom/pan
- [x] 26 tests for all 7 analysis functions (106/106 total passing)
---
### ✅ Phase 5 — Prediction Engine
- [x] Write `core/predictor.py`
- [x] `hot_numbers(game_id, last_n=100)` — top-frequency + most-frequent bonus
- [x] `due_numbers(game_id)` — highest gap numbers from pool
- [x] `weighted_random(game_id)` — numpy weighted choice (min weight 1 for unseen)
- [x] `monte_carlo(game_id, simulations=10000)` — tally-based selection
- [x] `positional_pick(game_id)` — per-position best with dedup
- [x] All strategies: random fallback on empty DB
- [x] Write `ui/predictor_ui.py`
- [x] Game + strategy + ticket count dropdowns
- [x] Generate button (disables during generation)
- [x] Treeview results: #, zero-padded numbers, bonus
- [x] Save to DB (insert_prediction per ticket) + Clear
- [x] Strategy description label
- [x] 18 tests — all 5 strategies × validity + empty DB + edge cases (124/124 total)
---
### ✅ Phase 6 — Settings Screen
- [x] Write `ui/settings.py`
- [x] Game enable/disable checkboxes (set_game_active on toggle)
- [x] Fetch interval display (24 hours, read-only)
- [x] Manual fetch button (shared on_fetch callback from main.py)
- [x] Last fetch timestamp + added/skipped per source (colour-coded)
- [x] DB stats (draws per game + prediction count)
- [x] Scrollable canvas layout for future growth
- [x] Wire settings to DB reads/writes via main.py callback injection
- [x] 14 tests (138/138 total passing)
---
### ✅ Phase 7 — Export + Polish
- [x] Write `core/exporter.py`
- [x] `export_draws_excel(filepath, game_id, date_from, date_to)` — openpyxl, styled header
- [x] `export_draws_csv(filepath, game_id, date_from, date_to)` — stdlib csv
- [x] `export_predictions_excel(filepath, game_id)` — openpyxl
- [x] `export_predictions_csv(filepath, game_id)` — stdlib csv
- [x] `export_frequency_excel(filepath, game_id, last_n)` — number + freq + gap
- [x] `create_icon_png(path, size)` — valid PNG, stdlib only (struct + zlib)
- [x] Add Export Excel + Export CSV buttons to History screen (filter-aware)
- [x] Add Export Excel + Export CSV buttons to Predictor screen (DB predictions)
- [x] Add Export Frequency button to Analysis screen (respects frequency window)
- [x] Auto-create `assets/icon.png` on startup if missing
- [x] Window min-size set (900×600) + resizable layout (all screens)
- [x] Error handling — try/except + messagebox.showerror on all export paths
- [x] 27 tests for exporter (165/165 total passing)
---
### ✅ Phase 13 — Ticket Checker
- [x] Write `core/checker.py`
- [x] `prize_tier(main_matches, bonus_match)` — maps match counts to Jackpot/Match 5/Match 4+Bonus/… tier labels
- [x] `check_ticket(game_id, numbers, bonus)` — compares ticket against all draws, returns matches sorted by quality desc
- [x] `parse_numbers(raw)` — parses space- or comma-separated user input, raises ValueError on bad input
- [x] Add "Check Ticket" tab to `ui/predictor_ui.py` (3rd tab in Notebook)
- [x] Game dropdown, Numbers entry, Bonus entry, Check / Clear buttons
- [x] Hint label with accepted input format
- [x] Results treeview: Draw Date, Draw Numbers, Bonus, Main Hits, Bonus Hit, Prize Tier
- [x] Row colours: purple=Jackpot, green=≥3 matches or bonus hit, grey=low match
- [x] Summary bar: "N draws matched • Best: <tier>"
- [x] Input validation: number count, range check, bonus range
- [x] 27 tests in `tests/test_checker.py` (254/254 total passing)
---
### ✅ Phase 19 — Dashboard Overdue Alert + DB Backup
- [x] Add `backup_db(dest_dir=None) -> str` to `db/database.py` — copies live DB to timestamped file, creates dest dir if needed
- [x] Add `restore_db(source_path: str) -> None` to `db/database.py` — overwrites live DB with backup file, raises FileNotFoundError if missing
- [x] Update `ui/dashboard.py`
- [x] Import `gap_analysis` from `core.analyzer`
- [x] Add "Most Overdue Numbers" section below Hot Numbers
- [x] `_refresh_overdue()` — top-5 highest-gap numbers per active game, shown as orange-highlighted `BallsBar` with gap counts
- [x] Called from `refresh()`
- [x] Update `ui/settings.py`
- [x] Import `backup_db, restore_db` from `db.database`
- [x] Add "Database Actions" section with Backup DB + Restore DB buttons
- [x] `_backup_db()` — runs backup, shows path in success dialog
- [x] `_restore_db()` — file dialog, confirm prompt, restores, advises restart
- [x] Write `tests/test_backup.py` — 10 tests (361/361 total passing)
---
### ✅ Phase 18 — Draw CSV Import
- [x] Write `core/importer.py`
- [x] `_parse_date(text)` — accepts ISO (YYYY-MM-DD), US slash (MM/DD/YYYY), US dash, day-first formats
- [x] `_is_header(row)` — heuristic: first cell is not a valid date
- [x] `_parse_row_lottosight` — handles LottoSight's own export format (Game, Date, Numbers, Bonus, …)
- [x] `_parse_row_generic` — handles wide format (Date, N1, N2, …) and packed format (Date, "N1,N2,…")
- [x] `import_draws_csv(game_id, filepath) -> {added, skipped, errors}` — validates count + range, calls `insert_draw`, deduplicates via existing DB logic
- [x] Update `ui/settings.py`
- [x] Add `filedialog` import
- [x] "Import CSV" button on every game row (builtin and custom)
- [x] `_import_csv(game_id, game_name)` — file dialog → import → show summary with error preview → refresh games
- [x] Write `tests/test_importer.py` — 23 tests (351/351 total passing)
---
### ✅ Phase 17 — Predictor Power Features
- [x] Add `quick_pick(game_id, exclude=None)` to `core/predictor.py` — pure random, no draw history required
- [x] Add `exclude: set | None = None` parameter to all 5 existing strategies + `_random_ticket` + `_fill_to_count`
- [x] Falls back silently to full pool if excluded numbers would leave fewer candidates than `main_count`
- [x] Add `_safe_pool(game, exclude)` helper used by weighted_random and monte_carlo
- [x] Update `ui/predictor_ui.py`
- [x] Add "Quick Pick" to `_STRATEGIES` and `_DESCRIPTIONS`
- [x] Add "Exclude:" entry field to Generate tab bar (comma/space-separated integers)
- [x] Pass parsed exclusion set to strategy on generate
- [x] Add "Copy" button — copies all generated tickets to clipboard as formatted text; enabled/disabled with generate/clear
- [x] Write `tests/test_quick_pick.py` — 20 tests (326/326 total passing)
---
### ✅ Phase 16 — Lottery Ball Display
- [x] Write `ui/widgets.py`
- [x] `ball_color(number, is_bonus) -> (bg, fg)` — range-based colour lookup (pure, no Tk)
- [x] `BallsBar(tk.Canvas)` — draws numbered balls; supports `highlights` dict for per-ball colour override; inherits parent background
- [x] Update `ui/dashboard.py` — last-draw cards use `BallsBar` for numbers; hot-numbers section uses small-radius (`r=11`) balls
- [x] Update `ui/history.py` — ball detail strip below treeview; shows `BallsBar` for selected row; clears on filter/populate
- [x] Update `ui/predictor_ui.py`
- [x] Generate tab: detail strip shows selected ticket as balls
- [x] Check Ticket tab: detail strip shows ticket (green=matched, grey=unmatched) and draw numbers (green=matched) side by side
- [x] Write `tests/test_widgets.py` — 12 tests (308/308 total passing)
---
### ✅ Phase 15 — Odds Calculator
- [x] Write `core/odds.py`
- [x] `total_combinations(main_count, main_max, bonus_count, bonus_max) -> int`
- [x] `prize_odds(main_count, main_max, bonus_count, bonus_max) -> list[dict]` — generates all prize tiers dynamically, sorted hardest-first by odds
- [x] `game_odds(game_id) -> list[dict]` — DB-backed convenience wrapper
- [x] Handles 0-bonus and 1-bonus games; bonus_count ≥ 2 returns jackpot only
- [x] Verified against known Powerball (1 in 292,201,338) and Mega Millions (1 in 302,575,350) jackpot odds
- [x] Add "Odds" tab to `ui/analysis.py` (8th tab, ttk.Treeview — no matplotlib)
- [x] Columns: Prize Tier | Odds (1 in X) | Probability
- [x] Refreshes when game dropdown changes
- [x] Write `tests/test_odds.py` — 22 tests (296/296 total passing)
---
### ✅ Phase 14 — Custom Game Management
- [x] Add `_BUILTIN_GAMES = {"Powerball", "Mega Millions"}` constant to `db/models.py`
- [x] Add `add_game(name, main_count, main_max, bonus_count, bonus_max) -> int`
- [x] Rejects blank names, duplicate names, and builtin game names (ValueError)
- [x] Inserts with `active=1`, returns new row id
- [x] Add `delete_game(game_id) -> bool`
- [x] Refuses builtin games → returns False
- [x] Refuses games with existing draw records → returns False
- [x] Cascades: deletes predictions for game first, then removes game → returns True
- [x] Update `ui/settings.py`
- [x] Delete button on each non-builtin game row; warns if draws exist, asks confirm
- [x] "+ Add Custom Game" button at bottom of Games section
- [x] `_AddGameDialog` modal (tk.Toplevel): name + main_count/max + bonus_count/max spinboxes
- [x] Error label in dialog for inline validation feedback
- [x] Refresh games list after add or delete
- [x] Write `tests/test_custom_games.py` — 20 tests (274/274 total passing)
---
### ✅ Phase 12 — Number Search + Next Draw Schedule
- [x] Number search in History screen
- [x] Add `number` param to `get_draws_with_game()` — SQL: `',' || numbers || ',' LIKE ?` for exact boundary matching
- [x] Add "Number:" entry field to History filter bar; bound to `<Return>` for quick search
- [x] Wire through `_apply_filter()` and `_clear_filter()`
- [x] 7 new tests — exact match, boundary false-positive prevention, combined with game/date filter
- [x] Next draw schedule on Dashboard
- [x] `_next_draw_date(game_name, from_date)` — computes next Powerball (Mon/Wed/Sat) or Mega Millions (Tue/Fri) draw date
- [x] Shown on each Last Draw card as "Next draw: Sat, May 24" in green
- [x] 6 new tests — day-of-week transitions, unknown game, always-in-future guarantee
- [x] 227/227 tests passing
---
### ✅ Phase 11 — Saved Predictions Viewer
- [x] Add `delete_prediction(pred_id)` and `delete_all_predictions(game_id=None)` to `db/models.py`
- [x] Refactor `ui/predictor_ui.py` to ttk.Notebook with two tabs
- [x] Generate tab — existing UI unchanged
- [x] Saved tab — treeview of all DB predictions (ID, Game, Strategy, Numbers, Bonus, Matches, Saved)
- [x] Match column — shows "X/5" comparing prediction vs last real draw (green if > 0, grey if zero)
- [x] Game filter dropdown — show all games or filter to one
- [x] Delete Selected — removes checked rows from DB + refreshes
- [x] Clear All — confirm dialog, then wipes all (or game-filtered) predictions
- [x] Auto-refreshes Saved tab after Save to DB
- [x] `_count_matches()` helper — set intersection between prediction and last draw numbers
- [x] `tests/test_saved_predictions.py` — 17 tests (214/214 total passing)
---
### ✅ Phase 10 — Complete Analysis Screen (7 Charts)
- [x] Extend `ui/analysis.py` with 4 new chart tabs (was 3, now 7)
- [x] Pairs — horizontal bar chart, top-20 most common number pairs
- [x] Odd/Even — bar chart of draw-split distribution (e.g. 3O/2E = 38%)
- [x] Sum Range — histogram of total draw-sum distribution
- [x] Deltas — bar chart of gaps between consecutive numbers within draws
- [x] Add `_draw_pairs`, `_draw_odd_even`, `_draw_sum_range`, `_draw_deltas` pure functions
- [x] `tests/test_analysis_charts.py` — 23 tests using Agg backend (no display needed)
- [x] 200/200 total tests passing
---
### ✅ Phase 9 — Dashboard Screen
- [x] Write `ui/dashboard.py`
- [x] Last Draw Results — card per active game (date, numbers, bonus, multiplier)
- [x] Database Summary — draw counts per game + prediction total
- [x] Hot Numbers — top-5 most-frequent per game (last 100 draws) with counts
- [x] Scrollable canvas layout (same pattern as Settings)
- [x] `on_fetch` callback injection for future toolbar wiring
- [x] Wire Dashboard into `main.py` — replaces "coming soon" placeholder; opens on launch
- [x] 12 tests (177/177 total passing)
---
### ✅ Phase 8 — Packaging
- [x] Write `core/paths.py` — `user_data_dir()` + `bundled_asset()` helpers (frozen-aware)
- [x] Update `db/database.py` + `core/exporter.py` to use `user_data_dir()` so DB + exports land next to the .exe when frozen
- [x] Update `main.py` to use `bundled_asset()` for icon lookup
- [x] Write `lottosight.spec` — directory build, no console, bundles `assets/`, matplotlib data, openpyxl templates, APScheduler hidden imports
- [x] Build verified: `pyinstaller lottosight.spec --clean` succeeds → `dist/LottoSight/` (~115 MB)
- [x] `assets/icon.png` confirmed present in `dist/LottoSight/_internal/assets/`
- [x] Fix `.gitignore` — un-ignore `*.spec`, add `data/*.db` + `exports/`
- [x] 165/165 tests still passing after paths refactor
- [ ] Test packaged app on a clean machine (no Python installed)
---
## Notes
- SQLite DB file stored at: `lottosight/data/lottosight.db`
- Exports land in: `lottosight/exports/`
- All fetch operations run in **background threads** to keep UI responsive
- NY Open Data API supports `$limit` and `$offset` params for pagination
- Texas Lottery CSV requires parsing — columns vary, needs header detection
- Powerball matrix changed Oct 7, 2015 (5/69+1/26) — pre-2015 data has different pool sizes, flag in DB or filter
Reference in New Issue View Git Blame Copy Permalink