388 lines
17 KiB
Markdown
388 lines
17 KiB
Markdown
# LottoSight — Project Documentation
|
||
|
||
## App Overview
|
||
|
||
**Name:** LottoSight
|
||
**Type:** Desktop Application
|
||
**Framework:** Python + Tkinter
|
||
**Database:** SQLite (local, zero-config)
|
||
**Purpose:** Analyze historical lottery drawing data and predict next winning numbers using multiple statistical strategies.
|
||
|
||
---
|
||
|
||
## Tech Stack
|
||
|
||
| Component | Technology |
|
||
|---------------|-----------------------------------|
|
||
| UI | Tkinter + ttk |
|
||
| Charts | Matplotlib (embedded in Tkinter) |
|
||
| Database | SQLite via sqlite3 |
|
||
| Data Fetch | requests + BeautifulSoup |
|
||
| Analysis | collections, statistics, numpy |
|
||
| Scheduler | APScheduler |
|
||
| Export | openpyxl |
|
||
| Packaging | PyInstaller |
|
||
|
||
---
|
||
|
||
## Supported Games
|
||
|
||
| Game | Main Balls | Pool | Bonus Ball | Pool |
|
||
|---------------|------------|-------|------------|-------|
|
||
| Powerball | 5 | 1–69 | 1 | 1–26 |
|
||
| Mega Millions | 5 | 1–70 | 1 | 1–25 |
|
||
| Custom | config | config| optional | config|
|
||
|
||
---
|
||
|
||
## Data Sources
|
||
|
||
| Source | Game | Coverage | Method |
|
||
|-------------------------|---------------|------------------|--------------|
|
||
| NY Open Data API | Powerball | 2010–present | REST API |
|
||
| NY Open Data API | Mega Millions | 2002–present | REST API |
|
||
| Texas Lottery CSV | Mega Millions | 2003–present | CSV download |
|
||
|
||
### API Endpoints
|
||
- **Powerball (NY):** `https://data.ny.gov/resource/d6yy-54nr.json`
|
||
- **Mega Millions (NY):** `https://data.ny.gov/resource/5xaw-6ayf.json`
|
||
- **Mega Millions (TX):** `https://www.texaslottery.com/export/sites/lottery/Games/Mega_Millions/Winning_Numbers/download.html`
|
||
|
||
---
|
||
|
||
## File Structure
|
||
|
||
```
|
||
lottosight/
|
||
├── main.py # Entry point, app window, toolbar
|
||
├── db/
|
||
│ ├── database.py # SQLite setup, schema, migrations
|
||
│ └── models.py # CRUD operations, duplicate check
|
||
├── core/
|
||
│ ├── analyzer.py # All analysis logic
|
||
│ ├── predictor.py # Prediction strategies (5 total)
|
||
│ └── fetcher.py # Fetch logic — auto + manual, all 3 sources
|
||
├── ui/
|
||
│ ├── dashboard.py # Home/summary screen
|
||
│ ├── history.py # Draw history browser
|
||
│ ├── analysis.py # Charts and stats screen
|
||
│ ├── predictor_ui.py # Prediction generator screen
|
||
│ ├── settings.py # Settings screen + manual fetch button
|
||
│ └── statusbar.py # Bottom status bar component
|
||
├── assets/
|
||
│ └── icon.png
|
||
├── exports/ # Excel/CSV exports output folder
|
||
├── requirements.txt
|
||
└── CLAUDE.md # This file
|
||
```
|
||
|
||
---
|
||
|
||
## Database Schema
|
||
|
||
### Table: `games`
|
||
| Column | Type | Description |
|
||
|--------------|---------|------------------------------------|
|
||
| id | INTEGER | Primary key |
|
||
| name | TEXT | Game name (e.g. Powerball) |
|
||
| main_count | INTEGER | Number of main balls |
|
||
| main_max | INTEGER | Max value for main balls |
|
||
| bonus_count | INTEGER | Number of bonus balls (0 if none) |
|
||
| bonus_max | INTEGER | Max value for bonus ball |
|
||
| active | INTEGER | 1 = active, 0 = disabled |
|
||
|
||
### Table: `draws`
|
||
| Column | Type | Description |
|
||
|--------------|---------|------------------------------------|
|
||
| id | INTEGER | Primary key |
|
||
| game_id | INTEGER | Foreign key → games.id |
|
||
| draw_date | TEXT | ISO date string (YYYY-MM-DD) |
|
||
| numbers | TEXT | Comma-separated main numbers |
|
||
| bonus | TEXT | Bonus ball number(s) |
|
||
| multiplier | TEXT | Power Play / Megaplier (nullable) |
|
||
| source | TEXT | Data source identifier |
|
||
| created_at | TEXT | Record insert timestamp |
|
||
|
||
### Table: `predictions`
|
||
| Column | Type | Description |
|
||
|--------------|---------|------------------------------------|
|
||
| id | INTEGER | Primary key |
|
||
| game_id | INTEGER | Foreign key → games.id |
|
||
| strategy | TEXT | Strategy name used |
|
||
| numbers | TEXT | Predicted main numbers |
|
||
| bonus | TEXT | Predicted bonus number |
|
||
| created_at | TEXT | Prediction timestamp |
|
||
|
||
### Table: `fetch_log`
|
||
| Column | Type | Description |
|
||
|--------------|---------|------------------------------------|
|
||
| id | INTEGER | Primary key |
|
||
| source | TEXT | Source name |
|
||
| fetched_at | TEXT | Timestamp of fetch |
|
||
| added | INTEGER | New records inserted |
|
||
| skipped | INTEGER | Duplicate records skipped |
|
||
| status | TEXT | success / error |
|
||
| message | TEXT | Error message or notes |
|
||
|
||
---
|
||
|
||
## Fetch System
|
||
|
||
### Auto-Fetch
|
||
- Triggers on **app launch** (background thread)
|
||
- Repeats every **24 hours** via APScheduler
|
||
- Runs in background — does not block UI
|
||
|
||
### Manual Fetch
|
||
- Button in **Toolbar** (always visible, all screens)
|
||
- Button in **Settings screen**
|
||
- Shows "Fetching…" state while running
|
||
|
||
### Duplicate Handling
|
||
- Compare incoming records by `draw_date` + `game_id`
|
||
- Skip if already exists in DB
|
||
- Report result in **status bar**: `X added, Y skipped`
|
||
|
||
### Status Bar Format
|
||
```
|
||
Last fetch: Powerball — 3 added, 2 skipped | Mega Millions — 5 added, 0 skipped | 2026-05-23 08:42 AM
|
||
```
|
||
|
||
---
|
||
|
||
## Analysis Features
|
||
|
||
| Feature | Description |
|
||
|-----------------------|------------------------------------------------------|
|
||
| Frequency analysis | Hot/cold numbers by total draw count |
|
||
| Gap/skip analysis | How many draws since each number last appeared |
|
||
| Positional frequency | Which numbers appear most in each draw position |
|
||
| Pair/triplet patterns | Number combinations that appear together often |
|
||
| Odd/even ratio | Ratio of odd vs even numbers per draw |
|
||
| Sum range analysis | Distribution of total sums across all draws |
|
||
| Delta patterns | Differences between consecutive numbers in a draw |
|
||
|
||
---
|
||
|
||
## Prediction Strategies
|
||
|
||
| # | Strategy | Description |
|
||
|---|--------------------|-----------------------------------------------------------|
|
||
| 1 | Hot Numbers | Top N most frequent numbers in last X draws |
|
||
| 2 | Due Numbers | Numbers overdue based on expected frequency gap |
|
||
| 3 | Weighted Random | numpy random choice weighted by historical frequency |
|
||
| 4 | Monte Carlo | 10,000 simulations, pick most common result |
|
||
| 5 | Positional | Most frequent number per draw position |
|
||
|
||
---
|
||
|
||
## UI Screens
|
||
|
||
| Screen | Description |
|
||
|--------------|----------------------------------------------------------|
|
||
| Dashboard | Summary stats, last draw result, quick actions |
|
||
| History | Searchable/sortable draw history table |
|
||
| Analysis | Charts — frequency bar, heatmap, gap chart |
|
||
| Predictor | Pick strategy → generate ticket numbers |
|
||
| Settings | Game config, data sources, fetch interval, manual fetch |
|
||
|
||
---
|
||
|
||
## Logging
|
||
|
||
All actions are logged to console and optionally to a log file:
|
||
- `[FETCH]` — auto/manual fetch events
|
||
- `[DB]` — insert, skip, error events
|
||
- `[PREDICT]` — prediction generated
|
||
- `[EXPORT]` — export actions
|
||
- `[ERROR]` — any exception with traceback
|
||
|
||
---
|
||
|
||
## Build Phases & To-Do
|
||
|
||
### ✅ Phase 0 — Planning
|
||
- [x] Define app scope and features
|
||
- [x] Choose tech stack
|
||
- [x] Design database schema
|
||
- [x] Design fetch system
|
||
- [x] Define data sources and API endpoints
|
||
- [x] Create CLAUDE.md
|
||
|
||
---
|
||
|
||
### ✅ Phase 1 — Database Setup
|
||
- [x] Create `lottosight/` project folder structure
|
||
- [x] Write `db/database.py` — SQLite init, create all tables
|
||
- [x] Write `db/models.py` — CRUD: insert draw, get draws, check duplicate, insert prediction, insert fetch log
|
||
- [x] Seed default game configs (Powerball, Mega Millions)
|
||
- [x] Write `requirements.txt`
|
||
- [x] Test DB creation and seed on fresh run
|
||
- [x] Write `tests/conftest.py` — shared tmp_db fixture
|
||
- [x] Write `tests/test_database.py` — 8 tests for database.py (47/47 pass)
|
||
- [x] Write `tests/test_models.py` — 39 tests for models.py (47/47 pass)
|
||
- [x] Write `.gitea/workflows/ci.yml` — CI on every push (syntax check + pytest)
|
||
|
||
---
|
||
|
||
### ✅ Phase 2 — Fetch System
|
||
- [x] Write `core/fetcher.py`
|
||
- [x] `fetch_powerball_ny()` — NY Open Data API
|
||
- [x] `fetch_megamillions_ny()` — NY Open Data API
|
||
- [x] `fetch_megamillions_tx()` — Texas Lottery CSV
|
||
- [x] `fetch_all()` — calls all 3, aggregates results
|
||
- [x] Duplicate detection logic
|
||
- [x] Return added/skipped counts per source
|
||
- [x] Write `ui/statusbar.py` — bottom status bar widget
|
||
- [x] Wire auto-fetch on app launch (background thread)
|
||
- [x] Wire 24hr scheduled fetch (APScheduler)
|
||
- [x] Write `main.py` — app window, toolbar with manual fetch button
|
||
- [x] Wire manual fetch button → `fetch_all()` → update status bar
|
||
- [x] Test: fresh DB → fetch → verify records inserted (67/67 pass)
|
||
- [x] Test: second fetch → verify duplicates skipped, counts correct
|
||
- [x] Test: cross-source dedup (same date, different source → skipped)
|
||
|
||
---
|
||
|
||
### ✅ Phase 3 — History Browser
|
||
- [x] Write `ui/history.py`
|
||
- [x] Treeview table with columns: Game, Date, Numbers, Bonus, Mult., Source
|
||
- [x] Filter by game (dropdown)
|
||
- [x] Search by date range (From / To entries)
|
||
- [x] Sort by column headers (▲/▼ indicators, numeric sort for bonus/multiplier)
|
||
- [x] Row count display
|
||
- [x] Connect history screen to DB reads via `get_draws_with_game()`
|
||
- [x] Auto-refresh after fetch completes
|
||
- [x] 13 tests (80/80 total passing)
|
||
|
||
---
|
||
|
||
### ✅ Phase 4 — Analysis Engine + Charts
|
||
- [x] Write `core/analyzer.py`
|
||
- [x] `frequency_analysis(game_id, last_n)`
|
||
- [x] `gap_analysis(game_id)`
|
||
- [x] `positional_frequency(game_id)`
|
||
- [x] `pair_analysis(game_id)`
|
||
- [x] `odd_even_ratio(game_id)`
|
||
- [x] `sum_range_analysis(game_id)`
|
||
- [x] `delta_analysis(game_id)`
|
||
- [x] Write `ui/analysis.py`
|
||
- [x] Frequency bar chart (Matplotlib + FigureCanvasTkAgg)
|
||
- [x] Positional frequency heatmap (imshow, YlOrRd colormap)
|
||
- [x] Gap chart (color-coded: blue=recent, red=due)
|
||
- [x] Game dropdown + frequency window selector
|
||
- [x] NavigationToolbar on each tab for zoom/pan
|
||
- [x] 26 tests for all 7 analysis functions (106/106 total passing)
|
||
|
||
---
|
||
|
||
### ✅ Phase 5 — Prediction Engine
|
||
- [x] Write `core/predictor.py`
|
||
- [x] `hot_numbers(game_id, last_n=100)` — top-frequency + most-frequent bonus
|
||
- [x] `due_numbers(game_id)` — highest gap numbers from pool
|
||
- [x] `weighted_random(game_id)` — numpy weighted choice (min weight 1 for unseen)
|
||
- [x] `monte_carlo(game_id, simulations=10000)` — tally-based selection
|
||
- [x] `positional_pick(game_id)` — per-position best with dedup
|
||
- [x] All strategies: random fallback on empty DB
|
||
- [x] Write `ui/predictor_ui.py`
|
||
- [x] Game + strategy + ticket count dropdowns
|
||
- [x] Generate button (disables during generation)
|
||
- [x] Treeview results: #, zero-padded numbers, bonus
|
||
- [x] Save to DB (insert_prediction per ticket) + Clear
|
||
- [x] Strategy description label
|
||
- [x] 18 tests — all 5 strategies × validity + empty DB + edge cases (124/124 total)
|
||
|
||
---
|
||
|
||
### ✅ Phase 6 — Settings Screen
|
||
- [x] Write `ui/settings.py`
|
||
- [x] Game enable/disable checkboxes (set_game_active on toggle)
|
||
- [x] Fetch interval display (24 hours, read-only)
|
||
- [x] Manual fetch button (shared on_fetch callback from main.py)
|
||
- [x] Last fetch timestamp + added/skipped per source (colour-coded)
|
||
- [x] DB stats (draws per game + prediction count)
|
||
- [x] Scrollable canvas layout for future growth
|
||
- [x] Wire settings to DB reads/writes via main.py callback injection
|
||
- [x] 14 tests (138/138 total passing)
|
||
|
||
---
|
||
|
||
### ✅ Phase 7 — Export + Polish
|
||
- [x] Write `core/exporter.py`
|
||
- [x] `export_draws_excel(filepath, game_id, date_from, date_to)` — openpyxl, styled header
|
||
- [x] `export_draws_csv(filepath, game_id, date_from, date_to)` — stdlib csv
|
||
- [x] `export_predictions_excel(filepath, game_id)` — openpyxl
|
||
- [x] `export_predictions_csv(filepath, game_id)` — stdlib csv
|
||
- [x] `export_frequency_excel(filepath, game_id, last_n)` — number + freq + gap
|
||
- [x] `create_icon_png(path, size)` — valid PNG, stdlib only (struct + zlib)
|
||
- [x] Add Export Excel + Export CSV buttons to History screen (filter-aware)
|
||
- [x] Add Export Excel + Export CSV buttons to Predictor screen (DB predictions)
|
||
- [x] Add Export Frequency button to Analysis screen (respects frequency window)
|
||
- [x] Auto-create `assets/icon.png` on startup if missing
|
||
- [x] Window min-size set (900×600) + resizable layout (all screens)
|
||
- [x] Error handling — try/except + messagebox.showerror on all export paths
|
||
- [x] 27 tests for exporter (165/165 total passing)
|
||
|
||
---
|
||
|
||
### ✅ Phase 11 — Saved Predictions Viewer
|
||
- [x] Add `delete_prediction(pred_id)` and `delete_all_predictions(game_id=None)` to `db/models.py`
|
||
- [x] Refactor `ui/predictor_ui.py` to ttk.Notebook with two tabs
|
||
- [x] Generate tab — existing UI unchanged
|
||
- [x] Saved tab — treeview of all DB predictions (ID, Game, Strategy, Numbers, Bonus, Matches, Saved)
|
||
- [x] Match column — shows "X/5" comparing prediction vs last real draw (green if > 0, grey if zero)
|
||
- [x] Game filter dropdown — show all games or filter to one
|
||
- [x] Delete Selected — removes checked rows from DB + refreshes
|
||
- [x] Clear All — confirm dialog, then wipes all (or game-filtered) predictions
|
||
- [x] Auto-refreshes Saved tab after Save to DB
|
||
- [x] `_count_matches()` helper — set intersection between prediction and last draw numbers
|
||
- [x] `tests/test_saved_predictions.py` — 17 tests (214/214 total passing)
|
||
|
||
---
|
||
|
||
### ✅ Phase 10 — Complete Analysis Screen (7 Charts)
|
||
- [x] Extend `ui/analysis.py` with 4 new chart tabs (was 3, now 7)
|
||
- [x] Pairs — horizontal bar chart, top-20 most common number pairs
|
||
- [x] Odd/Even — bar chart of draw-split distribution (e.g. 3O/2E = 38%)
|
||
- [x] Sum Range — histogram of total draw-sum distribution
|
||
- [x] Deltas — bar chart of gaps between consecutive numbers within draws
|
||
- [x] Add `_draw_pairs`, `_draw_odd_even`, `_draw_sum_range`, `_draw_deltas` pure functions
|
||
- [x] `tests/test_analysis_charts.py` — 23 tests using Agg backend (no display needed)
|
||
- [x] 200/200 total tests passing
|
||
|
||
---
|
||
|
||
### ✅ Phase 9 — Dashboard Screen
|
||
- [x] Write `ui/dashboard.py`
|
||
- [x] Last Draw Results — card per active game (date, numbers, bonus, multiplier)
|
||
- [x] Database Summary — draw counts per game + prediction total
|
||
- [x] Hot Numbers — top-5 most-frequent per game (last 100 draws) with counts
|
||
- [x] Scrollable canvas layout (same pattern as Settings)
|
||
- [x] `on_fetch` callback injection for future toolbar wiring
|
||
- [x] Wire Dashboard into `main.py` — replaces "coming soon" placeholder; opens on launch
|
||
- [x] 12 tests (177/177 total passing)
|
||
|
||
---
|
||
|
||
### ✅ Phase 8 — Packaging
|
||
- [x] Write `core/paths.py` — `user_data_dir()` + `bundled_asset()` helpers (frozen-aware)
|
||
- [x] Update `db/database.py` + `core/exporter.py` to use `user_data_dir()` so DB + exports land next to the .exe when frozen
|
||
- [x] Update `main.py` to use `bundled_asset()` for icon lookup
|
||
- [x] Write `lottosight.spec` — directory build, no console, bundles `assets/`, matplotlib data, openpyxl templates, APScheduler hidden imports
|
||
- [x] Build verified: `pyinstaller lottosight.spec --clean` succeeds → `dist/LottoSight/` (~115 MB)
|
||
- [x] `assets/icon.png` confirmed present in `dist/LottoSight/_internal/assets/`
|
||
- [x] Fix `.gitignore` — un-ignore `*.spec`, add `data/*.db` + `exports/`
|
||
- [x] 165/165 tests still passing after paths refactor
|
||
- [ ] Test packaged app on a clean machine (no Python installed)
|
||
|
||
---
|
||
|
||
## Notes
|
||
|
||
- SQLite DB file stored at: `lottosight/data/lottosight.db`
|
||
- Exports land in: `lottosight/exports/`
|
||
- All fetch operations run in **background threads** to keep UI responsive
|
||
- NY Open Data API supports `$limit` and `$offset` params for pagination
|
||
- Texas Lottery CSV requires parsing — columns vary, needs header detection
|
||
- Powerball matrix changed Oct 7, 2015 (5/69+1/26) — pre-2015 data has different pool sizes, flag in DB or filter
|