nngo/Personal-Finance-Management
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4fe1a4acefdee5ffd83254bc84e0d4158204e5ef
Personal-Finance-Management/CLAUDE.md
T
nngo 8b1594da5a 05/31 Update documents
2026-05-31 17:28:50 -04:00

19 KiB
Raw Blame History

Personal Finance Management System (PFM)

Stack: Python Flask · MySQL · Ubuntu Server · Nginx · Gunicorn · Groq API (free AI) App URL: https://pfm.ngodanguyen.tech Code: /home/pfm/web · User: pfm · Gitea: gitea.ngodanguyen.tech

1. Project Overview

Self-hosted personal finance web app. Tracks income, expenses, investments. AI assistant powered by Groq API (free tier, fast inference, no local hardware). Receipt OCR via Groq vision model. Everything runs on Ubuntu server behind Nginx + Certbot SSL.

Status: All 7 phases complete + Receipt OCR post-MVP feature.

2. Core Features (Implemented)

2.1 Dashboard

  • Net worth snapshot (assets − liabilities)
  • Monthly cash flow bar chart (6 months)
  • Budget utilization per category
  • Recent transactions feed (last 8)
  • AI daily insight card (Groq-generated, stored in DB)
  • USD → VND exchange rate widget — reference only, independent of app currency
    • Click to expand 30-day history chart
    • ↻ refresh button (force-fetches fresh rate without page reload)
    • Source label shown (yfinance / exchangerate-api)
    • Stale indicator if rate > 1 day old
  • Period selector: This Month / Last Month / Custom date range

2.2 Transactions

  • Income + Expense entry with Income/Expense tabs
  • Transfer between accounts
  • Filter: search, category, account, date range
  • Pagination (30/page)
  • Receipt upload (PNG/JPG/WEBP/GIF/PDF, max 10MB)
  • AI Receipt OCR — drag-drop receipt image → Groq vision extracts amount/date/merchant/category → auto-fills form
  • Re-extract from already-uploaded receipt (edit mode)
  • Export to CSV / Excel

2.3 Accounts

  • Types: checking, savings, cash, credit_card, crypto, investment, other
  • Balance auto-calculated from all transactions (not manually entered)
  • Color + icon picker
  • Soft delete

2.4 Categories

  • Expense + Income categories with color/icon
  • System categories (protected from delete)
  • Custom categories (user-created)
  • 14 expense + 7 income defaults seeded on init

2.5 Budget Planner

  • Monthly limits per expense category
  • Progress bars: green → amber (80%) → red (100%+)
  • Rollover unused budget to next month (toggle)
  • Copy previous month's budgets in one click
  • Unbudgeted spending shown with "Set Budget" prompt

2.6 Goals & Savings

  • Goals with target amount, target date, color, icon
  • Contribution tracking + history
  • Progress bar + projected completion date (based on avg monthly contrib)
  • Auto-complete on 100%
  • Emergency fund tracker (liquid assets vs 3-month / 6-month expense targets)

2.7 Investments

  • Asset types: stock, ETF, crypto, real_estate, bond, cash, other
  • Buy/sell/dividend/split transaction log
  • FIFO cost basis auto-recalculated from transaction log
  • yfinance price auto-fetch (daily 4PM weekdays via cron)
  • Manual price refresh button (portfolio page)
  • Live ticker check on add form
  • Doughnut allocation chart
  • P&L per holding + portfolio total

2.8 AI Financial Assistant

  • Chat UI with SSE streaming (Groq API, word-by-word response)
  • Context: last 90 days transactions + budget status + goals + investments
  • 8 suggested question buttons
  • Daily auto-insight generated at midnight (stored in ai_insights table)
  • Manual "Generate Now" button
  • Chat history page
  • Model: llama-3.3-70b-versatile (default) or llama-3.1-8b-instant (fast)
  • Fallback messages for rate limit / invalid key / unavailable

2.9 Receipt OCR (Groq Vision)

  • Model: meta-llama/llama-4-scout-17b-16e-instruct
  • Drag-drop or click-to-upload on new expense form
  • Extracts: amount, date, merchant name, category suggestion, notes
  • Maps category suggestion → system category ID
  • Auto-fills form fields with green flash animation
  • Re-extract button on existing receipt (edit mode)
  • Auto-triggers OCR when image file selected in edit mode
  • Handles fenced markdown JSON output from LLM
  • Sanity check: rejects amounts < 1000 (catches garbage values)
  • Full error handling: 400/401/429/timeout/bad JSON

2.10 Reports & Export

  • Monthly / Quarterly / Yearly summary reports
  • Tax year summary (all income by source, all expenses by category)
  • Net worth history line chart (from monthly snapshots)
  • Category spending trends (top 6 categories, 6-month line chart)
  • "Snapshot Now" manual button
  • Export: CSV, Excel (color-coded, formatted), PDF (WeasyPrint)

2.11 Settings

  • Profile: name, email, timezone, currency, Groq model
  • 8 currency options (USD/VND/EUR/GBP/JPY/AUD/CAD/SGD) — auto-updates symbol
  • Password change (requires current password)
  • Recurring rules: CRUD, pause/enable, frequency (daily/weekly/biweekly/monthly/quarterly/yearly)
  • "Run Now" button to process due rules immediately
  • CSV import: upload → preview with ⚠ warnings → confirm
  • Upcoming recurring transactions (30-day view)

2.12 USD → VND Exchange Rate

  • Reference widget only — not used in transaction calculations
  • Primary source: yfinance USDVND=X (Yahoo Finance forex)
  • Fallback: open.er-api.com (free, no key)
  • Sanity check: rate must be > 1000 (rejects garbage values)
  • force_refresh() — always fetches fresh on cron, bypasses cache
  • DB caches one record per day (fx_rates table)
  • Dashboard: shows rate, date, source, stale warning, ↻ refresh button
  • 30-day history chart (click widget to expand)

3. Database Schema (MySQL)

All 14 Tables

users                   — single user, hashed password, currency/timezone prefs
accounts                — bank/wallet accounts (balance auto-calc from txns)
categories              — expense/income categories with color/icon
transactions            — income/expense/transfer, receipt_id, recurring_rule_id
receipts                — receipt file metadata (filename, size, mime_type)
recurring_rules         — templates: frequency, next_run, start/end date
budgets                 — monthly limits per category, rollover support
goals                   — savings goals with target amount/date
goal_contributions      — individual deposits toward each goal
investments             — holdings: ticker, shares, avg_cost_basis, current_price
investment_transactions — buy/sell/dividend/split log
net_worth_snapshots     — monthly snapshots: assets, liabilities, net_worth (JSON)
ai_insights             — stored AI responses: daily_summary / chat_response
fx_rates                — daily USD/VND rate cache (date UNIQUE, source)

Key Column Notes

  • transactions.balance — NOT stored; calculated on-demand via account_service.calc_balance()
  • investments.shares / avg_cost_basis — recalculated from investment_transactions (FIFO)
  • goals.current_amount — updated on each contribution add/delete
  • net_worth_snapshots.account_balances — JSON snapshot of each account balance at time of snapshot

4. Project File Structure (Actual)

pfm/                             # /home/pfm/web on server
├── wsgi.py                      # Gunicorn entry — sys.path fix included
├── requirements.txt
├── .env                         # Not committed
├── .env.example
├── .gitignore
│
├── app/
│   ├── __init__.py              # Flask app factory, all blueprints registered
│   ├── config.py                # Dev/Prod configs, SESSION_COOKIE_SECURE in prod
│   ├── extensions.py            # db, login_manager, migrate, csrf
│   │
│   ├── models/
│   │   ├── __init__.py          # Imports all models (required for Flask-Migrate)
│   │   ├── user.py              # UserMixin, set/check password, load_user hook
│   │   ├── account.py           # account_type enum, color, icon
│   │   ├── category.py          # Self-referential (subcategories), is_system flag
│   │   ├── transaction.py       # Dual FK to accounts (account_id + to_account_id)
│   │   ├── receipt.py           # filename, original_filename, mime_type
│   │   ├── recurring_rule.py    # frequency enum, next_run date
│   │   ├── budget.py            # UniqueConstraint(category_id, month)
│   │   ├── goal.py              # progress_percent property
│   │   ├── investment.py        # total_cost/current_value/unrealized_gain properties
│   │   ├── net_worth_snapshot.py # account_balances JSON field
│   │   ├── ai_insight.py        # insight_type enum
│   │   └── fx_rate.py           # date UNIQUE index
│   │
│   ├── routes/
│   │   ├── auth.py              # /auth/login, /auth/logout
│   │   ├── dashboard.py         # /, /api/fx-history, /api/fx-refresh (POST)
│   │   ├── accounts.py          # /accounts/
│   │   ├── categories.py        # /categories/
│   │   ├── transactions.py      # /transactions/, /transactions/ocr (POST),
│   │   │                        # /transactions/ocr-file (POST)
│   │   ├── budgets.py           # /budgets/, /budgets/copy (POST)
│   │   ├── goals.py             # /goals/, contribute, contributions, delete_contribution
│   │   ├── investments.py       # /investments/, detail, add_transaction,
│   │   │                        # refresh-prices, /api/price/<ticker>
│   │   ├── reports.py           # /reports/monthly|quarterly|yearly|tax
│   │   │                        # /reports/export/csv|excel|pdf
│   │   ├── ai.py                # /ai/, /ai/stream (SSE), /ai/history,
│   │   │                        # /ai/generate-insight (POST)
│   │   └── settings.py          # /settings/, profile, password, recurring,
│   │                            # import, upload_receipt, delete_receipt, view_receipt
│   │
│   ├── services/
│   │   ├── account_service.py   # calc_balance(), recalc_all(), get_total_assets/liabilities()
│   │   ├── ai_service.py        # build_context(), stream_chat() SSE gen, generate_daily_insight()
│   │   ├── budget_service.py    # get_budget_summary(), apply_rollovers()
│   │   ├── export_service.py    # transactions_to_csv/excel(), report_to_pdf(), build_report_html()
│   │   ├── fx_service.py        # get_today_rate(), force_refresh(), _fetch_yfinance(),
│   │   │                        # _fetch_er_api(), get_rate_history()
│   │   ├── goal_service.py      # get_projected_completion(), get_emergency_fund_status()
│   │   ├── import_service.py    # parse_csv(), import_rows(), duplicate detection
│   │   ├── investment_service.py # fetch_price(), update_prices(), get_portfolio_summary()
│   │   ├── ocr_service.py       # extract_from_file(), extract_from_bytes(),
│   │   │                        # Groq vision model, JSON parse + sanitise
│   │   ├── recurring_service.py # process_due_rules(), next_occurrence(), get_upcoming()
│   │   └── report_service.py    # monthly/quarterly/yearly/tax reports,
│   │                            # net_worth_history(), category_trends(), take_net_worth_snapshot()
│   │
│   ├── templates/
│   │   ├── base.html            # Collapsible sidebar, topbar, flash messages,
│   │   │                        # @keyframes spin, Bootstrap 5 + Bootstrap Icons
│   │   ├── auth/login.html      # Dark themed, password toggle
│   │   ├── dashboard/index.html # All widgets, FX refresh JS, SSE-compatible
│   │   ├── accounts/            # index.html, form.html (color/icon picker)
│   │   ├── categories/          # index.html, form.html
│   │   ├── transactions/        # index.html (tabs+filter+pagination)
│   │   │                        # form.html (OCR panel + drag-drop + field flash)
│   │   │                        # transfer.html
│   │   ├── budgets/             # index.html (progress bars), form.html
│   │   ├── goals/               # index.html (emergency fund + cards), form.html,
│   │   │                        # contribute.html, contributions.html
│   │   ├── investments/         # index.html (doughnut chart), detail.html,
│   │   │                        # form.html (ticker check), transaction_form.html
│   │   ├── reports/             # index.html (4 chart types), tax.html
│   │   ├── ai/                  # index.html (SSE chat + suggestions), history.html
│   │   └── settings/            # index.html, profile.html, password.html,
│   │                            # recurring.html, recurring_form.html, import.html
│   │
│   ├── static/
│   │   ├── css/                 # (empty — all CSS inline in templates)
│   │   ├── js/                  # (empty — all JS inline in templates)
│   │   └── img/
│   │
│   └── utils/
│       ├── formatters.py        # format_currency(), format_percent(), format_large_number()
│       └── decorators.py        # login_required_custom (unused — Flask-Login handles it)
│
├── migrations/                  # Flask-Migrate / Alembic
│
├── scripts/                     # All cron scripts — sys.path fix at top of each
│   ├── init_db.py               # Seeds 21 default categories, creates admin user interactively
│   ├── process_recurring.py     # Processes due recurring rules, creates transactions
│   ├── fetch_fx_rate.py         # force_refresh() — always fetches fresh USD/VND
│   ├── fetch_prices.py          # yfinance price update for all investment tickers
│   ├── daily_snapshot.py        # Saves net worth snapshot (1st of month)
│   └── daily_ai_insight.py      # Generates daily AI summary via Groq
│
└── tests/

5. Python Dependencies (requirements.txt)

flask==3.1.0
flask-sqlalchemy==3.1.1
flask-login==0.6.3
flask-migrate==4.1.0
flask-wtf==1.2.2
pymysql==1.1.1
python-dotenv==1.0.1
gunicorn==23.0.0
groq==0.13.1
yfinance==0.2.54
weasyprint==63.1
openpyxl==3.1.5
Pillow==11.1.0
apscheduler==3.10.4
requests==2.32.3
cryptography==44.0.2
python-dateutil==2.9.0

6. AI Integration — Groq API

Chat + Daily Insights

  • Model: llama-3.3-70b-versatile (default) / llama-3.1-8b-instant (fast)
  • Context: last 90 days transactions, budget status, goals, investments (anonymised)
  • SSE streaming: stream_chat() yields data: <chunk>\n\n
  • Daily insight: non-streaming, stored in ai_insights table, max 400 tokens

Receipt OCR (Vision)

  • Model: meta-llama/llama-4-scout-17b-16e-instruct
  • Input: base64-encoded image (JPEG/PNG/GIF/WEBP)
  • Prompt: structured JSON extraction (amount, date, merchant, category, notes)
  • Temperature: 0.1 (low, for consistent output)
  • Post-processing: strips markdown fences, extracts JSON via regex fallback, normalises amount/date, maps category to system names
  • Endpoints: POST /transactions/ocr (upload bytes), POST /transactions/ocr-file (stored file)

Free Tier Limits

Metric Limit
Requests/day 14,400
Tokens/minute 500,000
Cost Free

7. USD → VND Exchange Rate

Reference widget only. All transactions use the single configured app currency.

Fetch Priority

  1. DB cache — same day record in fx_rates
  2. yfinance USDVND=X forex ticker (primary — most reliable)
  3. open.er-api.com free REST API (fallback)
  4. Last known DB record (stale fallback, shows ⚠ indicator)

Sanity Check

Rate must be > 1000 — rejects garbage values (e.g. 1.0, 0.0) that some APIs return.

Dashboard Widget

  • Shows rate, date, source
  • ↻ button → POST /api/fx-refresh → updates rate in-place without page reload
  • Click widget → toggles 30-day Chart.js line chart
  • force_refresh() used by daily cron — always bypasses cache

8. UI/UX

  • Sidebar: collapsible (desktop state saved in localStorage), mobile overlay
  • Charts: Chart.js 4.x (CDN)
  • Forms: WTForms + Bootstrap 5.3
  • Icons: Bootstrap Icons 1.11
  • Fonts: DM Sans + DM Mono (Google Fonts CDN)
  • Color scheme: #0f172a sidebar, #f1f5f9 body, #10b981 income, #ef4444 expense, #3b82f6 invest
  • CSS: All inline in templates (no build step)
  • SSE: used for AI chat stream + FX refresh

9. Authentication

  • Single-user, Flask-Login, session-based
  • Hashed password (Werkzeug generate_password_hash)
  • SESSION_COOKIE_SECURE=True in production
  • SESSION_COOKIE_HTTPONLY=True, SESSION_COOKIE_SAMESITE='Lax'
  • CSRF protection on all forms (Flask-WTF)

10. Scheduled Jobs

Job Schedule Script Notes
Process recurring transactions Daily 6AM process_recurring.py Creates missed occurrences
Fetch USD/VND rate Daily 8AM fetch_fx_rate.py force_refresh(), yfinance primary
Fetch investment prices Mon-Fri 4PM fetch_prices.py yfinance, all tickers
Net worth snapshot 1st of month 00:05 daily_snapshot.py Saves to net_worth_snapshots
AI daily insight Daily 00:01 daily_ai_insight.py Skips if already done today
DB backup Daily 2AM pfm-backup (systemd) mysqldump → gzip, keep 30 days

11. Environment Variables (.env)

SECRET_KEY=your-secret-key
DATABASE_URL=mysql+pymysql://pfm_user:password@localhost/pfm_db
GROQ_API_KEY=your-groq-api-key-here
GROQ_MODEL=llama-3.3-70b-versatile
UPLOAD_FOLDER=/home/pfm/web/uploads
MAX_CONTENT_LENGTH=10485760
FLASK_ENV=production
FLASK_APP=wsgi:app
APP_CURRENCY=USD
APP_CURRENCY_SYMBOL=$
APP_TIMEZONE=Asia/Ho_Chi_Minh

12. Blueprints Registered (11 total)

Blueprint Prefix Key routes
auth /auth login, logout
dashboard / index, api/fx-history, api/fx-refresh
accounts /accounts CRUD
categories /categories CRUD
transactions /transactions index, new, edit, delete, transfer, ocr, ocr-file
budgets /budgets index, new, edit, delete, copy
goals /goals index, new, edit, delete, contribute, contributions
investments /investments index, new, detail, edit, delete, add_transaction, refresh-prices, api/price
reports /reports monthly, quarterly, yearly, tax, export/csv
ai /ai index, stream (SSE), history, generate-insight
settings /settings index, profile, password, recurring, import, upload_receipt, delete_receipt, view_receipt

13. Security Notes

  • All routes @login_required
  • CSRF on all POST forms
  • SQLAlchemy ORM (no raw SQL)
  • Receipt file path: os.path.basename() prevents path traversal
  • HTTPS via Let's Encrypt (Certbot) — pfm.ngodanguyen.tech
  • Groq receives anonymised transaction summaries (no account/personal names)
  • GROQ_API_KEY in .env (chmod 600), never in frontend

14. Known Issues / Notes

  • wsgi.py has sys.path.insert(0, ...) fix — required because app deploys at /home/pfm/web/ which would otherwise be treated as a Python package
  • FX rate widget: open.er-api.com may return stale values; yfinance is the reliable primary
  • WeasyPrint PDF: requires libpango* system libs (included in deploy.md apt install)
  • Import preview uses Flask session to pass rows to confirm step — requires SECRET_KEY to be set

15. Post-MVP Roadmap

  • iOS companion app
  • Bank statement PDF auto-import (parse PDF → extract transactions)
  • Budget alerts + Twilio SMS/email notifications
  • Shared household mode (2 users, row-level isolation)
  • Mobile responsiveness pass
Reference in New Issue View Git Blame Copy Permalink