nngo/Personal-Finance-Management
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

05/31 Update documents

Browse Source
This commit is contained in:
nngo
2026-05-31 17:28:50 -04:00
parent 5983794217
commit 8b1594da5a
3 changed files with 1249 additions and 626 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+325 -376
View File
@@ -1,236 +1,262 @@
# Personal Finance Management System (PFMS)
# Personal Finance Management System (PFM)
> Stack: Python Flask · MySQL · Ubuntu Server · Nginx · Gunicorn · Groq API (free AI)
> App URL: https://pfm.ngodanguyen.tech
> Code: /home/pfm/web · User: pfm · Gitea: gitea.ngodanguyen.tech
---
## 1. Project Overview
Self-hosted personal finance web app. Tracks income, expenses, investments. AI assistant powered by **Groq API** (free tier, extremely fast inference, no local hardware needed). Everything runs on your Ubuntu server behind Nginx.
Self-hosted personal finance web app. Tracks income, expenses, investments. AI assistant powered by **Groq API** (free tier, fast inference, no local hardware). Receipt OCR via Groq vision model. Everything runs on Ubuntu server behind Nginx + Certbot SSL.
**Status: All 7 phases complete + Receipt OCR post-MVP feature.**
---
## 2. Core Features
## 2. Core Features (Implemented)
### 2.1 Dashboard
- Net worth snapshot (assets − liabilities)
- Monthly cash flow chart (income vs expenses)
- Budget utilization gauges per category
- Recent transactions feed
- AI insight card (auto-generated daily summary)
- Investment portfolio mini-widget
- **USD → VND exchange rate widget** (daily rate, fetched once/day, cached in DB — reference only, independent of app currency)
- Monthly cash flow bar chart (6 months)
- Budget utilization per category
- Recent transactions feed (last 8)
- AI daily insight card (Groq-generated, stored in DB)
- USD → VND exchange rate widget — reference only, independent of app currency
- Click to expand 30-day history chart
- ↻ refresh button (force-fetches fresh rate without page reload)
- Source label shown (yfinance / exchangerate-api)
- Stale indicator if rate > 1 day old
- Period selector: This Month / Last Month / Custom date range
### 2.2 Income Management
- Log income entries (salary, freelance, passive, other)
- Recurring income templates (auto-create entries on schedule)
- Income source breakdown (chart by source)
- Month-over-month comparison
- Export to CSV/Excel
### 2.2 Transactions
- Income + Expense entry with Income/Expense tabs
- Transfer between accounts
- Filter: search, category, account, date range
- Pagination (30/page)
- Receipt upload (PNG/JPG/WEBP/GIF/PDF, max 10MB)
- **AI Receipt OCR** — drag-drop receipt image → Groq vision extracts amount/date/merchant/category → auto-fills form
- Re-extract from already-uploaded receipt (edit mode)
- Export to CSV / Excel
### 2.3 Expense Management
- Manual expense entry
- Category tagging (custom + predefined: Food, Rent, Utilities, Transport, Health, Entertainment, etc.)
- Subcategory support
- Receipt photo upload (stored locally)
- Recurring expense detection
- Budget limits per category with alert thresholds
- Expense search + filter (date range, category, amount range, keyword)
- Export to CSV/Excel
### 2.3 Accounts
- Types: checking, savings, cash, credit_card, crypto, investment, other
- Balance auto-calculated from all transactions (not manually entered)
- Color + icon picker
- Soft delete
### 2.4 Investment Portfolio
- Asset types: Stocks, ETF, Crypto, Real Estate, Bonds, Cash, Other
- Holdings tracker (ticker, shares/units, buy price, current price)
- Manual price update OR auto-fetch via free API (Yahoo Finance via `yfinance`)
- P&L per holding (unrealized gain/loss)
- Portfolio allocation pie chart
- Transaction log (buy/sell history per asset)
- Cost basis tracking (FIFO)
### 2.4 Categories
- Expense + Income categories with color/icon
- System categories (protected from delete)
- Custom categories (user-created)
- 14 expense + 7 income defaults seeded on init
### 2.5 Budget Planner
- Monthly budget templates
- Set budget limits per category
- Real-time spending vs budget comparison
- Rollover unused budget (optional toggle)
- Budget history archive
- Monthly limits per expense category
- Progress bars: green → amber (80%) → red (100%+)
- Rollover unused budget to next month (toggle)
- Copy previous month's budgets in one click
- Unbudgeted spending shown with "Set Budget" prompt
### 2.6 Goals & Savings
- Create savings goals (name, target amount, target date, linked account)
- Track contributions toward each goal
- Progress bar + projected completion date
- Emergency fund tracker (X months of expenses)
- Goals with target amount, target date, color, icon
- Contribution tracking + history
- Progress bar + projected completion date (based on avg monthly contrib)
- Auto-complete on 100%
- Emergency fund tracker (liquid assets vs 3-month / 6-month expense targets)
### 2.7 Reports & Analytics
- Monthly/quarterly/yearly summary reports
- Category spending trends (line chart over time)
- Income growth chart
- Net worth over time (historical snapshots, monthly auto-saved)
- Tax year summary (income + deductible expenses)
- Printable PDF report (via WeasyPrint)
### 2.7 Investments
- Asset types: stock, ETF, crypto, real_estate, bond, cash, other
- Buy/sell/dividend/split transaction log
- FIFO cost basis auto-recalculated from transaction log
- yfinance price auto-fetch (daily 4PM weekdays via cron)
- Manual price refresh button (portfolio page)
- Live ticker check on add form
- Doughnut allocation chart
- P&L per holding + portfolio total
### 2.8 Accounts & Wallets
- Multiple accounts (bank checking, savings, cash, credit card, crypto wallet)
- Account balances tracked manually
- Transfer between accounts (internal transaction)
- Credit card balance + due date tracking
### 2.8 AI Financial Assistant
- Chat UI with SSE streaming (Groq API, word-by-word response)
- Context: last 90 days transactions + budget status + goals + investments
- 8 suggested question buttons
- Daily auto-insight generated at midnight (stored in `ai_insights` table)
- Manual "Generate Now" button
- Chat history page
- Model: `llama-3.3-70b-versatile` (default) or `llama-3.1-8b-instant` (fast)
- Fallback messages for rate limit / invalid key / unavailable
### 2.9 AI Financial Assistant (Groq API — Free Tier)
- Chat interface (ask questions about your finances)
- Context: last 90 days of transactions injected into prompt
- Example queries:
- "Where did I overspend this month?"
- "Am I on track for my vacation goal?"
- "Summarize my Q1 spending"
- "What categories can I cut to save $500/month?"
- Auto-insight: daily AI summary generated at midnight via cron
- Model: `llama-3.3-70b-versatile` or `llama-3.1-8b-instant` via Groq (configurable in `.env`)
- Streaming response (SSE) for real-time chat feel
### 2.9 Receipt OCR (Groq Vision)
- Model: `meta-llama/llama-4-scout-17b-16e-instruct`
- Drag-drop or click-to-upload on new expense form
- Extracts: amount, date, merchant name, category suggestion, notes
- Maps category suggestion → system category ID
- Auto-fills form fields with green flash animation
- Re-extract button on existing receipt (edit mode)
- Auto-triggers OCR when image file selected in edit mode
- Handles fenced markdown JSON output from LLM
- Sanity check: rejects amounts < 1000 (catches garbage values)
- Full error handling: 400/401/429/timeout/bad JSON
### 2.10 Notifications & Alerts
- Browser notifications (via Web Push or in-app toast)
- Budget threshold alerts (e.g., 80% of category budget used)
- Bill/recurring expense due reminders
- Goal milestone celebrations
### 2.10 Reports & Export
- Monthly / Quarterly / Yearly summary reports
- Tax year summary (all income by source, all expenses by category)
- Net worth history line chart (from monthly snapshots)
- Category spending trends (top 6 categories, 6-month line chart)
- "Snapshot Now" manual button
- Export: CSV, Excel (color-coded, formatted), PDF (WeasyPrint)
### 2.11 Settings & Config
- Profile (name, timezone)
- **App currency** — single configurable currency (e.g. USD, VND, EUR — set once, used everywhere for all transactions/display)
- USD→VND rate source preference (ExchangeRate-API free or VCB scrape fallback)
- Category management (add/edit/delete custom categories)
- Data backup (export full MySQL dump)
- Data import (CSV import for bulk transactions)
- Groq model selector (choose speed vs quality)
### 2.11 Settings
- Profile: name, email, timezone, currency, Groq model
- 8 currency options (USD/VND/EUR/GBP/JPY/AUD/CAD/SGD) — auto-updates symbol
- Password change (requires current password)
- Recurring rules: CRUD, pause/enable, frequency (daily/weekly/biweekly/monthly/quarterly/yearly)
- "Run Now" button to process due rules immediately
- CSV import: upload → preview with ⚠ warnings → confirm
- Upcoming recurring transactions (30-day view)
### 2.12 USD → VND Exchange Rate
- **Reference widget only** — not used in transaction calculations
- Primary source: yfinance `USDVND=X` (Yahoo Finance forex)
- Fallback: `open.er-api.com` (free, no key)
- Sanity check: rate must be > 1000 (rejects garbage values)
- `force_refresh()` — always fetches fresh on cron, bypasses cache
- DB caches one record per day (`fx_rates` table)
- Dashboard: shows rate, date, source, stale warning, ↻ refresh button
- 30-day history chart (click widget to expand)
---
## 3. Database Schema (MySQL)
### Tables
### All 14 Tables
```
users — single user (self-hosted, no multi-tenant)
accounts — bank/wallet accounts
categories — expense/income categories
transactions — all money movements (income/expense/transfer)
investments — holdings/portfolio positions
investment_transactions — buy/sell log
budgets — monthly budget limits per category
goals — savings goals
goal_contributions — deposits toward each goal
net_worth_snapshots — monthly net worth history
ai_insights — stored daily AI summaries
recurring_rules — templates for recurring income/expenses
receipts — receipt image metadata
fx_rates — daily USD/VND rate cache (date, rate, source)
users — single user, hashed password, currency/timezone prefs
accounts — bank/wallet accounts (balance auto-calc from txns)
categories — expense/income categories with color/icon
transactions — income/expense/transfer, receipt_id, recurring_rule_id
receipts — receipt file metadata (filename, size, mime_type)
recurring_rules — templates: frequency, next_run, start/end date
budgets — monthly limits per category, rollover support
goals — savings goals with target amount/date
goal_contributions — individual deposits toward each goal
investments — holdings: ticker, shares, avg_cost_basis, current_price
investment_transactions — buy/sell/dividend/split log
net_worth_snapshots — monthly snapshots: assets, liabilities, net_worth (JSON)
ai_insights — stored AI responses: daily_summary / chat_response
fx_rates — daily USD/VND rate cache (date UNIQUE, source)
```
### Key Table: `transactions`
```sql
id, account_id, category_id, type (income/expense/transfer),
amount, currency, description, date, notes,
is_recurring, recurring_rule_id, receipt_id,
created_at, updated_at
```
### Key Table: `investments`
```sql
id, asset_name, ticker, asset_type (stock/etf/crypto/real_estate/bond/cash/other),
shares, avg_cost_basis, current_price, last_price_update,
currency, notes, created_at
```
### Key Table: `budgets`
```sql
id, category_id, month (YYYY-MM), limit_amount,
rollover_enabled, rollover_amount, created_at
```
### Key Column Notes
- `transactions.balance` — NOT stored; calculated on-demand via `account_service.calc_balance()`
- `investments.shares` / `avg_cost_basis` — recalculated from `investment_transactions` (FIFO)
- `goals.current_amount` — updated on each contribution add/delete
- `net_worth_snapshots.account_balances` — JSON snapshot of each account balance at time of snapshot
---
## 4. Project File Structure
## 4. Project File Structure (Actual)
```
pfm/
pfm/ # /home/pfm/web on server
├── wsgi.py # Gunicorn entry — sys.path fix included
├── requirements.txt
├── .env # Not committed
├── .env.example
├── .gitignore
│
├── app/
│ ├── __init__.py # Flask app factory
│ ├── config.py # Config classes (dev/prod)
│ ├── extensions.py # db, login_manager, etc.
│ ├── __init__.py # Flask app factory, all blueprints registered
│ ├── config.py # Dev/Prod configs, SESSION_COOKIE_SECURE in prod
│ ├── extensions.py # db, login_manager, migrate, csrf
│ │
│ ├── models/
│ │ ├── user.py
│ │ ├── account.py
│ │ ├── transaction.py
│ │ ├── category.py
│ │ ├── investment.py
│ │ ├── budget.py
│ │ ├── goal.py
│ │ ├── ai_insight.py
│ │ ├── fx_rate.py
│ │ └── recurring_rule.py
│ │ ├── __init__.py # Imports all models (required for Flask-Migrate)
│ │ ├── user.py # UserMixin, set/check password, load_user hook
│ │ ├── account.py # account_type enum, color, icon
│ │ ├── category.py # Self-referential (subcategories), is_system flag
│ │ ├── transaction.py # Dual FK to accounts (account_id + to_account_id)
│ │ ├── receipt.py # filename, original_filename, mime_type
│ │ ├── recurring_rule.py # frequency enum, next_run date
│ │ ├── budget.py # UniqueConstraint(category_id, month)
│ │ ├── goal.py # progress_percent property
│ │ ├── investment.py # total_cost/current_value/unrealized_gain properties
│ │ ├── net_worth_snapshot.py # account_balances JSON field
│ │ ├── ai_insight.py # insight_type enum
│ │ └── fx_rate.py # date UNIQUE index
│ │
│ ├── routes/
│ │ ├── auth.py # Login/logout (single user)
│ │ ├── dashboard.py
│ │ ├── transactions.py
│ │ ├── income.py
│ │ ├── expenses.py
│ │ ├── investments.py
│ │ ├── budgets.py
│ │ ├── goals.py
│ │ ├── accounts.py
│ │ ├── reports.py
│ │ ├── ai.py # AI chat + insights (SSE)
│ │ ├── settings.py
│ │ └── api.py # Internal JSON API endpoints
│ │ ├── auth.py # /auth/login, /auth/logout
│ │ ├── dashboard.py # /, /api/fx-history, /api/fx-refresh (POST)
│ │ ├── accounts.py # /accounts/
│ │ ├── categories.py # /categories/
│ │ ├── transactions.py # /transactions/, /transactions/ocr (POST),
│ │ │ # /transactions/ocr-file (POST)
│ │ ├── budgets.py # /budgets/, /budgets/copy (POST)
│ │ ├── goals.py # /goals/, contribute, contributions, delete_contribution
│ │ ├── investments.py # /investments/, detail, add_transaction,
│ │ │ # refresh-prices, /api/price/<ticker>
│ │ ├── reports.py # /reports/monthly|quarterly|yearly|tax
│ │ │ # /reports/export/csv|excel|pdf
│ │ ├── ai.py # /ai/, /ai/stream (SSE), /ai/history,
│ │ │ # /ai/generate-insight (POST)
│ │ └── settings.py # /settings/, profile, password, recurring,
│ │ # import, upload_receipt, delete_receipt, view_receipt
│ │
│ ├── services/
│ │ ├── ai_service.py # Groq API integration + context builder
│ │ ├── budget_service.py # Budget calc + alert logic
│ │ ├── investment_service.py # yfinance price fetcher
│ │ ├── fx_service.py # USD/VND rate fetch + cache logic
│ │ ├── report_service.py # PDF generation (WeasyPrint)
│ │ ├── recurring_service.py # Recurring rule processor
│ │ ├── import_service.py # CSV import parser
│ │ └── snapshot_service.py # Net worth snapshot scheduler
│ │ ├── account_service.py # calc_balance(), recalc_all(), get_total_assets/liabilities()
│ │ ├── ai_service.py # build_context(), stream_chat() SSE gen, generate_daily_insight()
│ │ ├── budget_service.py # get_budget_summary(), apply_rollovers()
│ │ ├── export_service.py # transactions_to_csv/excel(), report_to_pdf(), build_report_html()
│ │ ├── fx_service.py # get_today_rate(), force_refresh(), _fetch_yfinance(),
│ │ │ # _fetch_er_api(), get_rate_history()
│ │ ├── goal_service.py # get_projected_completion(), get_emergency_fund_status()
│ │ ├── import_service.py # parse_csv(), import_rows(), duplicate detection
│ │ ├── investment_service.py # fetch_price(), update_prices(), get_portfolio_summary()
│ │ ├── ocr_service.py # extract_from_file(), extract_from_bytes(),
│ │ │ # Groq vision model, JSON parse + sanitise
│ │ ├── recurring_service.py # process_due_rules(), next_occurrence(), get_upcoming()
│ │ └── report_service.py # monthly/quarterly/yearly/tax reports,
│ │ # net_worth_history(), category_trends(), take_net_worth_snapshot()
│ │
│ ├── templates/
│ │ ├── base.html
│ │ ├── auth/
│ │ ├── dashboard/
│ │ ├── transactions/
│ │ ├── investments/
│ │ ├── budgets/
│ │ ├── goals/
│ │ ├── accounts/
│ │ ├── reports/
│ │ ├── ai/
│ │ └── settings/
│ │ ├── base.html # Collapsible sidebar, topbar, flash messages,
│ │ │ # @keyframes spin, Bootstrap 5 + Bootstrap Icons
│ │ ├── auth/login.html # Dark themed, password toggle
│ │ ├── dashboard/index.html # All widgets, FX refresh JS, SSE-compatible
│ │ ├── accounts/ # index.html, form.html (color/icon picker)
│ │ ├── categories/ # index.html, form.html
│ │ ├── transactions/ # index.html (tabs+filter+pagination)
│ │ │ # form.html (OCR panel + drag-drop + field flash)
│ │ │ # transfer.html
│ │ ├── budgets/ # index.html (progress bars), form.html
│ │ ├── goals/ # index.html (emergency fund + cards), form.html,
│ │ │ # contribute.html, contributions.html
│ │ ├── investments/ # index.html (doughnut chart), detail.html,
│ │ │ # form.html (ticker check), transaction_form.html
│ │ ├── reports/ # index.html (4 chart types), tax.html
│ │ ├── ai/ # index.html (SSE chat + suggestions), history.html
│ │ └── settings/ # index.html, profile.html, password.html,
│ │ # recurring.html, recurring_form.html, import.html
│ │
│ ├── static/
│ │ ├── css/
│ │ ├── js/
│ │ └── img/ # Static images
│ │ ├── css/ # (empty — all CSS inline in templates)
│ │ ├── js/ # (empty — all JS inline in templates)
│ │ └── img/
│ │
│ └── utils/
│ ├── decorators.py # Auth required, etc.
│ ├── formatters.py # Currency, date formatting
│ └── validators.py
│ ├── formatters.py # format_currency(), format_percent(), format_large_number()
│ └── decorators.py # login_required_custom (unused — Flask-Login handles it)
│
├── migrations/ # Flask-Migrate (Alembic)
├── scripts/
│ ├── init_db.py # First-run DB setup + seed categories
│ ├── daily_snapshot.py # Cron: net worth snapshot
│ ├── daily_ai_insight.py # Cron: generate AI summary
│ ├── process_recurring.py # Cron: create recurring transactions
│ ├── fetch_prices.py # Cron: update investment prices
│ └── fetch_fx_rate.py # Cron: fetch daily USD/VND rate
├── migrations/ # Flask-Migrate / Alembic
│
├── tests/
├── .env # Secrets (not committed)
├── .env.example
├── requirements.txt
├── wsgi.py
├── CLAUDE.md # This file
└── deploy.md # Server setup guide
├── scripts/ # All cron scripts — sys.path fix at top of each
│ ├── init_db.py # Seeds 21 default categories, creates admin user interactively
│ ├── process_recurring.py # Processes due recurring rules, creates transactions
│ ├── fetch_fx_rate.py # force_refresh() — always fetches fresh USD/VND
│ ├── fetch_prices.py # yfinance price update for all investment tickers
│ ├── daily_snapshot.py # Saves net worth snapshot (1st of month)
│ └── daily_ai_insight.py # Generates daily AI summary via Groq
│
└── tests/
```
---
@@ -238,248 +264,171 @@ pfm/
## 5. Python Dependencies (`requirements.txt`)
```
flask
flask-sqlalchemy
flask-login
flask-migrate
flask-wtf
pymysql
python-dotenv
gunicorn
groq # Groq official Python SDK
yfinance # Investment price fetching
weasyprint # PDF report generation
openpyxl # Excel export
Pillow # Receipt image processing
apscheduler # In-process scheduler (alternative to cron)
flask==3.1.0
flask-sqlalchemy==3.1.1
flask-login==0.6.3
flask-migrate==4.1.0
flask-wtf==1.2.2
pymysql==1.1.1
python-dotenv==1.0.1
gunicorn==23.0.0
groq==0.13.1
yfinance==0.2.54
weasyprint==63.1
openpyxl==3.1.5
Pillow==11.1.0
apscheduler==3.10.4
requests==2.32.3
cryptography==44.0.2
python-dateutil==2.9.0
```
---
## 6. AI Integration — Groq API (Free Tier)
## 6. AI Integration — Groq API
### Why Groq
- Free tier: 14,400 requests/day, 500,000 tokens/minute
- Fastest inference available (LPU hardware) — responses feel instant
- No local GPU/RAM needed
- Models: `llama-3.3-70b-versatile` (best quality), `llama-3.1-8b-instant` (fastest)
### Chat + Daily Insights
- Model: `llama-3.3-70b-versatile` (default) / `llama-3.1-8b-instant` (fast)
- Context: last 90 days transactions, budget status, goals, investments (anonymised)
- SSE streaming: `stream_chat()` yields `data: <chunk>\n\n`
- Daily insight: non-streaming, stored in `ai_insights` table, max 400 tokens
### How It Works
```
User sends chat message
→ ai_service.py builds context (last 90 days transactions summary)
→ POST to Groq API (`/chat/completions`) with Bearer token
→ Stream response back via SSE to browser
→ Response stored in ai_insights table
```
### Receipt OCR (Vision)
- Model: `meta-llama/llama-4-scout-17b-16e-instruct`
- Input: base64-encoded image (JPEG/PNG/GIF/WEBP)
- Prompt: structured JSON extraction (amount, date, merchant, category, notes)
- Temperature: 0.1 (low, for consistent output)
- Post-processing: strips markdown fences, extracts JSON via regex fallback,
normalises amount/date, maps category to system names
- Endpoints: `POST /transactions/ocr` (upload bytes), `POST /transactions/ocr-file` (stored file)
### Context Injection Strategy
```python
# ai_service.py builds a prompt like:
"""
You are a personal finance assistant. Here is the user's financial data:
CURRENT MONTH SUMMARY:
- Total Income: $X
- Total Expenses: $Y
- Top spending categories: Food ($A), Transport ($B), ...
- Budget alerts: Entertainment 92% used
RECENT TRANSACTIONS (last 20):
[date] [category] [amount] [description]
...
NET WORTH: $Z
ACTIVE GOALS: Vacation Fund ($1,200 / $3,000)
User question: {user_message}
Answer concisely and specifically based on the data above.
"""
```
### Fallback
If Groq API key missing/invalid → show friendly message "AI assistant unavailable. Check GROQ_API_KEY in settings."
If Groq rate limit hit → show "AI rate limit reached. Try again shortly."
### Free Tier Limits
| Metric | Limit |
|--------|-------|
| Requests/day | 14,400 |
| Tokens/minute | 500,000 |
| Cost | Free |
---
## 7. USD → VND Exchange Rate
> **Reference widget only** — independent of the app's transaction currency. All income/expenses/investments use the single configured app currency. This widget is informational display only.
> **Reference widget only.** All transactions use the single configured app currency.
### Data Source — Free, No API Key Required
### Fetch Priority
1. DB cache — same day record in `fx_rates`
2. **yfinance** `USDVND=X` forex ticker (primary — most reliable)
3. `open.er-api.com` free REST API (fallback)
4. Last known DB record (stale fallback, shows ⚠ indicator)
Primary: **ExchangeRate-API open endpoint**
```
https://open.er-api.com/v6/latest/USD
```
Returns JSON with all rates including VND. Free tier, no key, 1,500 req/month.
### Sanity Check
Rate must be `> 1000` — rejects garbage values (e.g. 1.0, 0.0) that some APIs return.
Fallback: **Vietcombank (VCB) rate scrape**
```
https://www.vietcombank.com.vn/en/KHCN/Chuyen-trang-KHCN/Pages/ty-gia.aspx
```
Scrape VCB's official buying/selling rate as backup.
### DB Table: `fx_rates`
```sql
id INT AUTO_INCREMENT PRIMARY KEY
date DATE NOT NULL UNIQUE -- one record per day
usd_to_vnd DECIMAL(12,2) NOT NULL -- e.g. 25,450.00
source VARCHAR(50) -- 'exchangerate-api' | 'vcb' | 'manual'
fetched_at DATETIME
```
### `fx_service.py` Logic
```
get_today_rate():
1. Check fx_rates table for today's date
2. If found → return cached rate (no API call)
3. If not found → fetch from ExchangeRate-API
4. If API fails → try VCB scrape
5. If both fail → return last known rate from DB + show "rate may be outdated" flag
6. Save new rate to DB
```
### Dashboard Widget Display
- Card on dashboard header area (top bar or sidebar)
- Shows: `1 USD = 25,450 ₫` with date label
- Color: neutral/info (blue or grey — not green/red, it's informational)
- Click → opens 30-day rate history mini-chart (Chart.js, line chart)
- Stale indicator: if rate is >1 day old, show small warning icon
### 30-Day Rate History
- Stored in `fx_rates` table (one row/day, auto-accumulates)
- Chart available on dashboard click or Reports page
- Shows trend: flat/up/down with % change label
### Scheduled Job
- Daily 8AM fetch (after markets open in Vietnam)
- `scripts/fetch_fx_rate.py`
- systemd timer unit: `pfm-fxrate.timer`
### Dashboard Widget
- Shows rate, date, source
- ↻ button → `POST /api/fx-refresh` → updates rate in-place without page reload
- Click widget → toggles 30-day Chart.js line chart
- `force_refresh()` used by daily cron — always bypasses cache
---
## 8. UI/UX Design Direction
## 8. UI/UX
- **Style**: Clean financial dashboard — dark sidebar, white/light content area
- **Charts**: Chart.js (CDN, no build step needed)
- **Tables**: DataTables.js for sortable/searchable transaction tables
- **Forms**: WTForms + Bootstrap 5
- **Icons**: Bootstrap Icons or Feather Icons
- **Color scheme**: Deep navy sidebar, white cards, green (income), red (expense), blue (investment)
- **Mobile**: Responsive (Bootstrap grid)
- **AI Chat**: Floating chat panel (slide-in from right), SSE streaming text
- **Sidebar**: collapsible (desktop state saved in localStorage), mobile overlay
- **Charts**: Chart.js 4.x (CDN)
- **Forms**: WTForms + Bootstrap 5.3
- **Icons**: Bootstrap Icons 1.11
- **Fonts**: DM Sans + DM Mono (Google Fonts CDN)
- **Color scheme**: `#0f172a` sidebar, `#f1f5f9` body, `#10b981` income, `#ef4444` expense, `#3b82f6` invest
- **CSS**: All inline in templates (no build step)
- **SSE**: used for AI chat stream + FX refresh
---
## 9. Authentication
- Single-user app (self-hosted)
- Flask-Login with username/password
- Session-based auth
- Password hashed with Werkzeug (bcrypt)
- Optional: IP whitelist via Nginx (allow only LAN access)
- Single-user, Flask-Login, session-based
- Hashed password (Werkzeug `generate_password_hash`)
- `SESSION_COOKIE_SECURE=True` in production
- `SESSION_COOKIE_HTTPONLY=True`, `SESSION_COOKIE_SAMESITE='Lax'`
- CSRF protection on all forms (Flask-WTF)
---
## 10. Scheduled Jobs (systemd timers or APScheduler)
## 10. Scheduled Jobs
| Job | Schedule | Script |
|-----|----------|--------|
| Process recurring transactions | Daily 6AM | `process_recurring.py` |
| Fetch USD/VND exchange rate | Daily 8AM | `fetch_fx_rate.py` |
| Fetch investment prices | Daily 4PM | `fetch_prices.py` |
| Save net worth snapshot | 1st of month | `daily_snapshot.py` |
| Generate AI daily insight | Daily midnight | `daily_ai_insight.py` |
Recommend **APScheduler** inside Flask app (simpler) OR separate systemd timer units (more robust).
| Job | Schedule | Script | Notes |
|-----|----------|--------|-------|
| Process recurring transactions | Daily 6AM | `process_recurring.py` | Creates missed occurrences |
| Fetch USD/VND rate | Daily 8AM | `fetch_fx_rate.py` | force_refresh(), yfinance primary |
| Fetch investment prices | Mon-Fri 4PM | `fetch_prices.py` | yfinance, all tickers |
| Net worth snapshot | 1st of month 00:05 | `daily_snapshot.py` | Saves to net_worth_snapshots |
| AI daily insight | Daily 00:01 | `daily_ai_insight.py` | Skips if already done today |
| DB backup | Daily 2AM | pfm-backup (systemd) | mysqldump → gzip, keep 30 days |
---
## 11. Development Phases
### Phase 1 — Foundation
- [ ] Flask app factory + config
- [ ] MySQL models + migrations
- [ ] Auth (login/logout)
- [ ] Base template + sidebar nav
### Phase 2 — Core Transactions
- [ ] Accounts CRUD
- [ ] Categories CRUD
- [ ] Transaction entry (income + expense)
- [ ] Transaction list with filter/search
- [ ] Dashboard basics (totals, recent feed)
- [ ] USD→VND rate widget + `fx_service.py` + daily fetch job
### Phase 3 — Budget & Goals
- [ ] Budget planner (set limits)
- [ ] Budget vs actual comparison
- [ ] Goals CRUD + contribution tracking
### Phase 4 — Investments
- [ ] Holdings CRUD
- [ ] Buy/sell transaction log
- [ ] yfinance price auto-fetch
- [ ] Portfolio charts
### Phase 5 — AI Assistant
- [ ] Groq API integration + context builder
- [ ] Chat UI with SSE streaming
- [ ] Daily auto-insight cron
### Phase 6 — Reports & Export
- [ ] Monthly summary page
- [ ] PDF export (WeasyPrint)
- [ ] CSV/Excel export
- [ ] Net worth history chart
### Phase 7 — Polish
- [ ] Recurring transaction engine
- [ ] Receipt upload
- [ ] CSV import
- [ ] Budget alerts + notifications
- [ ] Mobile responsiveness pass
---
## 12. Environment Variables (`.env`)
## 11. Environment Variables (`.env`)
```
SECRET_KEY=your-secret-key
DATABASE_URL=mysql+pymysql://pfm_user:password@localhost/pfm_db
GROQ_API_KEY=your-groq-api-key-here
GROQ_MODEL=llama-3.3-70b-versatile
UPLOAD_FOLDER=/home/pfm/app/uploads
UPLOAD_FOLDER=/home/pfm/web/uploads
MAX_CONTENT_LENGTH=10485760
FLASK_ENV=production
APP_CURRENCY=USD # Single currency for all transactions (configurable)
APP_CURRENCY_SYMBOL=$ # Display symbol
APP_TIMEZONE=Asia/Ho_Chi_Minh # Server timezone for scheduled jobs
FLASK_APP=wsgi:app
APP_CURRENCY=USD
APP_CURRENCY_SYMBOL=$
APP_TIMEZONE=Asia/Ho_Chi_Minh
```
---
## 12. Blueprints Registered (11 total)
| Blueprint | Prefix | Key routes |
|-----------|--------|------------|
| auth | /auth | login, logout |
| dashboard | / | index, api/fx-history, api/fx-refresh |
| accounts | /accounts | CRUD |
| categories | /categories | CRUD |
| transactions | /transactions | index, new, edit, delete, transfer, ocr, ocr-file |
| budgets | /budgets | index, new, edit, delete, copy |
| goals | /goals | index, new, edit, delete, contribute, contributions |
| investments | /investments | index, new, detail, edit, delete, add_transaction, refresh-prices, api/price |
| reports | /reports | monthly, quarterly, yearly, tax, export/csv|excel|pdf, snapshot |
| ai | /ai | index, stream (SSE), history, generate-insight |
| settings | /settings | index, profile, password, recurring, import, upload_receipt, delete_receipt, view_receipt |
---
## 13. Security Notes
- All routes protected by `@login_required`
- CSRF protection via Flask-WTF
- SQL injection prevented by SQLAlchemy ORM
- File upload validation (type + size limit)
- Nginx: restrict access to local network if desired
- All routes `@login_required`
- CSRF on all POST forms
- SQLAlchemy ORM (no raw SQL)
- Receipt file path: `os.path.basename()` prevents path traversal
- HTTPS via Let's Encrypt (Certbot) — `pfm.ngodanguyen.tech`
- Groq API receives only anonymized transaction summaries (no account names/personal details in prompt)
- GROQ_API_KEY stored in `.env`, never exposed to frontend
- Groq receives anonymised transaction summaries (no account/personal names)
- `GROQ_API_KEY` in `.env` (chmod 600), never in frontend
---
## 14. Future Enhancements (Post-MVP)
## 14. Known Issues / Notes
- Mobile app companion (iOS — fits your skill set)
- Bank statement auto-import (parse PDF bank statements)
- Multi-currency with live FX rates (via free API)
- Expense photo OCR (extract amount from receipt image via Groq vision model)
- Email/SMS alerts (integrate Twilio — you already know it from FaxDesk)
- Shared household mode (2 users)
- `wsgi.py` has `sys.path.insert(0, ...)` fix — required because app deploys at `/home/pfm/web/` which would otherwise be treated as a Python package
- FX rate widget: `open.er-api.com` may return stale values; yfinance is the reliable primary
- WeasyPrint PDF: requires `libpango*` system libs (included in deploy.md apt install)
- Import preview uses Flask session to pass rows to confirm step — requires `SECRET_KEY` to be set
---
## 15. Post-MVP Roadmap
- [ ] iOS companion app
- [ ] Bank statement PDF auto-import (parse PDF → extract transactions)
- [ ] Budget alerts + Twilio SMS/email notifications
- [ ] Shared household mode (2 users, row-level isolation)
- [ ] Mobile responsiveness pass