486 lines
16 KiB
Markdown
486 lines
16 KiB
Markdown
# Personal Finance Management System (PFMS)
|
||
> Stack: Python Flask · MySQL · Ubuntu Server · Nginx · Gunicorn · Groq API (free AI)
|
||
|
||
---
|
||
|
||
## 1. Project Overview
|
||
|
||
Self-hosted personal finance web app. Tracks income, expenses, investments. AI assistant powered by **Groq API** (free tier, extremely fast inference, no local hardware needed). Everything runs on your Ubuntu server behind Nginx.
|
||
|
||
---
|
||
|
||
## 2. Core Features
|
||
|
||
### 2.1 Dashboard
|
||
- Net worth snapshot (assets − liabilities)
|
||
- Monthly cash flow chart (income vs expenses)
|
||
- Budget utilization gauges per category
|
||
- Recent transactions feed
|
||
- AI insight card (auto-generated daily summary)
|
||
- Investment portfolio mini-widget
|
||
- **USD → VND exchange rate widget** (daily rate, fetched once/day, cached in DB — reference only, independent of app currency)
|
||
|
||
### 2.2 Income Management
|
||
- Log income entries (salary, freelance, passive, other)
|
||
- Recurring income templates (auto-create entries on schedule)
|
||
- Income source breakdown (chart by source)
|
||
- Month-over-month comparison
|
||
- Export to CSV/Excel
|
||
|
||
### 2.3 Expense Management
|
||
- Manual expense entry
|
||
- Category tagging (custom + predefined: Food, Rent, Utilities, Transport, Health, Entertainment, etc.)
|
||
- Subcategory support
|
||
- Receipt photo upload (stored locally)
|
||
- Recurring expense detection
|
||
- Budget limits per category with alert thresholds
|
||
- Expense search + filter (date range, category, amount range, keyword)
|
||
- Export to CSV/Excel
|
||
|
||
### 2.4 Investment Portfolio
|
||
- Asset types: Stocks, ETF, Crypto, Real Estate, Bonds, Cash, Other
|
||
- Holdings tracker (ticker, shares/units, buy price, current price)
|
||
- Manual price update OR auto-fetch via free API (Yahoo Finance via `yfinance`)
|
||
- P&L per holding (unrealized gain/loss)
|
||
- Portfolio allocation pie chart
|
||
- Transaction log (buy/sell history per asset)
|
||
- Cost basis tracking (FIFO)
|
||
|
||
### 2.5 Budget Planner
|
||
- Monthly budget templates
|
||
- Set budget limits per category
|
||
- Real-time spending vs budget comparison
|
||
- Rollover unused budget (optional toggle)
|
||
- Budget history archive
|
||
|
||
### 2.6 Goals & Savings
|
||
- Create savings goals (name, target amount, target date, linked account)
|
||
- Track contributions toward each goal
|
||
- Progress bar + projected completion date
|
||
- Emergency fund tracker (X months of expenses)
|
||
|
||
### 2.7 Reports & Analytics
|
||
- Monthly/quarterly/yearly summary reports
|
||
- Category spending trends (line chart over time)
|
||
- Income growth chart
|
||
- Net worth over time (historical snapshots, monthly auto-saved)
|
||
- Tax year summary (income + deductible expenses)
|
||
- Printable PDF report (via WeasyPrint)
|
||
|
||
### 2.8 Accounts & Wallets
|
||
- Multiple accounts (bank checking, savings, cash, credit card, crypto wallet)
|
||
- Account balances tracked manually
|
||
- Transfer between accounts (internal transaction)
|
||
- Credit card balance + due date tracking
|
||
|
||
### 2.9 AI Financial Assistant (Groq API — Free Tier)
|
||
- Chat interface (ask questions about your finances)
|
||
- Context: last 90 days of transactions injected into prompt
|
||
- Example queries:
|
||
- "Where did I overspend this month?"
|
||
- "Am I on track for my vacation goal?"
|
||
- "Summarize my Q1 spending"
|
||
- "What categories can I cut to save $500/month?"
|
||
- Auto-insight: daily AI summary generated at midnight via cron
|
||
- Model: `llama-3.3-70b-versatile` or `llama-3.1-8b-instant` via Groq (configurable in `.env`)
|
||
- Streaming response (SSE) for real-time chat feel
|
||
|
||
### 2.10 Notifications & Alerts
|
||
- Browser notifications (via Web Push or in-app toast)
|
||
- Budget threshold alerts (e.g., 80% of category budget used)
|
||
- Bill/recurring expense due reminders
|
||
- Goal milestone celebrations
|
||
|
||
### 2.11 Settings & Config
|
||
- Profile (name, timezone)
|
||
- **App currency** — single configurable currency (e.g. USD, VND, EUR — set once, used everywhere for all transactions/display)
|
||
- USD→VND rate source preference (ExchangeRate-API free or VCB scrape fallback)
|
||
- Category management (add/edit/delete custom categories)
|
||
- Data backup (export full MySQL dump)
|
||
- Data import (CSV import for bulk transactions)
|
||
- Groq model selector (choose speed vs quality)
|
||
|
||
---
|
||
|
||
## 3. Database Schema (MySQL)
|
||
|
||
### Tables
|
||
|
||
```
|
||
users — single user (self-hosted, no multi-tenant)
|
||
accounts — bank/wallet accounts
|
||
categories — expense/income categories
|
||
transactions — all money movements (income/expense/transfer)
|
||
investments — holdings/portfolio positions
|
||
investment_transactions — buy/sell log
|
||
budgets — monthly budget limits per category
|
||
goals — savings goals
|
||
goal_contributions — deposits toward each goal
|
||
net_worth_snapshots — monthly net worth history
|
||
ai_insights — stored daily AI summaries
|
||
recurring_rules — templates for recurring income/expenses
|
||
receipts — receipt image metadata
|
||
fx_rates — daily USD/VND rate cache (date, rate, source)
|
||
```
|
||
|
||
### Key Table: `transactions`
|
||
```sql
|
||
id, account_id, category_id, type (income/expense/transfer),
|
||
amount, currency, description, date, notes,
|
||
is_recurring, recurring_rule_id, receipt_id,
|
||
created_at, updated_at
|
||
```
|
||
|
||
### Key Table: `investments`
|
||
```sql
|
||
id, asset_name, ticker, asset_type (stock/etf/crypto/real_estate/bond/cash/other),
|
||
shares, avg_cost_basis, current_price, last_price_update,
|
||
currency, notes, created_at
|
||
```
|
||
|
||
### Key Table: `budgets`
|
||
```sql
|
||
id, category_id, month (YYYY-MM), limit_amount,
|
||
rollover_enabled, rollover_amount, created_at
|
||
```
|
||
|
||
---
|
||
|
||
## 4. Project File Structure
|
||
|
||
```
|
||
pfm/
|
||
├── app/
|
||
│ ├── __init__.py # Flask app factory
|
||
│ ├── config.py # Config classes (dev/prod)
|
||
│ ├── extensions.py # db, login_manager, etc.
|
||
│ │
|
||
│ ├── models/
|
||
│ │ ├── user.py
|
||
│ │ ├── account.py
|
||
│ │ ├── transaction.py
|
||
│ │ ├── category.py
|
||
│ │ ├── investment.py
|
||
│ │ ├── budget.py
|
||
│ │ ├── goal.py
|
||
│ │ ├── ai_insight.py
|
||
│ │ ├── fx_rate.py
|
||
│ │ └── recurring_rule.py
|
||
│ │
|
||
│ ├── routes/
|
||
│ │ ├── auth.py # Login/logout (single user)
|
||
│ │ ├── dashboard.py
|
||
│ │ ├── transactions.py
|
||
│ │ ├── income.py
|
||
│ │ ├── expenses.py
|
||
│ │ ├── investments.py
|
||
│ │ ├── budgets.py
|
||
│ │ ├── goals.py
|
||
│ │ ├── accounts.py
|
||
│ │ ├── reports.py
|
||
│ │ ├── ai.py # AI chat + insights (SSE)
|
||
│ │ ├── settings.py
|
||
│ │ └── api.py # Internal JSON API endpoints
|
||
│ │
|
||
│ ├── services/
|
||
│ │ ├── ai_service.py # Groq API integration + context builder
|
||
│ │ ├── budget_service.py # Budget calc + alert logic
|
||
│ │ ├── investment_service.py # yfinance price fetcher
|
||
│ │ ├── fx_service.py # USD/VND rate fetch + cache logic
|
||
│ │ ├── report_service.py # PDF generation (WeasyPrint)
|
||
│ │ ├── recurring_service.py # Recurring rule processor
|
||
│ │ ├── import_service.py # CSV import parser
|
||
│ │ └── snapshot_service.py # Net worth snapshot scheduler
|
||
│ │
|
||
│ ├── templates/
|
||
│ │ ├── base.html
|
||
│ │ ├── auth/
|
||
│ │ ├── dashboard/
|
||
│ │ ├── transactions/
|
||
│ │ ├── investments/
|
||
│ │ ├── budgets/
|
||
│ │ ├── goals/
|
||
│ │ ├── accounts/
|
||
│ │ ├── reports/
|
||
│ │ ├── ai/
|
||
│ │ └── settings/
|
||
│ │
|
||
│ ├── static/
|
||
│ │ ├── css/
|
||
│ │ ├── js/
|
||
│ │ └── img/ # Static images
|
||
│ │
|
||
│ └── utils/
|
||
│ ├── decorators.py # Auth required, etc.
|
||
│ ├── formatters.py # Currency, date formatting
|
||
│ └── validators.py
|
||
│
|
||
├── migrations/ # Flask-Migrate (Alembic)
|
||
├── scripts/
|
||
│ ├── init_db.py # First-run DB setup + seed categories
|
||
│ ├── daily_snapshot.py # Cron: net worth snapshot
|
||
│ ├── daily_ai_insight.py # Cron: generate AI summary
|
||
│ ├── process_recurring.py # Cron: create recurring transactions
|
||
│ ├── fetch_prices.py # Cron: update investment prices
|
||
│ └── fetch_fx_rate.py # Cron: fetch daily USD/VND rate
|
||
│
|
||
├── tests/
|
||
├── .env # Secrets (not committed)
|
||
├── .env.example
|
||
├── requirements.txt
|
||
├── wsgi.py
|
||
├── CLAUDE.md # This file
|
||
└── deploy.md # Server setup guide
|
||
```
|
||
|
||
---
|
||
|
||
## 5. Python Dependencies (`requirements.txt`)
|
||
|
||
```
|
||
flask
|
||
flask-sqlalchemy
|
||
flask-login
|
||
flask-migrate
|
||
flask-wtf
|
||
pymysql
|
||
python-dotenv
|
||
gunicorn
|
||
groq # Groq official Python SDK
|
||
yfinance # Investment price fetching
|
||
weasyprint # PDF report generation
|
||
openpyxl # Excel export
|
||
Pillow # Receipt image processing
|
||
apscheduler # In-process scheduler (alternative to cron)
|
||
```
|
||
|
||
---
|
||
|
||
## 6. AI Integration — Groq API (Free Tier)
|
||
|
||
### Why Groq
|
||
- Free tier: 14,400 requests/day, 500,000 tokens/minute
|
||
- Fastest inference available (LPU hardware) — responses feel instant
|
||
- No local GPU/RAM needed
|
||
- Models: `llama-3.3-70b-versatile` (best quality), `llama-3.1-8b-instant` (fastest)
|
||
|
||
### How It Works
|
||
```
|
||
User sends chat message
|
||
→ ai_service.py builds context (last 90 days transactions summary)
|
||
→ POST to Groq API (`/chat/completions`) with Bearer token
|
||
→ Stream response back via SSE to browser
|
||
→ Response stored in ai_insights table
|
||
```
|
||
|
||
### Context Injection Strategy
|
||
```python
|
||
# ai_service.py builds a prompt like:
|
||
"""
|
||
You are a personal finance assistant. Here is the user's financial data:
|
||
|
||
CURRENT MONTH SUMMARY:
|
||
- Total Income: $X
|
||
- Total Expenses: $Y
|
||
- Top spending categories: Food ($A), Transport ($B), ...
|
||
- Budget alerts: Entertainment 92% used
|
||
|
||
RECENT TRANSACTIONS (last 20):
|
||
[date] [category] [amount] [description]
|
||
...
|
||
|
||
NET WORTH: $Z
|
||
ACTIVE GOALS: Vacation Fund ($1,200 / $3,000)
|
||
|
||
User question: {user_message}
|
||
|
||
Answer concisely and specifically based on the data above.
|
||
"""
|
||
```
|
||
|
||
### Fallback
|
||
If Groq API key missing/invalid → show friendly message "AI assistant unavailable. Check GROQ_API_KEY in settings."
|
||
If Groq rate limit hit → show "AI rate limit reached. Try again shortly."
|
||
|
||
---
|
||
|
||
## 7. USD → VND Exchange Rate
|
||
|
||
> **Reference widget only** — independent of the app's transaction currency. All income/expenses/investments use the single configured app currency. This widget is informational display only.
|
||
|
||
### Data Source — Free, No API Key Required
|
||
|
||
Primary: **ExchangeRate-API open endpoint**
|
||
```
|
||
https://open.er-api.com/v6/latest/USD
|
||
```
|
||
Returns JSON with all rates including VND. Free tier, no key, 1,500 req/month.
|
||
|
||
Fallback: **Vietcombank (VCB) rate scrape**
|
||
```
|
||
https://www.vietcombank.com.vn/en/KHCN/Chuyen-trang-KHCN/Pages/ty-gia.aspx
|
||
```
|
||
Scrape VCB's official buying/selling rate as backup.
|
||
|
||
### DB Table: `fx_rates`
|
||
```sql
|
||
id INT AUTO_INCREMENT PRIMARY KEY
|
||
date DATE NOT NULL UNIQUE -- one record per day
|
||
usd_to_vnd DECIMAL(12,2) NOT NULL -- e.g. 25,450.00
|
||
source VARCHAR(50) -- 'exchangerate-api' | 'vcb' | 'manual'
|
||
fetched_at DATETIME
|
||
```
|
||
|
||
### `fx_service.py` Logic
|
||
```
|
||
get_today_rate():
|
||
1. Check fx_rates table for today's date
|
||
2. If found → return cached rate (no API call)
|
||
3. If not found → fetch from ExchangeRate-API
|
||
4. If API fails → try VCB scrape
|
||
5. If both fail → return last known rate from DB + show "rate may be outdated" flag
|
||
6. Save new rate to DB
|
||
```
|
||
|
||
### Dashboard Widget Display
|
||
- Card on dashboard header area (top bar or sidebar)
|
||
- Shows: `1 USD = 25,450 ₫` with date label
|
||
- Color: neutral/info (blue or grey — not green/red, it's informational)
|
||
- Click → opens 30-day rate history mini-chart (Chart.js, line chart)
|
||
- Stale indicator: if rate is >1 day old, show small warning icon
|
||
|
||
### 30-Day Rate History
|
||
- Stored in `fx_rates` table (one row/day, auto-accumulates)
|
||
- Chart available on dashboard click or Reports page
|
||
- Shows trend: flat/up/down with % change label
|
||
|
||
### Scheduled Job
|
||
- Daily 8AM fetch (after markets open in Vietnam)
|
||
- `scripts/fetch_fx_rate.py`
|
||
- systemd timer unit: `pfm-fxrate.timer`
|
||
|
||
---
|
||
|
||
## 8. UI/UX Design Direction
|
||
|
||
- **Style**: Clean financial dashboard — dark sidebar, white/light content area
|
||
- **Charts**: Chart.js (CDN, no build step needed)
|
||
- **Tables**: DataTables.js for sortable/searchable transaction tables
|
||
- **Forms**: WTForms + Bootstrap 5
|
||
- **Icons**: Bootstrap Icons or Feather Icons
|
||
- **Color scheme**: Deep navy sidebar, white cards, green (income), red (expense), blue (investment)
|
||
- **Mobile**: Responsive (Bootstrap grid)
|
||
- **AI Chat**: Floating chat panel (slide-in from right), SSE streaming text
|
||
|
||
---
|
||
|
||
## 9. Authentication
|
||
|
||
- Single-user app (self-hosted)
|
||
- Flask-Login with username/password
|
||
- Session-based auth
|
||
- Password hashed with Werkzeug (bcrypt)
|
||
- Optional: IP whitelist via Nginx (allow only LAN access)
|
||
|
||
---
|
||
|
||
## 10. Scheduled Jobs (systemd timers or APScheduler)
|
||
|
||
| Job | Schedule | Script |
|
||
|-----|----------|--------|
|
||
| Process recurring transactions | Daily 6AM | `process_recurring.py` |
|
||
| Fetch USD/VND exchange rate | Daily 8AM | `fetch_fx_rate.py` |
|
||
| Fetch investment prices | Daily 4PM | `fetch_prices.py` |
|
||
| Save net worth snapshot | 1st of month | `daily_snapshot.py` |
|
||
| Generate AI daily insight | Daily midnight | `daily_ai_insight.py` |
|
||
|
||
Recommend **APScheduler** inside Flask app (simpler) OR separate systemd timer units (more robust).
|
||
|
||
---
|
||
|
||
## 11. Development Phases
|
||
|
||
### Phase 1 — Foundation
|
||
- [ ] Flask app factory + config
|
||
- [ ] MySQL models + migrations
|
||
- [ ] Auth (login/logout)
|
||
- [ ] Base template + sidebar nav
|
||
|
||
### Phase 2 — Core Transactions
|
||
- [ ] Accounts CRUD
|
||
- [ ] Categories CRUD
|
||
- [ ] Transaction entry (income + expense)
|
||
- [ ] Transaction list with filter/search
|
||
- [ ] Dashboard basics (totals, recent feed)
|
||
- [ ] USD→VND rate widget + `fx_service.py` + daily fetch job
|
||
|
||
### Phase 3 — Budget & Goals
|
||
- [ ] Budget planner (set limits)
|
||
- [ ] Budget vs actual comparison
|
||
- [ ] Goals CRUD + contribution tracking
|
||
|
||
### Phase 4 — Investments
|
||
- [ ] Holdings CRUD
|
||
- [ ] Buy/sell transaction log
|
||
- [ ] yfinance price auto-fetch
|
||
- [ ] Portfolio charts
|
||
|
||
### Phase 5 — AI Assistant
|
||
- [ ] Groq API integration + context builder
|
||
- [ ] Chat UI with SSE streaming
|
||
- [ ] Daily auto-insight cron
|
||
|
||
### Phase 6 — Reports & Export
|
||
- [ ] Monthly summary page
|
||
- [ ] PDF export (WeasyPrint)
|
||
- [ ] CSV/Excel export
|
||
- [ ] Net worth history chart
|
||
|
||
### Phase 7 — Polish
|
||
- [ ] Recurring transaction engine
|
||
- [ ] Receipt upload
|
||
- [ ] CSV import
|
||
- [ ] Budget alerts + notifications
|
||
- [ ] Mobile responsiveness pass
|
||
|
||
---
|
||
|
||
## 12. Environment Variables (`.env`)
|
||
|
||
```
|
||
SECRET_KEY=your-secret-key
|
||
DATABASE_URL=mysql+pymysql://pfm_user:password@localhost/pfm_db
|
||
GROQ_API_KEY=your-groq-api-key-here
|
||
GROQ_MODEL=llama-3.3-70b-versatile
|
||
UPLOAD_FOLDER=/home/pfm/app/uploads
|
||
MAX_CONTENT_LENGTH=10485760
|
||
FLASK_ENV=production
|
||
APP_CURRENCY=USD # Single currency for all transactions (configurable)
|
||
APP_CURRENCY_SYMBOL=$ # Display symbol
|
||
APP_TIMEZONE=Asia/Ho_Chi_Minh # Server timezone for scheduled jobs
|
||
```
|
||
|
||
---
|
||
|
||
## 13. Security Notes
|
||
|
||
- All routes protected by `@login_required`
|
||
- CSRF protection via Flask-WTF
|
||
- SQL injection prevented by SQLAlchemy ORM
|
||
- File upload validation (type + size limit)
|
||
- Nginx: restrict access to local network if desired
|
||
- HTTPS via Let's Encrypt (Certbot) — `pfm.ngodanguyen.tech`
|
||
- Groq API receives only anonymized transaction summaries (no account names/personal details in prompt)
|
||
- GROQ_API_KEY stored in `.env`, never exposed to frontend
|
||
|
||
---
|
||
|
||
## 14. Future Enhancements (Post-MVP)
|
||
|
||
- Mobile app companion (iOS — fits your skill set)
|
||
- Bank statement auto-import (parse PDF bank statements)
|
||
- Multi-currency with live FX rates (via free API)
|
||
- Expense photo OCR (extract amount from receipt image via Groq vision model)
|
||
- Email/SMS alerts (integrate Twilio — you already know it from FaxDesk)
|
||
- Shared household mode (2 users)
|