05/31 Phase 1: initial codes

2026-05-31 10:03:28 -04:00
parent b0a5ce5399
commit b0c0bd363b
35 changed files with 2871 additions and 164 deletions
@@ -0,0 +1,485 @@
+# Personal Finance Management System (PFMS)
+> Stack: Python Flask · MySQL · Ubuntu Server · Nginx · Gunicorn · Groq API (free AI)
+
+---
+
+## 1. Project Overview
+
+Self-hosted personal finance web app. Tracks income, expenses, investments. AI assistant powered by **Groq API** (free tier, extremely fast inference, no local hardware needed). Everything runs on your Ubuntu server behind Nginx.
+
+---
+
+## 2. Core Features
+
+### 2.1 Dashboard
+- Net worth snapshot (assets − liabilities)
+- Monthly cash flow chart (income vs expenses)
+- Budget utilization gauges per category
+- Recent transactions feed
+- AI insight card (auto-generated daily summary)
+- Investment portfolio mini-widget
+- **USD → VND exchange rate widget** (daily rate, fetched once/day, cached in DB — reference only, independent of app currency)
+
+### 2.2 Income Management
+- Log income entries (salary, freelance, passive, other)
+- Recurring income templates (auto-create entries on schedule)
+- Income source breakdown (chart by source)
+- Month-over-month comparison
+- Export to CSV/Excel
+
+### 2.3 Expense Management
+- Manual expense entry
+- Category tagging (custom + predefined: Food, Rent, Utilities, Transport, Health, Entertainment, etc.)
+- Subcategory support
+- Receipt photo upload (stored locally)
+- Recurring expense detection
+- Budget limits per category with alert thresholds
+- Expense search + filter (date range, category, amount range, keyword)
+- Export to CSV/Excel
+
+### 2.4 Investment Portfolio
+- Asset types: Stocks, ETF, Crypto, Real Estate, Bonds, Cash, Other
+- Holdings tracker (ticker, shares/units, buy price, current price)
+- Manual price update OR auto-fetch via free API (Yahoo Finance via `yfinance`)
+- P&L per holding (unrealized gain/loss)
+- Portfolio allocation pie chart
+- Transaction log (buy/sell history per asset)
+- Cost basis tracking (FIFO)
+
+### 2.5 Budget Planner
+- Monthly budget templates
+- Set budget limits per category
+- Real-time spending vs budget comparison
+- Rollover unused budget (optional toggle)
+- Budget history archive
+
+### 2.6 Goals & Savings
+- Create savings goals (name, target amount, target date, linked account)
+- Track contributions toward each goal
+- Progress bar + projected completion date
+- Emergency fund tracker (X months of expenses)
+
+### 2.7 Reports & Analytics
+- Monthly/quarterly/yearly summary reports
+- Category spending trends (line chart over time)
+- Income growth chart
+- Net worth over time (historical snapshots, monthly auto-saved)
+- Tax year summary (income + deductible expenses)
+- Printable PDF report (via WeasyPrint)
+
+### 2.8 Accounts & Wallets
+- Multiple accounts (bank checking, savings, cash, credit card, crypto wallet)
+- Account balances tracked manually
+- Transfer between accounts (internal transaction)
+- Credit card balance + due date tracking
+
+### 2.9 AI Financial Assistant (Groq API — Free Tier)
+- Chat interface (ask questions about your finances)
+- Context: last 90 days of transactions injected into prompt
+- Example queries:
+  - "Where did I overspend this month?"
+  - "Am I on track for my vacation goal?"
+  - "Summarize my Q1 spending"
+  - "What categories can I cut to save $500/month?"
+- Auto-insight: daily AI summary generated at midnight via cron
+- Model: `llama-3.3-70b-versatile` or `llama-3.1-8b-instant` via Groq (configurable in `.env`)
+- Streaming response (SSE) for real-time chat feel
+
+### 2.10 Notifications & Alerts
+- Browser notifications (via Web Push or in-app toast)
+- Budget threshold alerts (e.g., 80% of category budget used)
+- Bill/recurring expense due reminders
+- Goal milestone celebrations
+
+### 2.11 Settings & Config
+- Profile (name, timezone)
+- **App currency** — single configurable currency (e.g. USD, VND, EUR — set once, used everywhere for all transactions/display)
+- USD→VND rate source preference (ExchangeRate-API free or VCB scrape fallback)
+- Category management (add/edit/delete custom categories)
+- Data backup (export full MySQL dump)
+- Data import (CSV import for bulk transactions)
+- Groq model selector (choose speed vs quality)
+
+---
+
+## 3. Database Schema (MySQL)
+
+### Tables
+
+```
+users                  — single user (self-hosted, no multi-tenant)
+accounts               — bank/wallet accounts
+categories             — expense/income categories
+transactions           — all money movements (income/expense/transfer)
+investments            — holdings/portfolio positions
+investment_transactions — buy/sell log
+budgets                — monthly budget limits per category
+goals                  — savings goals
+goal_contributions     — deposits toward each goal
+net_worth_snapshots    — monthly net worth history
+ai_insights            — stored daily AI summaries
+recurring_rules        — templates for recurring income/expenses
+receipts               — receipt image metadata
+fx_rates               — daily USD/VND rate cache (date, rate, source)
+```
+
+### Key Table: `transactions`
+```sql
+id, account_id, category_id, type (income/expense/transfer),
+amount, currency, description, date, notes,
+is_recurring, recurring_rule_id, receipt_id,
+created_at, updated_at
+```
+
+### Key Table: `investments`
+```sql
+id, asset_name, ticker, asset_type (stock/etf/crypto/real_estate/bond/cash/other),
+shares, avg_cost_basis, current_price, last_price_update,
+currency, notes, created_at
+```
+
+### Key Table: `budgets`
+```sql
+id, category_id, month (YYYY-MM), limit_amount,
+rollover_enabled, rollover_amount, created_at
+```
+
+---
+
+## 4. Project File Structure
+
+```
+pfm/
+├── app/
+│   ├── __init__.py              # Flask app factory
+│   ├── config.py                # Config classes (dev/prod)
+│   ├── extensions.py            # db, login_manager, etc.
+│   │
+│   ├── models/
+│   │   ├── user.py
+│   │   ├── account.py
+│   │   ├── transaction.py
+│   │   ├── category.py
+│   │   ├── investment.py
+│   │   ├── budget.py
+│   │   ├── goal.py
+│   │   ├── ai_insight.py
+│   │   ├── fx_rate.py
+│   │   └── recurring_rule.py
+│   │
+│   ├── routes/
+│   │   ├── auth.py              # Login/logout (single user)
+│   │   ├── dashboard.py
+│   │   ├── transactions.py
+│   │   ├── income.py
+│   │   ├── expenses.py
+│   │   ├── investments.py
+│   │   ├── budgets.py
+│   │   ├── goals.py
+│   │   ├── accounts.py
+│   │   ├── reports.py
+│   │   ├── ai.py                # AI chat + insights (SSE)
+│   │   ├── settings.py
+│   │   └── api.py               # Internal JSON API endpoints
+│   │
+│   ├── services/
+│   │   ├── ai_service.py        # Groq API integration + context builder
+│   │   ├── budget_service.py    # Budget calc + alert logic
+│   │   ├── investment_service.py # yfinance price fetcher
+│   │   ├── fx_service.py        # USD/VND rate fetch + cache logic
+│   │   ├── report_service.py    # PDF generation (WeasyPrint)
+│   │   ├── recurring_service.py # Recurring rule processor
+│   │   ├── import_service.py    # CSV import parser
+│   │   └── snapshot_service.py  # Net worth snapshot scheduler
+│   │
+│   ├── templates/
+│   │   ├── base.html
+│   │   ├── auth/
+│   │   ├── dashboard/
+│   │   ├── transactions/
+│   │   ├── investments/
+│   │   ├── budgets/
+│   │   ├── goals/
+│   │   ├── accounts/
+│   │   ├── reports/
+│   │   ├── ai/
+│   │   └── settings/
+│   │
+│   ├── static/
+│   │   ├── css/
+│   │   ├── js/
+│   │   └── img/                 # Static images
+│   │
+│   └── utils/
+│       ├── decorators.py        # Auth required, etc.
+│       ├── formatters.py        # Currency, date formatting
+│       └── validators.py
+│
+├── migrations/                  # Flask-Migrate (Alembic)
+├── scripts/
+│   ├── init_db.py               # First-run DB setup + seed categories
+│   ├── daily_snapshot.py        # Cron: net worth snapshot
+│   ├── daily_ai_insight.py      # Cron: generate AI summary
+│   ├── process_recurring.py     # Cron: create recurring transactions
+│   ├── fetch_prices.py          # Cron: update investment prices
+│   └── fetch_fx_rate.py         # Cron: fetch daily USD/VND rate
+│
+├── tests/
+├── .env                         # Secrets (not committed)
+├── .env.example
+├── requirements.txt
+├── wsgi.py
+├── CLAUDE.md                    # This file
+└── deploy.md                    # Server setup guide
+```
+
+---
+
+## 5. Python Dependencies (`requirements.txt`)
+
+```
+flask
+flask-sqlalchemy
+flask-login
+flask-migrate
+flask-wtf
+pymysql
+python-dotenv
+gunicorn
+groq                  # Groq official Python SDK
+yfinance              # Investment price fetching
+weasyprint            # PDF report generation
+openpyxl              # Excel export
+Pillow                # Receipt image processing
+apscheduler           # In-process scheduler (alternative to cron)
+```
+
+---
+
+## 6. AI Integration — Groq API (Free Tier)
+
+### Why Groq
+- Free tier: 14,400 requests/day, 500,000 tokens/minute
+- Fastest inference available (LPU hardware) — responses feel instant
+- No local GPU/RAM needed
+- Models: `llama-3.3-70b-versatile` (best quality), `llama-3.1-8b-instant` (fastest)
+
+### How It Works
+```
+User sends chat message
+  → ai_service.py builds context (last 90 days transactions summary)
+  → POST to Groq API (`/chat/completions`) with Bearer token
+  → Stream response back via SSE to browser
+  → Response stored in ai_insights table
+```
+
+### Context Injection Strategy
+```python
+# ai_service.py builds a prompt like:
+"""
+You are a personal finance assistant. Here is the user's financial data:
+
+CURRENT MONTH SUMMARY:
+- Total Income: $X
+- Total Expenses: $Y  
+- Top spending categories: Food ($A), Transport ($B), ...
+- Budget alerts: Entertainment 92% used
+
+RECENT TRANSACTIONS (last 20):
+[date] [category] [amount] [description]
+...
+
+NET WORTH: $Z
+ACTIVE GOALS: Vacation Fund ($1,200 / $3,000)
+
+User question: {user_message}
+
+Answer concisely and specifically based on the data above.
+"""
+```
+
+### Fallback
+If Groq API key missing/invalid → show friendly message "AI assistant unavailable. Check GROQ_API_KEY in settings."
+If Groq rate limit hit → show "AI rate limit reached. Try again shortly."
+
+---
+
+## 7. USD → VND Exchange Rate
+
+> **Reference widget only** — independent of the app's transaction currency. All income/expenses/investments use the single configured app currency. This widget is informational display only.
+
+### Data Source — Free, No API Key Required
+
+Primary: **ExchangeRate-API open endpoint**
+```
+https://open.er-api.com/v6/latest/USD
+```
+Returns JSON with all rates including VND. Free tier, no key, 1,500 req/month.
+
+Fallback: **Vietcombank (VCB) rate scrape**
+```
+https://www.vietcombank.com.vn/en/KHCN/Chuyen-trang-KHCN/Pages/ty-gia.aspx
+```
+Scrape VCB's official buying/selling rate as backup.
+
+### DB Table: `fx_rates`
+```sql
+id            INT AUTO_INCREMENT PRIMARY KEY
+date          DATE NOT NULL UNIQUE      -- one record per day
+usd_to_vnd    DECIMAL(12,2) NOT NULL   -- e.g. 25,450.00
+source        VARCHAR(50)              -- 'exchangerate-api' | 'vcb' | 'manual'
+fetched_at    DATETIME
+```
+
+### `fx_service.py` Logic
+```
+get_today_rate():
+  1. Check fx_rates table for today's date
+  2. If found → return cached rate (no API call)
+  3. If not found → fetch from ExchangeRate-API
+  4. If API fails → try VCB scrape
+  5. If both fail → return last known rate from DB + show "rate may be outdated" flag
+  6. Save new rate to DB
+```
+
+### Dashboard Widget Display
+- Card on dashboard header area (top bar or sidebar)
+- Shows: `1 USD = 25,450 ₫` with date label
+- Color: neutral/info (blue or grey — not green/red, it's informational)
+- Click → opens 30-day rate history mini-chart (Chart.js, line chart)
+- Stale indicator: if rate is >1 day old, show small warning icon
+
+### 30-Day Rate History
+- Stored in `fx_rates` table (one row/day, auto-accumulates)
+- Chart available on dashboard click or Reports page
+- Shows trend: flat/up/down with % change label
+
+### Scheduled Job
+- Daily 8AM fetch (after markets open in Vietnam)
+- `scripts/fetch_fx_rate.py`
+- systemd timer unit: `pfm-fxrate.timer`
+
+---
+
+## 8. UI/UX Design Direction
+
+- **Style**: Clean financial dashboard — dark sidebar, white/light content area
+- **Charts**: Chart.js (CDN, no build step needed)
+- **Tables**: DataTables.js for sortable/searchable transaction tables
+- **Forms**: WTForms + Bootstrap 5
+- **Icons**: Bootstrap Icons or Feather Icons
+- **Color scheme**: Deep navy sidebar, white cards, green (income), red (expense), blue (investment)
+- **Mobile**: Responsive (Bootstrap grid)
+- **AI Chat**: Floating chat panel (slide-in from right), SSE streaming text
+
+---
+
+## 9. Authentication
+
+- Single-user app (self-hosted)
+- Flask-Login with username/password
+- Session-based auth
+- Password hashed with Werkzeug (bcrypt)
+- Optional: IP whitelist via Nginx (allow only LAN access)
+
+---
+
+## 10. Scheduled Jobs (systemd timers or APScheduler)
+
+| Job | Schedule | Script |
+|-----|----------|--------|
+| Process recurring transactions | Daily 6AM | `process_recurring.py` |
+| Fetch USD/VND exchange rate | Daily 8AM | `fetch_fx_rate.py` |
+| Fetch investment prices | Daily 4PM | `fetch_prices.py` |
+| Save net worth snapshot | 1st of month | `daily_snapshot.py` |
+| Generate AI daily insight | Daily midnight | `daily_ai_insight.py` |
+
+Recommend **APScheduler** inside Flask app (simpler) OR separate systemd timer units (more robust).
+
+---
+
+## 11. Development Phases
+
+### Phase 1 — Foundation
+- [ ] Flask app factory + config
+- [ ] MySQL models + migrations
+- [ ] Auth (login/logout)
+- [ ] Base template + sidebar nav
+
+### Phase 2 — Core Transactions
+- [ ] Accounts CRUD
+- [ ] Categories CRUD
+- [ ] Transaction entry (income + expense)
+- [ ] Transaction list with filter/search
+- [ ] Dashboard basics (totals, recent feed)
+- [ ] USD→VND rate widget + `fx_service.py` + daily fetch job
+
+### Phase 3 — Budget & Goals
+- [ ] Budget planner (set limits)
+- [ ] Budget vs actual comparison
+- [ ] Goals CRUD + contribution tracking
+
+### Phase 4 — Investments
+- [ ] Holdings CRUD
+- [ ] Buy/sell transaction log
+- [ ] yfinance price auto-fetch
+- [ ] Portfolio charts
+
+### Phase 5 — AI Assistant
+- [ ] Groq API integration + context builder
+- [ ] Chat UI with SSE streaming
+- [ ] Daily auto-insight cron
+
+### Phase 6 — Reports & Export
+- [ ] Monthly summary page
+- [ ] PDF export (WeasyPrint)
+- [ ] CSV/Excel export
+- [ ] Net worth history chart
+
+### Phase 7 — Polish
+- [ ] Recurring transaction engine
+- [ ] Receipt upload
+- [ ] CSV import
+- [ ] Budget alerts + notifications
+- [ ] Mobile responsiveness pass
+
+---
+
+## 12. Environment Variables (`.env`)
+
+```
+SECRET_KEY=your-secret-key
+DATABASE_URL=mysql+pymysql://pfm_user:password@localhost/pfm_db
+GROQ_API_KEY=your-groq-api-key-here
+GROQ_MODEL=llama-3.3-70b-versatile
+UPLOAD_FOLDER=/home/pfm/app/uploads
+MAX_CONTENT_LENGTH=10485760
+FLASK_ENV=production
+APP_CURRENCY=USD                 # Single currency for all transactions (configurable)
+APP_CURRENCY_SYMBOL=$            # Display symbol
+APP_TIMEZONE=Asia/Ho_Chi_Minh    # Server timezone for scheduled jobs
+```
+
+---
+
+## 13. Security Notes
+
+- All routes protected by `@login_required`
+- CSRF protection via Flask-WTF
+- SQL injection prevented by SQLAlchemy ORM
+- File upload validation (type + size limit)
+- Nginx: restrict access to local network if desired
+- HTTPS via Let's Encrypt (Certbot) — `pfm.ngodanguyen.tech`
+- Groq API receives only anonymized transaction summaries (no account names/personal details in prompt)
+- GROQ_API_KEY stored in `.env`, never exposed to frontend
+
+---
+
+## 14. Future Enhancements (Post-MVP)
+
+- Mobile app companion (iOS — fits your skill set)
+- Bank statement auto-import (parse PDF bank statements)
+- Multi-currency with live FX rates (via free API)
+- Expense photo OCR (extract amount from receipt image via Groq vision model)
+- Email/SMS alerts (integrate Twilio — you already know it from FaxDesk)
+- Shared household mode (2 users)