nngo/Personal-Finance-Management
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

06/04 Improve system logs UI/UX

Browse Source
This commit is contained in:
nngo
2026-06-04 11:58:04 -04:00
parent 1fc33ca60f
commit 48322b609f
5 changed files with 553 additions and 250 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+90 -37
View File
@@ -21,6 +21,8 @@ Self-hosted personal finance web app. Tracks income, expenses, investments. AI a
- Budget utilization per category - Budget utilization per category
- Recent transactions feed (last 8) - Recent transactions feed (last 8)
- AI daily insight card (Groq-generated, stored in DB) - AI daily insight card (Groq-generated, stored in DB)
- **Savings Rate** stat card — net cash flow ÷ income for the selected period; green ≥ 20%, blue > 0%, red negative
- **Schwab expiry warning** — banner shown when Schwab refresh token expires within 2 days
- USD → VND exchange rate widget — reference only, independent of app currency - USD → VND exchange rate widget — reference only, independent of app currency
- Click to expand 30-day history chart - Click to expand 30-day history chart
- ↻ refresh button (force-fetches fresh rate without page reload) - ↻ refresh button (force-fetches fresh rate without page reload)
@@ -89,6 +91,7 @@ Self-hosted personal finance web app. Tracks income, expenses, investments. AI a
- **Per-account sections** — when investments span multiple accounts (e.g. Individual + Roth IRA), holdings are grouped into one card per account, each showing account name, total value, and holdings table. Allocation sidebar also shows "By Account" breakdown - **Per-account sections** — when investments span multiple accounts (e.g. Individual + Roth IRA), holdings are grouped into one card per account, each showing account name, total value, and holdings table. Allocation sidebar also shows "By Account" breakdown
- `investments.account_id` FK — each Schwab-synced holding is stamped with its source account; same ticker in different accounts (AAPL in Individual vs Roth IRA) stays as separate rows - `investments.account_id` FK — each Schwab-synced holding is stamped with its source account; same ticker in different accounts (AAPL in Individual vs Roth IRA) stays as separate rows
- Holdings table uses a Jinja2 `{% macro %}` (reused across single and multi-account views) - Holdings table uses a Jinja2 `{% macro %}` (reused across single and multi-account views)
- **Price history chart** on investment detail page — 1W / 1M / 3M / 6M / 1Y timeframe buttons; fetches from `/investments/api/price-history/<ticker>`
### 2.8 AI Financial Assistant ### 2.8 AI Financial Assistant
- Chat UI with SSE streaming (Groq API, word-by-word response) - Chat UI with SSE streaming (Groq API, word-by-word response)
@@ -124,6 +127,8 @@ Self-hosted personal finance web app. Tracks income, expenses, investments. AI a
- Profile: name, email, timezone, currency, Groq model - Profile: name, email, timezone, currency, Groq model
- 8 currency options (USD/VND/EUR/GBP/JPY/AUD/CAD/SGD) — auto-updates symbol - 8 currency options (USD/VND/EUR/GBP/JPY/AUD/CAD/SGD) — auto-updates symbol
- Password change (requires current password) - Password change (requires current password)
- **Two-Factor Authentication (TOTP)** — enable/disable TOTP 2FA; setup shows QR code + manual key entry; disable requires password confirmation
- **Audit Log** (`/settings/audit`) — paginated log of login, 2FA, password, and bank-connection events with IP address; filterable by event type
- Recurring rules: CRUD, pause/enable, frequency (daily/weekly/biweekly/monthly/quarterly/yearly) - Recurring rules: CRUD, pause/enable, frequency (daily/weekly/biweekly/monthly/quarterly/yearly)
- "Run Now" button to process due rules immediately - "Run Now" button to process due rules immediately
- CSV import: upload → preview with ⚠ warnings → confirm - CSV import: upload → preview with ⚠ warnings → confirm
@@ -175,8 +180,10 @@ Self-hosted personal finance web app. Tracks income, expenses, investments. AI a
- Position zero-quantity and empty-symbol positions skipped; `null` positions array guarded - Position zero-quantity and empty-symbol positions skipped; `null` positions array guarded
- **Investments sync**: topbar "Sync Schwab" button on investments page → `POST /investments/sync-schwab` → runs snapshot for all mapped accounts (ignores stale `connection_id` — always uses active connection) - **Investments sync**: topbar "Sync Schwab" button on investments page → `POST /investments/sync-schwab` → runs snapshot for all mapped accounts (ignores stale `connection_id` — always uses active connection)
- Token auto-refresh: access token expires 30 min; refreshed automatically before API calls - Token auto-refresh: access token expires 30 min; refreshed automatically before API calls
- **Refresh token expiry**: `refresh_token_expires_at` tracked in DB; reset on every successful token exchange; dashboard warns when ≤ 2 days remain
- **Account type map**: CASH→checking, MARGIN/IRA/ROTH_IRA/ROLLOVER_IRA/TRADITIONAL_IRA/401K/ROTH_401K/BROKERAGE→investment; unknown types fall back to `'other'`
- Config: `SCHWAB_CLIENT_ID`, `SCHWAB_CLIENT_SECRET`, `SCHWAB_REDIRECT_URI` - Config: `SCHWAB_CLIENT_ID`, `SCHWAB_CLIENT_SECRET`, `SCHWAB_REDIRECT_URI`
- Models: `schwab_connections`, `schwab_accounts` (2 new tables) - Models: `schwab_connections`, `schwab_accounts` (2 tables)
### 2.15 Bank Statement Import ### 2.15 Bank Statement Import
- Sidebar link: "Import Statement" under Money section - Sidebar link: "Import Statement" under Money section
@@ -195,9 +202,9 @@ Self-hosted personal finance web app. Tracks income, expenses, investments. AI a
## 3. Database Schema (MySQL) ## 3. Database Schema (MySQL)
### All 18 Tables ### All 19 Tables
``` ```
users — single user, hashed password, currency/timezone prefs users — single user, hashed password, currency/timezone prefs, totp_secret, totp_enabled
accounts — bank/wallet accounts (balance managed per provider rules) accounts — bank/wallet accounts (balance managed per provider rules)
categories — expense/income categories with color/icon categories — expense/income categories with color/icon
transactions — income/expense/transfer, receipt_id, recurring_rule_id transactions — income/expense/transfer, receipt_id, recurring_rule_id
@@ -211,9 +218,10 @@ investment_transactions — buy/sell/dividend/split log
net_worth_snapshots — monthly snapshots: assets, liabilities, net_worth (JSON) net_worth_snapshots — monthly snapshots: assets, liabilities, net_worth (JSON)
ai_insights — stored AI responses: daily_summary / chat_response ai_insights — stored AI responses: daily_summary / chat_response
fx_rates — daily USD/VND rate cache (date UNIQUE, source) fx_rates — daily USD/VND rate cache (date UNIQUE, source)
teller_enrollments — Teller enrollment: enrollment_id, access_token, institution_name audit_logs — security event log: action, description, ip_address, timestamp
teller_enrollments — Teller enrollment: enrollment_id, access_token (TEXT, encrypted), institution_name
teller_accounts — Teller account ↔ PFM account mapping, last_sync_date teller_accounts — Teller account ↔ PFM account mapping, last_sync_date
schwab_connections — Schwab OAuth tokens: access_token, refresh_token, token_expires_at schwab_connections — Schwab OAuth tokens (TEXT, encrypted), token_expires_at, refresh_token_expires_at
schwab_accounts — Schwab account ↔ PFM account mapping, account_hash (hashValue) schwab_accounts — Schwab account ↔ PFM account mapping, account_hash (hashValue)
``` ```
@@ -223,10 +231,14 @@ schwab_accounts — Schwab account ↔ PFM account mapping, account_hash
- `investments.shares` / `avg_cost_basis` — recalculated from `investment_transactions` (FIFO) for manual holdings; overwritten directly by Schwab snapshot for synced holdings - `investments.shares` / `avg_cost_basis` — recalculated from `investment_transactions` (FIFO) for manual holdings; overwritten directly by Schwab snapshot for synced holdings
- `transactions.notes` — used to store import source IDs: `Teller:<id>`, `Schwab:<activityId>`, or `import:<fitid>` - `transactions.notes` — used to store import source IDs: `Teller:<id>`, `Schwab:<activityId>`, or `import:<fitid>`
- `schwab_accounts.account_hash` — Schwab `hashValue` (encrypted account number), required in all API paths - `schwab_accounts.account_hash` — Schwab `hashValue` (encrypted account number), required in all API paths
- `teller_enrollments.access_token` — stored as `TEXT` (widened from VARCHAR(128)); encrypted at rest via `EncryptedText` TypeDecorator
- `schwab_connections.access_token` / `refresh_token` — `TEXT`, encrypted at rest; `refresh_token_expires_at` reset on every token exchange
- `users.totp_secret` — base32 TOTP secret (VARCHAR 64); NULL when 2FA disabled
### Migration Scripts ### Migration Scripts
``` ```
scripts/add_investment_account.py — adds investments.account_id column (run once after deploy) scripts/add_investment_account.py — adds investments.account_id column (run once)
scripts/add_security_columns.py — adds totp columns, audit_logs table, widens token columns to TEXT (run once)
``` ```
--- ---
@@ -244,12 +256,12 @@ pfm/ # /home/pfm/web on server
│ └── app.log │ └── app.log
│ │
├── app/ ├── app/
│ ├── __init__.py │ ├── __init__.py # session idle timeout hook; Sentry init; limiter init
│ ├── config.py # SCHWAB_* vars added │ ├── config.py # SENTRY_DSN, SESSION_IDLE_MINUTES, RATELIMIT_STORAGE_URI added
│ ├── extensions.py │ ├── extensions.py # + limiter (flask-limiter, storage via RATELIMIT_STORAGE_URI)
│ │ │ │
│ ├── models/ │ ├── models/
│ │ ├── user.py │ │ ├── user.py # + totp_secret, totp_enabled columns
│ │ ├── account.py │ │ ├── account.py
│ │ ├── category.py │ │ ├── category.py
│ │ ├── transaction.py │ │ ├── transaction.py
@@ -261,23 +273,24 @@ pfm/ # /home/pfm/web on server
│ │ ├── net_worth_snapshot.py │ │ ├── net_worth_snapshot.py
│ │ ├── ai_insight.py │ │ ├── ai_insight.py
│ │ ├── fx_rate.py │ │ ├── fx_rate.py
│ │ ├── teller_enrollment.py # TellerEnrollment + TellerAccount │ │ ├── audit_log.py # AuditLog model (action, description, ip_address, timestamp)
│ │ └── schwab_connection.py # SchwabConnection + SchwabAccount (NEW) │ │ ├── teller_enrollment.py # access_token now EncryptedText (TEXT column)
│ │ └── schwab_connection.py # tokens now EncryptedText; + refresh_token_expires_at
│ │ │ │
│ ├── routes/ │ ├── routes/
│ │ ├── auth.py │ │ ├── auth.py # + TOTP verify/setup/disable routes; rate limits; audit calls
│ │ ├── dashboard.py # credit card "owed" display; no recalc_all │ │ ├── dashboard.py # + savings_rate; Schwab expiry warning
│ │ ├── accounts.py # teller_map + schwab_map; skip calc_balance for providers │ │ ├── accounts.py # teller_map + schwab_map; skip calc_balance for providers
│ │ ├── categories.py │ │ ├── categories.py
│ │ ├── transactions.py # + set-category AJAX endpoint │ │ ├── transactions.py # + set-category AJAX; fixed income-form-submits-as-expense bug
│ │ ├── budgets.py │ │ ├── budgets.py
│ │ ├── goals.py │ │ ├── goals.py
│ │ ├── investments.py # + sync-schwab route │ │ ├── investments.py # + sync-schwab route; price-history API
│ │ ├── reports.py │ │ ├── reports.py
│ │ ├── ai.py │ │ ├── ai.py
│ │ ├── settings.py │ │ ├── settings.py # + audit_log route; audit calls on password change
│ │ ├── teller.py # balance uses ledger/available correctly; full_resync honours next param │ │ ├── teller.py # balance uses ledger/available correctly
│ │ ├── schwab.py # OAuth, mapping, sync, snapshot (NEW) │ │ ├── schwab.py # OAuth, mapping, sync, snapshot; audit calls; fallback type 'other'
│ │ ├── bank_import.py │ │ ├── bank_import.py
│ │ └── logs.py │ │ └── logs.py
│ │ │ │
@@ -294,22 +307,29 @@ pfm/ # /home/pfm/web on server
│ │ ├── recurring_service.py │ │ ├── recurring_service.py
│ │ ├── report_service.py │ │ ├── report_service.py
│ │ ├── teller_service.py # auto_categorize; correct sign convention; live balance after sync │ │ ├── teller_service.py # auto_categorize; correct sign convention; live balance after sync
│ │ ├── schwab_service.py # OAuth, account hash, snapshot sync, position upsert (NEW) │ │ ├── schwab_service.py # + expanded ACCOUNT_TYPE_MAP; refresh_token_expires_at always reset
│ │ └── bank_import_service.py │ │ └── bank_import_service.py
│ │ │ │
│ ├── templates/ │ ├── templates/
│ │ ├── base.html │ │ ├── base.html # mobile responsive tweaks
│ │ ├── dashboard/index.html # credit card owed display │ │ ├── auth/totp_setup.html # QR code + manual key entry for 2FA setup
│ │ ├── auth/totp_verify.html # 6-digit code entry on login
│ │ ├── dashboard/index.html # + savings_rate card; Schwab warning banner
│ │ ├── accounts/index.html # Teller/Schwab badges + action buttons │ │ ├── accounts/index.html # Teller/Schwab badges + action buttons
│ │ ├── transactions/index.html # inline category <select> + AJAX │ │ ├── transactions/index.html # inline category <select> + AJAX; pagination info
│ │ ├── investments/index.html # per-account sections; holdings_table macro; Sync Schwab btn │ │ ├── investments/index.html # per-account sections; Sync Schwab btn
│ │ ├── teller/index.html # connect/disconnect only (sync buttons removed) │ │ ├── investments/detail.html # + price history chart (1W/1M/3M/6M/1Y)
│ │ ├── schwab/ # index.html, map_accounts.html, preview.html (NEW) │ │ ├── settings/audit.html # audit log viewer with event filter
│ │ ├── settings/index.html # + 2FA section; audit log nav card
│ │ ├── teller/index.html # connect/disconnect only
│ │ ├── schwab/ # index.html, map_accounts.html, preview.html
│ │ └── ... (other templates unchanged) │ │ └── ... (other templates unchanged)
│ │ │ │
│ └── utils/ │ └── utils/
│ ├── formatters.py │ ├── formatters.py
│ └── decorators.py │ ├── decorators.py
│ ├── audit.py # audit() helper — writes AuditLog rows; swallows DB errors
│ └── crypto.py # EncryptedText SQLAlchemy TypeDecorator (Fernet, key=SHA256(SECRET_KEY))
│ │
├── scripts/ ├── scripts/
│ ├── init_db.py │ ├── init_db.py
@@ -318,7 +338,9 @@ pfm/ # /home/pfm/web on server
│ ├── fetch_prices.py │ ├── fetch_prices.py
│ ├── daily_snapshot.py │ ├── daily_snapshot.py
│ ├── daily_ai_insight.py │ ├── daily_ai_insight.py
│ └── add_investment_account.py # NEW — adds investments.account_id column │ ├── add_investment_account.py # adds investments.account_id column
│ ├── add_security_columns.py # adds TOTP cols, audit_logs table, widens token cols to TEXT
│ └── sync_schwab.py # daily Schwab auto-sync (balance + positions + transactions)
│ │
└── tests/ └── tests/
``` ```
@@ -345,6 +367,12 @@ requests==2.32.3
cryptography==44.0.2 cryptography==44.0.2
python-dateutil==2.9.0 python-dateutil==2.9.0
pdfplumber==0.11.4 pdfplumber==0.11.4
# Security
flask-limiter==3.5.0
pyotp==2.9.0
qrcode==7.4.2
# Monitoring
sentry-sdk[flask]==2.7.0
``` ```
--- ---
@@ -452,10 +480,16 @@ pdfplumber==0.11.4
- Single-user, Flask-Login, session-based - Single-user, Flask-Login, session-based
- Hashed password (Werkzeug `generate_password_hash`) - Hashed password (Werkzeug `generate_password_hash`)
- `SESSION_COOKIE_SECURE=True`, `SESSION_COOKIE_HTTPONLY=True`, `SESSION_COOKIE_SAMESITE='Lax'` - `SESSION_COOKIE_SECURE=True`, `SESSION_COOKIE_HTTPONLY=True`, `SESSION_COOKIE_SAMESITE='Lax'`
- **Session idle timeout** — configurable via `SESSION_IDLE_MINUTES` (default 60); enforced in `before_request` hook
- **TOTP 2FA** — optional TOTP second factor (pyotp); setup via QR code; verify endpoint rate-limited `10/min; 30/hr`; 5 failed attempts clears pending session and forces re-login
- **Rate limiting** — flask-limiter on login (`10/min; 30/hr`), TOTP verify (`10/min; 30/hr`), TOTP setup (`10/min`); storage backend set via `RATELIMIT_STORAGE_URI` (use Redis in production to share limits across Gunicorn workers; defaults to `memory://` per-process if unset)
- **At-rest encryption** — Teller and Schwab OAuth tokens encrypted in DB via `EncryptedText` SQLAlchemy TypeDecorator (Fernet symmetric, key = SHA-256(SECRET_KEY)); columns are `TEXT` not `VARCHAR`
- **Audit log** — security events written to `audit_logs` table via `app/utils/audit.py`; events: `login_success`, `login_success_2fa`, `login_failed`, `login_failed_2fa`, `totp_enabled`, `totp_disabled`, `password_changed`, `schwab_connected`, `schwab_disconnected`
- CSRF protection on all forms (Flask-WTF); meta tag in base.html for AJAX - CSRF protection on all forms (Flask-WTF); meta tag in base.html for AJAX
- SQLAlchemy ORM (no raw SQL) - SQLAlchemy ORM (no raw SQL)
- Schwab OAuth state parameter validated on callback (CSRF protection) - Schwab OAuth state parameter validated on callback (CSRF protection)
- `next` redirect params validated to start with `/` (no open redirect) - `next` redirect params validated to start with `/` (no open redirect)
- **Sentry** (optional) — error monitoring; enable via `SENTRY_DSN` env var; `send_default_pii=False`
--- ---
@@ -468,6 +502,7 @@ pdfplumber==0.11.4
| Fetch investment prices | Mon-Fri 4PM | `fetch_prices.py` | yfinance, all tickers | | Fetch investment prices | Mon-Fri 4PM | `fetch_prices.py` | yfinance, all tickers |
| Net worth snapshot | 1st of month 00:05 | `daily_snapshot.py` | Saves to net_worth_snapshots | | Net worth snapshot | 1st of month 00:05 | `daily_snapshot.py` | Saves to net_worth_snapshots |
| AI daily insight | Daily 00:01 | `daily_ai_insight.py` | Skips if already done today | | AI daily insight | Daily 00:01 | `daily_ai_insight.py` | Skips if already done today |
| Schwab auto-sync | Daily 7AM | `sync_schwab.py` | Balance + positions + transactions; warns if refresh token expires soon |
| DB backup | Daily 2AM | pfm-backup (systemd) | mysqldump → gzip, keep 30 days | | DB backup | Daily 2AM | pfm-backup (systemd) | mysqldump → gzip, keep 30 days |
--- ---
@@ -497,6 +532,10 @@ TELLER_WEBHOOK_SECRET=your-webhook-secret
SCHWAB_CLIENT_ID=your-schwab-client-id SCHWAB_CLIENT_ID=your-schwab-client-id
SCHWAB_CLIENT_SECRET=your-schwab-client-secret SCHWAB_CLIENT_SECRET=your-schwab-client-secret
SCHWAB_REDIRECT_URI=https://pfm.ngodanguyen.tech/schwab/callback SCHWAB_REDIRECT_URI=https://pfm.ngodanguyen.tech/schwab/callback
# Security (optional)
SENTRY_DSN= # leave blank to disable Sentry
SESSION_IDLE_MINUTES=60 # session idle timeout in minutes
RATELIMIT_STORAGE_URI=redis://localhost:6379 # use Redis to share rate limits across Gunicorn workers
``` ```
--- ---
@@ -505,7 +544,7 @@ SCHWAB_REDIRECT_URI=https://pfm.ngodanguyen.tech/schwab/callback
| Blueprint | Prefix | Key routes | | Blueprint | Prefix | Key routes |
|-----------|--------|------------| |-----------|--------|------------|
| auth | /auth | login, logout | | auth | /auth | login, logout, totp/verify, totp/setup, totp/disable |
| dashboard | / | index, api/fx-history, api/fx-refresh | | dashboard | / | index, api/fx-history, api/fx-refresh |
| accounts | /accounts | CRUD, adjust | | accounts | /accounts | CRUD, adjust |
| categories | /categories | CRUD | | categories | /categories | CRUD |
@@ -515,7 +554,7 @@ SCHWAB_REDIRECT_URI=https://pfm.ngodanguyen.tech/schwab/callback
| investments | /investments | index, new, detail, edit, delete, add_transaction, refresh-prices, sync-schwab, api/price, api/daychange, api/price-history | | investments | /investments | index, new, detail, edit, delete, add_transaction, refresh-prices, sync-schwab, api/price, api/daychange, api/price-history |
| reports | /reports | monthly, quarterly, yearly, tax, export/csv\|excel\|pdf, snapshot | | reports | /reports | monthly, quarterly, yearly, tax, export/csv\|excel\|pdf, snapshot |
| ai | /ai | index, stream (SSE), history, generate-insight | | ai | /ai | index, stream (SSE), history, generate-insight |
| settings | /settings | index, profile, password, recurring, import, recalc-balances, upload_receipt, delete_receipt, view_receipt | | settings | /settings | index, profile, password, audit, recurring, import, recalc-balances, upload_receipt, delete_receipt, view_receipt |
| teller | /teller | callback, map, index, sync, sync/confirm, sync/all, balance, balance/all, resync, disconnect, webhook | | teller | /teller | callback, map, index, sync, sync/confirm, sync/all, balance, balance/all, resync, disconnect, webhook |
| schwab | /schwab | connect, callback, index, map, sync/`<id>`, sync/confirm, resync, snapshot/`<id>`, disconnect | | schwab | /schwab | connect, callback, index, map, sync/`<id>`, sync/confirm, resync, snapshot/`<id>`, disconnect |
| bank_import | /bank-import | index, parse (AJAX), import (AJAX) | | bank_import | /bank-import | index, parse (AJAX), import (AJAX) |
@@ -529,9 +568,13 @@ SCHWAB_REDIRECT_URI=https://pfm.ngodanguyen.tech/schwab/callback
- FX rate widget: `open.er-api.com` may return stale values; yfinance is the reliable primary - FX rate widget: `open.er-api.com` may return stale values; yfinance is the reliable primary
- WeasyPrint PDF: requires `libpango*` system libs on server - WeasyPrint PDF: requires `libpango*` system libs on server
- Bank statement PDF import: scanned/image PDFs have no text layer; must use digital download - Bank statement PDF import: scanned/image PDFs have no text layer; must use digital download
- Schwab: `investments.account_id` column requires migration — run `scripts/add_investment_account.py` once after deploy - Schwab: run `scripts/add_investment_account.py` then `scripts/add_security_columns.py` once after fresh deploy
- Schwab: after first connect, run "Balance & Positions" to populate investments; then re-sync if holdings were already added manually (they will be updated to link to the account) - Schwab: after first connect, run "Balance & Positions" to populate investments; then re-sync if holdings were already added manually
- Schwab refresh token: Schwab tokens last ~7 days; `refresh_token_expires_at` is reset on every token exchange (including access-only refreshes); dashboard warns at ≤ 2 days
- Teller: `access_token` column is `TEXT` (widened from VARCHAR(128) to fit Fernet-encrypted values); run `scripts/add_security_columns.py` to apply
- Teller: development environment only; requires cert/key from Teller Dashboard - Teller: development environment only; requires cert/key from Teller Dashboard
- Rate limiter: defaults to `memory://` per-process if `RATELIMIT_STORAGE_URI` is not set — effective limit is `stated_limit × num_workers`; set `RATELIMIT_STORAGE_URI=redis://localhost:6379` in production
- `EncryptedText` TypeDecorator: key = SHA-256(SECRET_KEY); changing SECRET_KEY invalidates all stored tokens (requires reconnect)
- MySQL does not support `NULLS LAST`; use `func.isnull(column)` for null-last ordering - MySQL does not support `NULLS LAST`; use `func.isnull(column)` for null-last ordering
--- ---
@@ -553,27 +596,37 @@ SCHWAB_REDIRECT_URI=https://pfm.ngodanguyen.tech/schwab/callback
| 2026-06 | Schwab uses `hashValue` (not raw account number) in API paths | schwab.py, schwab_service.py | | 2026-06 | Schwab uses `hashValue` (not raw account number) in API paths | schwab.py, schwab_service.py |
| 2026-06 | `next` redirect params validated to start with `/` (no open redirect) | teller.py, schwab.py | | 2026-06 | `next` redirect params validated to start with `/` (no open redirect) | teller.py, schwab.py |
| 2026-06 | Teller income/expense type corrected (positive = income) | teller_service.py | | 2026-06 | Teller income/expense type corrected (positive = income) | teller_service.py |
| 2026-06 | `EncryptedText.process_bind_param` removed silent plaintext fallback — raises on encrypt failure | crypto.py |
| 2026-06 | Rate limiter storage moved to `RATELIMIT_STORAGE_URI` config (was hardcoded `memory://` per-worker) | extensions.py, config.py |
| 2026-06 | TOTP verify: added hourly rate limit (`30/hr`) + per-session attempt counter (locks out after 5 failures) | auth.py |
| 2026-06 | TOTP setup endpoint: added `@limiter.limit('10 per minute')` | auth.py |
| 2026-06 | `refresh_token_expires_at` now reset on every token exchange, not only when Schwab rotates the token | schwab_service.py |
| 2026-06 | Schwab unknown account type fallback changed back to `'other'` (was incorrectly changed to `'investment'`) | schwab.py |
| 2026-06 | `teller_enrollments.access_token` widened VARCHAR(128) → TEXT to fit Fernet-encrypted values | add_security_columns.py |
| 2026-06 | New income transaction submitted as expense — `form.transaction_type.data` not set on GET | transactions.py |
--- ---
## 19. To-Do / Roadmap ## 19. To-Do / Roadmap
### High priority ### High priority
- [ ] **Mobile responsiveness pass** — sidebar auto-collapses on mobile; tables scroll horizontally - [ ] **Mobile responsiveness pass** — sidebar auto-collapses on mobile; tables scroll horizontally *(base.html mobile CSS improved; further work needed)*
- [ ] **Budget alerts** — email/Twilio SMS when category spending hits 80% / 100% - [ ] **Budget alerts** — email/Twilio SMS when category spending hits 80% / 100%
### Medium priority ### Medium priority
- [ ] **Pagination info** — show "Page X of Y" on AI history and other paginated pages
- [ ] **PDF export memory** — stream CSV/Excel exports for users with large transaction history - [ ] **PDF export memory** — stream CSV/Excel exports for users with large transaction history
- [ ] **Receipt MIME validation** — validate file magic bytes server-side, not just extension - [ ] **Receipt MIME validation** — validate file magic bytes server-side, not just extension
- [ ] **OCR ownership check** — verify re-extracted filename belongs to current user's transaction - [ ] **OCR ownership check** — verify re-extracted filename belongs to current user's transaction
- [ ] **Bank import progress** — show per-row import progress for large statement files - [ ] **Bank import progress** — show per-row import progress for large statement files
- [ ] **Schwab IRA account type** — map Schwab IRA account type to `investment` in ACCOUNT_TYPE_MAP
### Low priority / future ### Low priority / future
- [ ] iOS companion app - [ ] iOS companion app
- [ ] Shared household mode (2 users, row-level isolation) - [ ] Shared household mode (2 users, row-level isolation)
- [ ] Bank statement PDF: table-extraction fallback (pdfplumber tables API) before Groq call - [ ] Bank statement PDF: table-extraction fallback (pdfplumber tables API) before Groq call
- [ ] Investment price history chart per holding
- [ ] Dark mode toggle - [ ] Dark mode toggle
- [ ] Schwab auto-sync on schedule (currently manual only)
### Completed (removed from backlog)
- [x] **Pagination info** — "Page X of Y" added to transactions, AI history, accounts/payments, audit log
- [x] **Schwab IRA account type** — IRA/ROTH_IRA/401K/BROKERAGE types added to ACCOUNT_TYPE_MAP
- [x] **Investment price history chart** — 1W/1M/3M/6M/1Y chart on investment detail page
- [x] **Schwab auto-sync on schedule** — `scripts/sync_schwab.py` (cron at 7AM daily)
app/routes/logs.py
+19 -2
View File
@@ -49,13 +49,26 @@ def _parse_line(raw):
@logs_bp.route('/') @logs_bp.route('/')
@login_required @login_required
def index(): def index():
import datetime
log_file = current_app.config.get('LOG_FILE_PATH', '') log_file = current_app.config.get('LOG_FILE_PATH', '')
file_exists = bool(log_file) and os.path.isfile(log_file) file_exists = bool(log_file) and os.path.isfile(log_file)
file_size = os.path.getsize(log_file) if file_exists else 0 file_size = os.path.getsize(log_file) if file_exists else 0
file_mtime = None
if file_exists:
ts = os.path.getmtime(log_file)
file_mtime = datetime.datetime.fromtimestamp(ts).strftime('%b %d, %H:%M')
def fmt_size(b):
if b < 1024: return f'{b} B'
if b < 1024**2: return f'{b/1024:.1f} KB'
return f'{b/1024**2:.1f} MB'
return render_template('logs/index.html', return render_template('logs/index.html',
log_file=log_file, log_file=log_file,
file_exists=file_exists, file_exists=file_exists,
file_size=file_size) file_size=file_size,
file_size_fmt=fmt_size(file_size),
file_mtime=file_mtime)
@logs_bp.route('/api') @logs_bp.route('/api')
@@ -87,17 +100,21 @@ def api():
# Most-recent first, capped at limit # Most-recent first, capped at limit
entries = list(reversed(entries))[:limit] entries = list(reversed(entries))[:limit]
# Count per level across ALL unfiltered lines (for the stats bar) # Count per level + unique modules across ALL unfiltered lines
all_entries = [_parse_line(l) for l in raw_lines if l.strip()] all_entries = [_parse_line(l) for l in raw_lines if l.strip()]
counts = {} counts = {}
modules_seen = set()
for e in all_entries: for e in all_entries:
counts[e['level']] = counts.get(e['level'], 0) + 1 counts[e['level']] = counts.get(e['level'], 0) + 1
if e['name']:
modules_seen.add(e['name'])
return jsonify({ return jsonify({
'entries': entries, 'entries': entries,
'counts': counts, 'counts': counts,
'total_raw': len(raw_lines), 'total_raw': len(raw_lines),
'log_file': log_file, 'log_file': log_file,
'modules': sorted(modules_seen),
}) })
app/templates/base.html
+1 -7
View File
@@ -303,15 +303,9 @@
> >
<i class="bi bi-tags"></i><span class="lt">Categories</span> <i class="bi bi-tags"></i><span class="lt">Categories</span>
</a> </a>
<a
href="{{ url_for('logs.index') }}"
class="sb-link {% if request.blueprint == 'logs' %}active{% endif %}"
>
<i class="bi bi-terminal"></i><span class="lt">System Logs</span>
</a>
<a <a
href="{{ url_for('settings.index') }}" href="{{ url_for('settings.index') }}"
class="sb-link {% if request.blueprint == 'settings' %}active{% endif %}" class="sb-link {% if request.blueprint in ('settings', 'logs') %}active{% endif %}"
> >
<i class="bi bi-gear"></i><span class="lt">Settings</span> <i class="bi bi-gear"></i><span class="lt">Settings</span>
</a> </a>
app/templates/logs/index.html
+392 -162
View File
@@ -4,151 +4,254 @@
{% block extra_css %} {% block extra_css %}
<style> <style>
.log-toolbar { /* ── Layout ─────────────────────────────────────────────────────────────────── */
.log-header {
display: flex; align-items: center; flex-wrap: wrap; gap: 10px;
margin-bottom: 16px;
}
.log-meta {
display: flex; flex-wrap: wrap; gap: 6px; align-items: center;
}
.log-meta-chip {
display: inline-flex; align-items: center; gap: 4px;
font-size: 11px; color: var(--muted); font-family: 'DM Mono', monospace;
background: #f1f5f9; border-radius: 4px; padding: 2px 8px;
}
.log-meta-chip i { font-size: 10px; }
/* ── Level pills / filter bar ────────────────────────────────────────────────── */
.level-pills {
display: flex; flex-wrap: wrap; gap: 6px; margin-bottom: 14px;
}
.lvl-pill {
display: inline-flex; align-items: center; gap: 5px;
padding: 4px 12px; border-radius: 20px; font-size: 12px; font-weight: 600;
cursor: pointer; border: 1.5px solid transparent; transition: all .15s;
user-select: none;
}
.lvl-pill .cnt { font-size: 11px; opacity: .8; }
.lvl-pill:not(.active) { opacity: .6; }
.lvl-pill:hover { opacity: 1; }
.lvl-pill.active { border-color: currentColor; opacity: 1; }
.lp-ALL { background:#f1f5f9; color:#475569; }
.lp-ERROR { background:#fee2e2; color:#991b1b; }
.lp-CRITICAL { background:#fce7f3; color:#9d174d; }
.lp-WARNING { background:#fef9c3; color:#854d0e; }
.lp-INFO { background:#dbeafe; color:#1e40af; }
.lp-DEBUG { background:#f0fdf4; color:#166534; }
/* ── Toolbar ────────────────────────────────────────────────────────────────── */
.log-toolbar {
display: flex; flex-wrap: wrap; gap: 8px; align-items: center; display: flex; flex-wrap: wrap; gap: 8px; align-items: center;
background: var(--card-bg); border: 1px solid var(--border); background: var(--card-bg); border: 1px solid var(--border);
border-radius: 12px; padding: 14px 16px; margin-bottom: 16px; border-radius: 10px; padding: 10px 14px; margin-bottom: 14px;
} }
.log-toolbar .sep { flex: 1; } .toolbar-sep { flex: 1; min-width: 8px; }
.level-badge { /* ── Auto-refresh indicator ──────────────────────────────────────────────────── */
display: inline-block; font-size: 11px; font-weight: 600; .ar-wrap {
padding: 2px 7px; border-radius: 4px; font-family: 'DM Mono', monospace; display: flex; align-items: center; gap: 6px;
white-space: nowrap; font-size: 12px; color: var(--muted); cursor: pointer;
} padding: 4px 8px; border-radius: 6px; transition: background .15s;
.level-INFO { background: #dbeafe; color: #1e40af; } user-select: none;
.level-WARNING { background: #fef9c3; color: #854d0e; } }
.level-ERROR { background: #fee2e2; color: #991b1b; } .ar-wrap:hover { background: #f1f5f9; }
.level-CRITICAL { background: #fce7f3; color: #9d174d; } .ar-dot {
.level-DEBUG { background: #f0fdf4; color: #166534; } width: 8px; height: 8px; border-radius: 50%; background: #cbd5e1;
.level-RAW { background: #f1f5f9; color: #475569; } flex-shrink: 0; transition: background .3s;
}
.ar-dot.on { background: #10b981; animation: pulse-dot 2s infinite; }
@keyframes pulse-dot { 0%,100%{opacity:1} 50%{opacity:.35} }
.ar-countdown { font-family:'DM Mono',monospace; font-size:11px; min-width:16px; }
.stat-pill { /* ── Log table ───────────────────────────────────────────────────────────────── */
display: inline-flex; align-items: center; gap: 5px; #log-wrap {
padding: 3px 10px; border-radius: 20px; font-size: 12px;
font-weight: 600; cursor: pointer; border: 1.5px solid transparent;
transition: all .15s;
}
.stat-pill:hover, .stat-pill.active { border-color: currentColor; opacity: 1; }
.stat-pill { opacity: .75; }
.pill-ALL { background: #f1f5f9; color: #475569; }
.pill-INFO { background: #dbeafe; color: #1e40af; }
.pill-WARNING { background: #fef9c3; color: #854d0e; }
.pill-ERROR { background: #fee2e2; color: #991b1b; }
.pill-CRITICAL { background: #fce7f3; color: #9d174d; }
.pill-DEBUG { background: #f0fdf4; color: #166534; }
#log-table-wrap {
background: var(--card-bg); border: 1px solid var(--border); background: var(--card-bg); border: 1px solid var(--border);
border-radius: 12px; overflow: hidden; border-radius: 12px; overflow: hidden;
} }
#log-table { width: 100%; border-collapse: collapse; font-size: 12.5px; } #log-table { width: 100%; border-collapse: collapse; font-size: 12.5px; }
#log-table thead th { #log-table thead th {
font-size: 10px; font-weight: 600; text-transform: uppercase; font-size: 10px; font-weight: 700; text-transform: uppercase;
letter-spacing: .07em; color: var(--muted); padding: 10px 14px; letter-spacing: .07em; color: var(--muted); padding: 9px 14px;
border-bottom: 1px solid var(--border); white-space: nowrap; border-bottom: 1px solid var(--border); background: #f8fafc;
background: #f8fafc; white-space: nowrap;
} }
#log-table tbody td { #log-table tbody tr { cursor: pointer; transition: background .1s; }
padding: 7px 14px; border-bottom: 1px solid var(--border); #log-table tbody tr:hover > td { background: #f1f5f9 !important; }
vertical-align: top; word-break: break-word; #log-table tbody td {
} padding: 6px 14px; border-bottom: 1px solid #f1f5f9;
#log-table tbody tr:last-child td { border-bottom: none; } vertical-align: top;
#log-table tbody tr:hover { background: #f8fafc; } }
#log-table tbody tr.row-ERROR td { background: #fff5f5; } #log-table tbody tr:last-child > td { border-bottom: none; }
#log-table tbody tr.row-CRITICAL td { background: #fdf2f8; } tr.row-ERROR > td { background: #fff8f8; }
#log-table tbody tr.row-WARNING td { background: #fffbeb; } tr.row-CRITICAL > td { background: #fdf4fb; }
tr.row-WARNING > td { background: #fffdf0; }
tr.row-RAW > td { background: #fafafa; }
.ts-col { white-space: nowrap; color: var(--muted); font-family: 'DM Mono', monospace; font-size: 11px; width: 148px; } /* ── Column widths ───────────────────────────────────────────────────────────── */
.name-col { color: var(--muted); font-family: 'DM Mono', monospace; font-size: 11px; width: 220px; } .col-ts { width: 145px; white-space: nowrap; }
.msg-col { font-family: 'DM Mono', monospace; } .col-lvl { width: 86px; }
.col-mod { width: 160px; }
.col-msg { }
#empty-state { .ts-text { font-family:'DM Mono',monospace; font-size:11px; color:var(--muted); }
text-align: center; padding: 60px 20px; color: var(--muted); .mod-text { font-family:'DM Mono',monospace; font-size:11px; color:#64748b;
} max-width:150px; overflow:hidden; text-overflow:ellipsis; white-space:nowrap; }
.msg-text { font-family:'DM Mono',monospace; font-size:12px; word-break:break-word; }
.msg-text.nowrap { white-space:nowrap; overflow:hidden; text-overflow:ellipsis; max-width:580px; }
.auto-refresh-dot { /* ── Level badge ─────────────────────────────────────────────────────────────── */
width: 8px; height: 8px; border-radius: 50%; .lvl-badge {
background: #94a3b8; display: inline-block; display: inline-block; font-size: 10px; font-weight: 700;
transition: background .3s; padding: 1px 6px; border-radius: 4px; font-family:'DM Mono',monospace;
} letter-spacing: .03em; white-space: nowrap;
.auto-refresh-dot.on { background: #10b981; animation: pulse-dot 2s infinite; } }
@keyframes pulse-dot { .lb-INFO { background:#dbeafe; color:#1e40af; }
0%, 100% { opacity: 1; } 50% { opacity: .4; } .lb-WARNING { background:#fef9c3; color:#854d0e; }
} .lb-ERROR { background:#fee2e2; color:#991b1b; }
.lb-CRITICAL { background:#fce7f3; color:#9d174d; }
.lb-DEBUG { background:#f0fdf4; color:#166534; }
.lb-RAW { background:#f1f5f9; color:#64748b; }
#spinner { display: none; } /* ── Expanded row ────────────────────────────────────────────────────────────── */
#spinner.on { display: inline-block; } tr.expanded-row > td {
background: #0f172a !important; padding: 0;
}
.expand-panel {
padding: 12px 16px; font-family:'DM Mono',monospace; font-size: 12px;
color: #e2e8f0; white-space: pre-wrap; word-break: break-all;
line-height: 1.6;
}
.expand-panel .ep-label {
font-size: 10px; font-weight: 700; text-transform: uppercase;
letter-spacing: .08em; color: #64748b; display: block; margin-bottom: 4px;
}
.expand-panel .ep-val { color: #e2e8f0; }
.expand-panel .ep-mod { color: #7dd3fc; }
.expand-panel .ep-ts { color: #a3e635; }
.expand-panel .ep-err { color: #fca5a5; }
.copy-btn {
font-size:11px; padding:2px 8px; border-radius:4px;
background:#1e293b; color:#94a3b8; border:1px solid #334155;
cursor:pointer; transition:all .15s; margin-top:8px;
}
.copy-btn:hover { background:#334155; color:#e2e8f0; }
/* ── Empty / loading states ──────────────────────────────────────────────────── */
.log-empty { text-align:center; padding:56px 20px; color:var(--muted); }
.log-empty i { font-size:2.5rem; opacity:.3; display:block; margin-bottom:10px; }
/* ── Spinner ─────────────────────────────────────────────────────────────────── */
#spinner { display:none; }
#spinner.on { display:inline-block; }
/* ── Wrap toggle ─────────────────────────────────────────────────────────────── */
.wrap-btn.active { background:#e0f2fe; color:#0369a1; border-color:#7dd3fc; }
/* ── Footer bar ──────────────────────────────────────────────────────────────── */
.log-footer {
display:flex; justify-content:space-between; align-items:center;
font-size:11px; color:var(--muted); padding:6px 2px; margin-top:8px;
}
</style> </style>
{% endblock %} {% endblock %}
{% block topbar_actions %}
<a href="{{ url_for('logs.download') }}" class="btn btn-sm btn-outline-secondary" title="Download log file">
<i class="bi bi-download"></i>
</a>
{% endblock %}
{% block content %} {% block content %}
<div class="d-flex align-items-center gap-2 mb-3"> <!-- Header -->
<h5 class="mb-0 fw-semibold">Application Logs</h5> <div class="log-header">
<span id="spinner" class="spinner-border spinner-border-sm text-secondary ms-1"></span> <div>
<span class="small text-muted ms-auto mono" id="log-file-path"> <a href="{{ url_for('settings.index') }}" class="btn btn-sm btn-outline-secondary me-2" style="font-size:12px;">
{% if file_exists %}{{ log_file }}{% else %}Log file not found{% endif %} <i class="bi bi-arrow-left me-1"></i>Settings
</a>
</div>
<div class="log-meta">
{% if file_exists %}
<span class="log-meta-chip"><i class="bi bi-hdd"></i>{{ file_size_fmt }}</span>
<span class="log-meta-chip"><i class="bi bi-clock"></i>{{ file_mtime }}</span>
<span class="log-meta-chip" style="max-width:360px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;" title="{{ log_file }}">
<i class="bi bi-file-text"></i>{{ log_file }}
</span> </span>
{% else %}
<span class="log-meta-chip" style="color:#ef4444;"><i class="bi bi-exclamation-triangle"></i>Log file not found</span>
{% endif %}
</div>
<div class="ms-auto">
<span id="spinner" class="spinner-border spinner-border-sm text-secondary"></span>
</div>
</div> </div>
<!-- Stats bar --> <!-- Level filter pills -->
<div class="d-flex flex-wrap gap-2 mb-3" id="stats-bar"> <div class="level-pills" id="level-pills">
<span class="stat-pill pill-ALL active" data-level="ALL">All <span class="ms-1" id="cnt-ALL">—</span></span> <span class="lvl-pill lp-ALL active" data-level="ALL">All <span class="cnt" id="cnt-ALL">—</span></span>
<span class="stat-pill pill-ERROR" data-level="ERROR">Error <span class="ms-1" id="cnt-ERROR">0</span></span> <span class="lvl-pill lp-ERROR" data-level="ERROR">Error <span class="cnt" id="cnt-ERROR">0</span></span>
<span class="stat-pill pill-WARNING" data-level="WARNING">Warning <span class="ms-1" id="cnt-WARNING">0</span></span> <span class="lvl-pill lp-CRITICAL" data-level="CRITICAL">Critical <span class="cnt" id="cnt-CRITICAL">0</span></span>
<span class="stat-pill pill-INFO" data-level="INFO">Info <span class="ms-1" id="cnt-INFO">0</span></span> <span class="lvl-pill lp-WARNING" data-level="WARNING">Warning <span class="cnt" id="cnt-WARNING">0</span></span>
<span class="stat-pill pill-DEBUG" data-level="DEBUG">Debug <span class="ms-1" id="cnt-DEBUG">0</span></span> <span class="lvl-pill lp-INFO" data-level="INFO">Info <span class="cnt" id="cnt-INFO">0</span></span>
<span class="lvl-pill lp-DEBUG" data-level="DEBUG">Debug <span class="cnt" id="cnt-DEBUG">0</span></span>
</div> </div>
<!-- Toolbar --> <!-- Toolbar -->
<div class="log-toolbar"> <div class="log-toolbar">
<div class="input-group input-group-sm" style="max-width:240px;">
<span class="input-group-text" style="background:#f8fafc;"><i class="bi bi-search" style="font-size:11px;"></i></span>
<input id="search-input" type="text" class="form-control form-control-sm" <input id="search-input" type="text" class="form-control form-control-sm"
placeholder="Search message or module…" style="max-width:240px;"> placeholder="Search message…" autocomplete="off">
<input id="module-input" type="text" class="form-control form-control-sm" </div>
placeholder="Module filter (e.g. teller)" style="max-width:180px;">
<select id="module-select" class="form-select form-select-sm" style="max-width:180px;">
<option value="">All modules</option>
</select>
<select id="limit-select" class="form-select form-select-sm" style="max-width:110px;"> <select id="limit-select" class="form-select form-select-sm" style="max-width:110px;">
<option value="100">Last 100</option> <option value="100">Last 100</option>
<option value="200" selected>Last 200</option> <option value="200" selected>Last 200</option>
<option value="500">Last 500</option> <option value="500">Last 500</option>
<option value="1000">Last 1000</option> <option value="1000">Last 1 000</option>
<option value="2000">Last 2 000</option>
</select> </select>
<div class="sep"></div>
<label class="d-flex align-items-center gap-2 small text-muted mb-0" style="cursor:pointer;"> <button class="btn btn-sm btn-outline-secondary wrap-btn" id="wrap-btn" title="Toggle message wrapping">
<span class="auto-refresh-dot" id="ar-dot"></span> <i class="bi bi-text-wrap"></i>
<input type="checkbox" id="auto-refresh" class="d-none"> Auto-refresh </button>
</label>
<div class="toolbar-sep"></div>
<div class="ar-wrap" id="ar-toggle" title="Toggle auto-refresh every 5 seconds">
<span class="ar-dot" id="ar-dot"></span>
<span>Auto-refresh</span>
<span class="ar-countdown" id="ar-countdown"></span>
</div>
<a href="{{ url_for('logs.download') }}" class="btn btn-sm btn-outline-secondary" title="Download full log file">
<i class="bi bi-download me-1"></i><span class="d-none d-sm-inline">Download</span>
</a>
<button class="btn btn-sm btn-outline-danger" id="clear-btn" title="Clear log file"> <button class="btn btn-sm btn-outline-danger" id="clear-btn" title="Clear log file">
<i class="bi bi-trash"></i> Clear <i class="bi bi-trash me-1"></i><span class="d-none d-sm-inline">Clear</span>
</button> </button>
</div> </div>
<!-- Log table --> <!-- Log table -->
<div id="log-table-wrap"> <div id="log-wrap">
<table id="log-table"> <table id="log-table">
<thead> <thead>
<tr> <tr>
<th class="ts-col">Timestamp</th> <th class="col-ts">Timestamp</th>
<th style="width:90px">Level</th> <th class="col-lvl">Level</th>
<th class="name-col">Module</th> <th class="col-mod d-mob-none">Module</th>
<th>Message</th> <th class="col-msg">Message</th>
</tr> </tr>
</thead> </thead>
<tbody id="log-body"> <tbody id="log-body">
<tr><td colspan="4" id="empty-state"> <tr><td colspan="4"><div class="log-empty">
<i class="bi bi-hourglass-split fs-3 d-block mb-2 opacity-25"></i> <i class="bi bi-hourglass-split"></i>Loading…
Loading… </div></td></tr>
</td></tr>
</tbody> </tbody>
</table> </table>
</div> </div>
<div class="d-flex justify-content-between align-items-center mt-2 small text-muted px-1"> <!-- Footer -->
<div class="log-footer">
<span id="result-count"></span> <span id="result-count"></span>
<span id="last-refresh"></span> <span id="last-refresh"></span>
</div> </div>
@@ -157,103 +260,217 @@
{% block extra_js %} {% block extra_js %}
<script> <script>
(function () { (function () {
const CSRF = document.querySelector('meta[name="csrf-token"]')?.content || ''; 'use strict';
let activeLevel = 'ALL'; const CSRF = document.querySelector('meta[name="csrf-token"]')?.content || '';
let arTimer = null;
// ── fetch & render ──────────────────────────────────────────────────────── let activeLevel = 'ALL';
function load() { let wrapMessages = false;
let arEnabled = false;
let arTimer = null;
let arCountdown = 0;
let arTick = null;
let allModules = [];
let expandedRows = new Set(); // track which row indices are expanded
// ── Fetch & render ──────────────────────────────────────────────────────────
function load() {
const search = document.getElementById('search-input').value.trim(); const search = document.getElementById('search-input').value.trim();
const module = document.getElementById('module-input').value.trim(); const module = document.getElementById('module-select').value;
const limit = document.getElementById('limit-select').value; const limit = document.getElementById('limit-select').value;
const spinner = document.getElementById('spinner'); document.getElementById('spinner').classList.add('on');
spinner.classList.add('on');
const params = new URLSearchParams({ level: activeLevel, search, module, limit }); const params = new URLSearchParams({ level: activeLevel, search, module, limit });
fetch(`{{ url_for('logs.api') }}?${params}`) fetch(`{{ url_for('logs.api') }}?${params}`)
.then(r => r.json()) .then(r => r.json())
.then(data => { .then(data => {
spinner.classList.remove('on'); document.getElementById('spinner').classList.remove('on');
expandedRows.clear();
renderTable(data.entries || []); renderTable(data.entries || []);
renderCounts(data.counts || {}); renderCounts(data.counts || {});
populateModules(data.modules || []);
document.getElementById('last-refresh').textContent = document.getElementById('last-refresh').textContent =
'Updated ' + new Date().toLocaleTimeString(); 'Refreshed ' + new Date().toLocaleTimeString();
const shown = (data.entries || []).length;
const total = data.total_raw || 0;
document.getElementById('result-count').textContent = document.getElementById('result-count').textContent =
`Showing ${(data.entries||[]).length} of ${data.total_raw || 0} lines`; shown + ' entr' + (shown === 1 ? 'y' : 'ies') +
(total ? ' · ' + total + ' lines in file' : '');
}) })
.catch(() => spinner.classList.remove('on')); .catch(() => document.getElementById('spinner').classList.remove('on'));
} }
function renderTable(entries) { // ── Table rendering ─────────────────────────────────────────────────────────
function esc(t) {
return (t || '').replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;');
}
function lvlBadge(lvl) {
const cls = { INFO:'lb-INFO', WARNING:'lb-WARNING', ERROR:'lb-ERROR',
CRITICAL:'lb-CRITICAL', DEBUG:'lb-DEBUG', RAW:'lb-RAW' }[lvl] || 'lb-RAW';
return `<span class="lvl-badge ${cls}">${esc(lvl)}</span>`;
}
function shortMod(name) {
// Show last two segments: app.services.teller_service → services.teller_service
const parts = (name || '').split('.');
return parts.length > 2 ? parts.slice(-2).join('.') : name;
}
function renderTable(entries) {
const tbody = document.getElementById('log-body'); const tbody = document.getElementById('log-body');
if (!entries.length) { if (!entries.length) {
tbody.innerHTML = `<tr><td colspan="4" id="empty-state"> tbody.innerHTML = `<tr><td colspan="4"><div class="log-empty">
<i class="bi bi-inbox fs-3 d-block mb-2 opacity-25"></i> <i class="bi bi-inbox"></i>No entries match your filters.
No log entries match your filters. </div></td></tr>`;
</td></tr>`;
return; return;
} }
const rows = entries.map(e => { const wrapCls = wrapMessages ? '' : ' nowrap';
const rows = entries.map((e, i) => {
const lvl = e.level || 'RAW'; const lvl = e.level || 'RAW';
const safe = t => (t||'').replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;'); return `<tr class="row-${esc(lvl)}" data-idx="${i}" data-ts="${esc(e.ts)}" data-lvl="${esc(lvl)}" data-name="${esc(e.name)}" data-msg="${esc(e.message).replace(/"/g,'&quot;')}">
return `<tr class="row-${lvl}"> <td class="col-ts"><span class="ts-text">${esc(e.ts)}</span></td>
<td class="ts-col">${safe(e.ts)}</td> <td class="col-lvl">${lvlBadge(lvl)}</td>
<td><span class="level-badge level-${lvl}">${lvl}</span></td> <td class="col-mod d-mob-none"><span class="mod-text" title="${esc(e.name)}">${esc(shortMod(e.name))}</span></td>
<td class="name-col" title="${safe(e.name)}">${safe(e.name.split('.').pop())}</td> <td class="col-msg"><span class="msg-text${wrapCls}">${esc(e.message)}</span></td>
<td class="msg-col">${safe(e.message)}</td>
</tr>`; </tr>`;
}); });
tbody.innerHTML = rows.join(''); tbody.innerHTML = rows.join('');
}
// ── Expand row on click ─────────────────────────────────────────────────────
document.getElementById('log-body').addEventListener('click', function(e) {
const row = e.target.closest('tr[data-idx]');
if (!row) return;
// If clicking copy button inside an expanded panel, let it work
if (e.target.closest('.copy-btn')) return;
const idx = row.dataset.idx;
const next = row.nextElementSibling;
// Collapse if already expanded
if (next && next.classList.contains('expanded-row')) {
next.remove();
row.classList.remove('_expanded');
return;
} }
function renderCounts(counts) { // Collapse any other open row first
const total = Object.values(counts).reduce((s,v)=>s+v, 0); document.querySelectorAll('.expanded-row').forEach(r => r.remove());
document.getElementById('cnt-ALL').textContent = total; document.querySelectorAll('._expanded').forEach(r => r.classList.remove('_expanded'));
// Build expanded panel
const ts = row.dataset.ts;
const lvl = row.dataset.lvl;
const name = row.dataset.name;
const msg = row.dataset.msg.replace(/&quot;/g, '"');
const raw = `${ts}|${lvl}|${name}|${msg}`;
const panel = document.createElement('tr');
panel.className = 'expanded-row';
panel.innerHTML = `<td colspan="4">
<div class="expand-panel">
<span class="ep-label">Timestamp</span><span class="ep-ts ep-val">${esc(ts)}</span>
<span class="ep-label" style="margin-top:8px;">Module</span><span class="ep-mod ep-val">${esc(name) || '—'}</span>
<span class="ep-label" style="margin-top:8px;">Message</span><span class="${lvl === 'ERROR' || lvl === 'CRITICAL' ? 'ep-err' : ''} ep-val">${esc(msg)}</span>
<button class="copy-btn" onclick="copyRaw(this, ${JSON.stringify(raw)})">
<i class="bi bi-clipboard me-1"></i>Copy raw line
</button>
</div>
</td>`;
row.after(panel);
row.classList.add('_expanded');
});
window.copyRaw = function(btn, text) {
navigator.clipboard.writeText(text).then(() => {
btn.innerHTML = '<i class="bi bi-check2 me-1"></i>Copied!';
setTimeout(() => { btn.innerHTML = '<i class="bi bi-clipboard me-1"></i>Copy raw line'; }, 1500);
});
};
// ── Counts & module dropdown ────────────────────────────────────────────────
function renderCounts(counts) {
const total = Object.values(counts).reduce((s, v) => s + v, 0);
document.getElementById('cnt-ALL').textContent = total || 0;
document.getElementById('cnt-ERROR').textContent = counts['ERROR'] || 0; document.getElementById('cnt-ERROR').textContent = counts['ERROR'] || 0;
document.getElementById('cnt-CRITICAL').textContent = counts['CRITICAL'] || 0;
document.getElementById('cnt-WARNING').textContent = counts['WARNING'] || 0; document.getElementById('cnt-WARNING').textContent = counts['WARNING'] || 0;
document.getElementById('cnt-INFO').textContent = counts['INFO'] || 0; document.getElementById('cnt-INFO').textContent = counts['INFO'] || 0;
document.getElementById('cnt-DEBUG').textContent = counts['DEBUG'] || 0; document.getElementById('cnt-DEBUG').textContent = counts['DEBUG'] || 0;
} }
// ── level pills ─────────────────────────────────────────────────────────── function populateModules(modules) {
document.getElementById('stats-bar').addEventListener('click', function (e) { if (JSON.stringify(modules) === JSON.stringify(allModules)) return;
const pill = e.target.closest('.stat-pill'); allModules = modules;
const sel = document.getElementById('module-select');
const cur = sel.value;
sel.innerHTML = '<option value="">All modules</option>' +
modules.map(m => `<option value="${esc(m)}"${m === cur ? ' selected' : ''}>${esc(m)}</option>`).join('');
}
// ── Level pill clicks ───────────────────────────────────────────────────────
document.getElementById('level-pills').addEventListener('click', function(e) {
const pill = e.target.closest('.lvl-pill');
if (!pill) return; if (!pill) return;
document.querySelectorAll('.stat-pill').forEach(p => p.classList.remove('active')); document.querySelectorAll('.lvl-pill').forEach(p => p.classList.remove('active'));
pill.classList.add('active'); pill.classList.add('active');
activeLevel = pill.dataset.level; activeLevel = pill.dataset.level;
load(); load();
}); });
// ── search / filter inputs ──────────────────────────────────────────────── // ── Search / filter / limit ─────────────────────────────────────────────────
let debounce; let debounce;
['search-input','module-input','limit-select'].forEach(id => { ['search-input', 'module-select', 'limit-select'].forEach(id => {
document.getElementById(id).addEventListener('input', function () { const el = document.getElementById(id);
const ev = id === 'limit-select' || id === 'module-select' ? 'change' : 'input';
el.addEventListener(ev, () => {
clearTimeout(debounce); clearTimeout(debounce);
debounce = setTimeout(load, 300); debounce = setTimeout(load, 280);
}); });
});
// ── Wrap toggle ─────────────────────────────────────────────────────────────
document.getElementById('wrap-btn').addEventListener('click', function() {
wrapMessages = !wrapMessages;
this.classList.toggle('active', wrapMessages);
document.querySelectorAll('.msg-text').forEach(el => {
el.classList.toggle('nowrap', !wrapMessages);
}); });
});
// ── auto-refresh ────────────────────────────────────────────────────────── // ── Auto-refresh ────────────────────────────────────────────────────────────
const arCheckbox = document.getElementById('auto-refresh'); const AR_INTERVAL = 5;
const arDot = document.getElementById('ar-dot');
document.querySelector('label[for="auto-refresh"]') ||
document.querySelector('label').addEventListener; // noop
// toggle by clicking the label area function startAR() {
document.querySelector('.auto-refresh-dot').parentElement.addEventListener('click', function () { arEnabled = true;
arCheckbox.checked = !arCheckbox.checked; document.getElementById('ar-dot').classList.add('on');
arDot.classList.toggle('on', arCheckbox.checked); arCountdown = AR_INTERVAL;
if (arCheckbox.checked) { document.getElementById('ar-countdown').textContent = arCountdown + 's';
arTimer = setInterval(load, 5000); arTick = setInterval(() => {
} else { arCountdown--;
clearInterval(arTimer); if (arCountdown <= 0) {
arCountdown = AR_INTERVAL;
load();
} }
}); document.getElementById('ar-countdown').textContent = arCountdown + 's';
}, 1000);
}
// ── clear ───────────────────────────────────────────────────────────────── function stopAR() {
document.getElementById('clear-btn').addEventListener('click', function () { arEnabled = false;
if (!confirm('Clear all log entries from the file? This cannot be undone.')) return; clearInterval(arTick);
document.getElementById('ar-dot').classList.remove('on');
document.getElementById('ar-countdown').textContent = '';
}
document.getElementById('ar-toggle').addEventListener('click', () => {
arEnabled ? stopAR() : startAR();
});
// ── Clear ───────────────────────────────────────────────────────────────────
document.getElementById('clear-btn').addEventListener('click', function() {
if (!confirm('Clear all log entries from the file?\nThis cannot be undone.')) return;
fetch('{{ url_for("logs.clear") }}', { fetch('{{ url_for("logs.clear") }}', {
method: 'POST', method: 'POST',
headers: { 'X-CSRFToken': CSRF, 'Content-Type': 'application/json' }, headers: { 'X-CSRFToken': CSRF, 'Content-Type': 'application/json' },
@@ -263,10 +480,23 @@
if (d.status === 'ok') load(); if (d.status === 'ok') load();
else alert('Clear failed: ' + d.error); else alert('Clear failed: ' + d.error);
}); });
}); });
// ── init ────────────────────────────────────────────────────────────────── // ── Keyboard shortcuts ──────────────────────────────────────────────────────
load(); document.addEventListener('keydown', function(e) {
if (e.target.tagName === 'INPUT' || e.target.tagName === 'SELECT') return;
if (e.key === 'r' || e.key === 'R') load();
if (e.key === 'a' || e.key === 'A') {
arEnabled ? stopAR() : startAR();
}
if (e.key === 'Escape') {
document.querySelectorAll('.expanded-row').forEach(r => r.remove());
document.querySelectorAll('._expanded').forEach(r => r.classList.remove('_expanded'));
}
});
// ── Init ────────────────────────────────────────────────────────────────────
load();
})(); })();
</script> </script>
{% endblock %} {% endblock %}
app/templates/settings/index.html
+10 -1
View File
@@ -61,7 +61,7 @@
</div> </div>
</div> </div>
<!-- Audit Log nav card --> <!-- Audit Log + System Logs nav cards -->
<div class="row g-3 mt-0"> <div class="row g-3 mt-0">
<div class="col-12 col-sm-6 col-lg-3"> <div class="col-12 col-sm-6 col-lg-3">
<a href="{{ url_for('settings.audit_log') }}" class="text-decoration-none"> <a href="{{ url_for('settings.audit_log') }}" class="text-decoration-none">
@@ -72,6 +72,15 @@
</div> </div>
</a> </a>
</div> </div>
<div class="col-12 col-sm-6 col-lg-3">
<a href="{{ url_for('logs.index') }}" class="text-decoration-none">
<div class="pcard text-center py-4" style="transition:all .15s;" onmouseover="this.style.borderColor='#0f172a'" onmouseout="this.style.borderColor='var(--border)'">
<i class="bi bi-terminal" style="font-size:2rem;color:#0f172a;"></i>
<div style="font-size:14px;font-weight:600;margin-top:10px;">System Logs</div>
<div style="font-size:12px;color:var(--muted);margin-top:4px;">Application log viewer</div>
</div>
</a>
</div>
</div> </div>
<!-- Two-Factor Authentication --> <!-- Two-Factor Authentication -->