diff --git a/CLAUDE.md b/CLAUDE.md index 2d6eabc..5c0162b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -108,6 +108,7 @@ Without this, sessions and CSRF tokens are broken across the 4 Gunicorn workers. | `templates/login.html` | Standalone login page | Does NOT extend `base.html`; has its own `` and no `app.js`; CSRF token must be a direct hidden input | | `templates/forgot_password.html` | Standalone forgot-password page | Same constraints as `login.html` — standalone, direct CSRF hidden input, no `app.js` | | `templates/reset_password.html` | Standalone reset-password page | Same constraints as `login.html` — standalone, direct CSRF hidden input, no `app.js` | +| `templates/profile.html` | User profile editor | Extends `base.html`; users can update `full_name` and `email`; username/role are read-only | --- @@ -121,6 +122,8 @@ Without this, sessions and CSRF tokens are broken across the 4 Gunicorn workers. - `@admin_required` — redirects to login or 403 if not admin - **Session fixation prevention** — `session.clear()` is called in `auth.login` immediately before setting `session["user"]` on successful authentication - **Keep-alive endpoint** — `GET /ping` (`routes/auth.py`, `@login_required`) touches `session.modified = True` and returns 204; called by the session-timeout warning in `app.js` +- **Password reset** — `POST /forgot-password` creates a 1-hour `password_reset_tokens` token and emails a link. `POST /reset-password/` validates, sets the new password, and deletes the token. Expired tokens are purged on every successful login via `purge_expired_reset_tokens()`. +- **Profile editing** — `GET/POST /profile` lets any authenticated user update their `full_name` and `email`. Changes are immediately reflected in `session["user"]` so the sidebar name refreshes without re-login. --- @@ -149,14 +152,19 @@ On startup, `initialize_database()` seeds blank DB rows from `.env` using `ON DU Route: `/ai-summary/` (`routes/ai_summary.py`) -- **Stage 1 prompt** (`_EXTRACTION_PROMPT`) — extracts 12 fields per document including driving distance from `2815 Hartland Road, Falls Church, VA 22043` to the work site +- **Stage 1 prompt** (`_EXTRACTION_PROMPT`) — extracts 12 fields; treats all uploaded documents as one combined source (do NOT analyze per-file); driving distance from `2815 Hartland Road, Falls Church, VA 22043` - **Stage 2 prompt** (`_CRITERIA_PROMPT_SUFFIX`) — appended only when active criteria exist; produces a machine-readable `RECOMMENDATION: PURSUE|PASS|UNCLEAR` line - **File extraction** — `pypdf` for PDF, `python-docx` (imported as `docx`) for DOCX/DOC, `openpyxl` for XLSX, plain decode for TXT/CSV/MD +- **Magic-byte validation** — `_validate_magic()` rejects files whose bytes don't match their declared extension (disguised uploads) - **API call** — direct `requests.post()` to `https://api.groq.com/openai/v1/chat/completions` — the `groq` Python SDK is **not** installed - **Model** — `claude-sonnet-4-20250514` should NOT be used here; use `llama-3.3-70b-versatile` (Groq) - **Max tokens** — 4,096; **temperature** — 0.2 -- **Text limit** — combined document text is truncated to 14,000 characters before being sent; the JSON response includes `"truncated": true` when this occurs so the UI can warn the user +- **Text limit** — combined document text is truncated to 14,000 characters before being sent; the JSON response includes `"truncated": true` when this occurs - **`sort_order` input** — always wrap `int(request.form.get("sort_order", 0) or 0)` in `try/except (ValueError, TypeError)` — bad input must degrade gracefully to 0, not raise a 500 +- **Output rendering** — `marked.js` (CDN) renders the AI summary as formatted markdown in both the live output and history modal +- **History detail** — `GET /ai-summary/history/` returns `criteria_snapshot` (JSON string of `[{title, description}]`); the history modal parses and displays the criteria titles used +- **Delete history** — `POST /ai-summary/history//delete` enforces ownership (user can delete own; admin can delete any); logs the action +- **Verdict filter** — client-side pill buttons (All / PURSUE / PASS / UNCLEAR / No verdict) above the history table toggle row visibility --- @@ -188,7 +196,7 @@ Route: `/dashboard/` (`routes/user_dashboard.py`) - Each site card has two rows: `.sc-row1` (main flex row) and `.sc-row2` (note row, only if `user_note` is set) - `.sc-actions` uses `margin-left:auto` to push buttons to the right edge of `.sc-row1` - `.site-card` is `display:block` — **not flex** — so `.sc-row1` and `.sc-row2` stack vertically -- Health dots probe site reachability using Google's favicon service (cross-origin safe) +- **Health dots** — `GET /dashboard/health/` does a server-side HEAD request (6s timeout); returns `{status, ms}` JSON. Colors: green <3000ms, orange slow/timeout, red unreachable, gray error - The `hhmm` Jinja filter (`app.py`) handles MySQL `TIME` columns returned as `datetime.timedelta` --- @@ -320,25 +328,43 @@ mysql -u webchecker_user -p webchecker -e "SELECT key_name, value FROM app_setti --- -## Pending Improvements (To-Do) +## Completed Improvements + +All planned items have been implemented. The list below serves as a record and cross-reference. ### Security -- [x] **File upload size cap** — `MAX_CONTENT_LENGTH = 20 MB` set in `create_app()`; `RequestEntityTooLarge` handler returns JSON for `/ai-summary/` paths and flash+redirect for form routes -- [x] **HTTP security headers** — `@app.after_request` in `app.py` sets `X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` on every response -- [x] **Prevent last-admin demotion/deactivation** — `update_user()` in `models.py` checks that demoting or deactivating an admin won't leave zero active admins; raises `ValueError` surfaced as a flash message -- [x] **File MIME type validation** — `_validate_magic()` in `ai_summary.py` checks magic bytes for PDF (`%PDF`), DOCX/XLSX (`PK\x03\x04`), DOC/XLS (OLE header), and UTF-8 decodability for text types; no extra dependency needed +- [x] File upload size cap (`MAX_CONTENT_LENGTH = 20 MB`); JSON error for `/ai-summary/`, flash+redirect elsewhere +- [x] HTTP security headers — `X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` on every response +- [x] Prevent last-admin demotion/deactivation — `update_user()` raises `ValueError` if zero active admins would remain +- [x] File MIME type validation — `_validate_magic()` checks magic bytes and UTF-8 decodability; no extra dependency +- [x] Session fixation prevention — `session.clear()` before setting session on login +- [x] Password reset tokens — single-use, 1-hour expiry, purged on every successful login ### UI / UX -- [ ] **Mobile-responsive sidebar** — No media queries exist; the sidebar + main-content layout is unusable on phones; implement a hamburger toggle that collapses the sidebar on small screens -- [ ] **Bid due-date urgency badges** — Client-side only: highlight bids due within 7 days with a warning badge and bids past due with a red "Overdue" pill in the bid list -- [ ] **Relative timestamps in Activity Log** — Format raw `YYYY-MM-DD HH:MM:SS` timestamps as "2 hours ago" / "3 days ago" using a small JS formatter -- [ ] **Empty state for AI analysis history** — When no analyses exist, show a call-to-action ("Upload your first document ↑") instead of a blank panel -- [ ] **Paginate bid list** — `get_all_bids()` fetches all rows with no limit; add `LIMIT`/`OFFSET` to the model query and a "load more" button in the split-pane list +- [x] Mobile-responsive sidebar — hamburger toggle (`sidebar-toggle` button in `base.html`), CSS `transform:translateX`, backdrop overlay; IIFE in `app.js` +- [x] Bid due-date urgency badges — "Overdue" (red) and "Due Soon" (yellow) client-side pills in bid list +- [x] Relative timestamps in Activity Log — `timeAgo()` in `app.js`; `[data-ts]` attribute on timestamp `` elements in `logs.html` +- [x] Empty state for AI analysis history — robot emoji + "Start Analyzing" CTA when history is empty +- [x] Paginate bid list — `LIMIT`/`OFFSET` in `get_all_bids()`; "Load more" button in bid list; `_bidOffset` / `_PAGE_SIZE` state vars +- [x] AI history verdict filter — pill buttons (All / PURSUE / PASS / UNCLEAR / No verdict) filter rows client-side +- [x] Highlight today in shift calendar — JS IIFE maps `getDay()` → MySQL DAYOFWEEK → column index; adds `.cal-today` CSS class +- [x] Website list search + type filter — inline search input + dropdown filter client-side in `admin/websites.html` ### Functionality -- [x] **Missed-shift alerting** — `get_missed_shifts_today()` in `models.py` queries shift+user pairs scheduled today with 0 checks; surfaced as a warning card on the admin dashboard -- [x] **Bid deadline email reminders** — `get_bids_due_soon()` + `get_admin_emails()` in `models.py`; `send_reminders` route in `bid_tracker.py`; "📧 Remind" button in bid tracker toolbar (admin only); uses `utils/email.py` -- [x] **Server-side health checks** — `/dashboard/health/` in `user_dashboard.py` does a server-side HEAD request; user dashboard JS updated to call this instead of the Google favicon proxy -- [x] **Password reset via email** — `password_reset_tokens` table (migration-safe DDL in `config.py`); model functions in `models.py`; `/forgot-password` and `/reset-password/` routes in `auth.py`; standalone templates `forgot_password.html` and `reset_password.html`; "Forgot password?" link on login page; uses `utils/email.py` -- [x] **"Copy password" button in Credentials modal** — Already implemented in the original code via `data-copy` attribute and delegated click handler in `user/dashboard.html` -- [x] **Shift calendar view** — Weekly grid tab added to admin shifts page; Mon–Sun columns, active shifts as rows; server-side rendered with Jinja2 using existing `day_map` data +- [x] Missed-shift alerting — `get_missed_shifts_today()` in `models.py`; warning card on admin dashboard +- [x] Bid deadline email reminders — `send_reminders` route; "📧 Remind" button (admin only) in bid toolbar +- [x] Server-side health checks — `/dashboard/health/` HEAD request with 6s timeout; green/orange/red/gray dots +- [x] Password reset via email — `password_reset_tokens` table; `/forgot-password` + `/reset-password/` routes; standalone templates; "Forgot password?" on login page +- [x] Shift calendar view — weekly grid tab; Mon–Sun columns; active shifts as rows; `day_map` server-rendered +- [x] Delete AI analysis history — `POST /ai-summary/history//delete`; ownership check; delete button in history modal +- [x] Criteria snapshot in history modal — `analysis_detail` returns `criteria_snapshot`; modal shows titles of criteria used +- [x] Purge expired password reset tokens — `purge_expired_reset_tokens()` called on every successful login (wrapped in try/except) +- [x] Test SMTP + Groq buttons — `POST /admin/settings/test-email` and `/test-groq`; inline AJAX result in settings page +- [x] Admin send password reset link — `POST /admin/users//send-reset`; generates token + sends email; "✉ Reset" button per user row +- [x] Export bids to CSV — `GET /bids/export.csv`; respects active status filter; "⬇ CSV" button in toolbar +- [x] Admin dashboard bid KPIs — Open Bids, Due This Week (red when >0), AI Analyses (30d) added to `get_admin_dashboard_stats()` +- [x] User profile page — `GET/POST /profile`; users edit own `full_name` and `email`; session updated immediately; "👤 My Profile" in sidebar +- [x] SMTP From Address field — `email.smtp_from` exposed in Admin → Settings form and saved alongside other email fields + +### Pending +- No known pending items. Add new items here as they are identified. diff --git a/DEPLOY.md b/DEPLOY.md index eb5819d..1c63f8c 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -141,8 +141,18 @@ DB_PORT=3306 DB_NAME=webchecker DB_USER=webchecker_user DB_PASSWORD=YourStrongPasswordHere -CRYPTO_SECRET= FLASK_ENV=production + +# AI analysis (can also be set via Admin → Settings after first login) +GROQ_API_KEY= +GROQ_MODEL=llama-3.3-70b-versatile + +# SMTP email for password reset and bid reminders (can also be set via Admin → Settings) +SMTP_HOST= +SMTP_PORT=587 +SMTP_USER= +SMTP_PASSWORD= +SMTP_FROM=noreply@yourdomain.com ``` Secure the file so only the application user can read it: @@ -392,7 +402,15 @@ Open a browser and navigate to `https://your-domain.com`. You should see the log ### 9.4 First login -Log in with the default admin credentials set during `initialize_database()`. **Change the password immediately** via Admin → Change Password. +Log in with the default admin credentials set during `initialize_database()`. **Change the password immediately** via Admin → Users (edit your user) or via My Profile → Change Password. + +### 9.5 Configure email and AI settings + +Navigate to **Admin → Settings** and fill in: +- **Email** — SMTP host, port, security mode, username, password, from address, and recipient list. Click **📬 Test** to verify without leaving the page. +- **Groq AI** — paste your API key and select a model. Click **🔌 Test** to confirm the key works. + +These values are stored in the database and take precedence over `.env` fallbacks. --- @@ -472,6 +490,9 @@ Create `/etc/logrotate.d/webchecker`: | Sessions expiring immediately | `SECRET_KEY` changed between restarts | Keep `SECRET_KEY` stable in `.env`; never regenerate it on a live system | | `413 Request Entity Too Large` | File upload exceeds `client_max_body_size` | Increase `client_max_body_size` in the Nginx config | | Static files returning 404 | Wrong `alias` path in Nginx | Confirm `/opt/webchecker/static/` exists and the `alias` directive ends with `/` | +| Password reset emails not delivered | SMTP not configured or wrong credentials | Go to Admin → Settings, check SMTP fields, click **📬 Test** — the error message shows the SMTP failure reason | +| Groq AI analysis fails with 401 | Invalid or missing API key | Go to Admin → Settings → Groq, paste a valid key, click **🔌 Test** to confirm before saving | +| `BuildError` for a new route after deploy | Gunicorn still running old bytecode | Run `sudo systemctl reload webchecker` — templates go live immediately but Python changes require a reload | --- diff --git a/README.md b/README.md index b02cbc9..d59b4ec 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,7 @@ Website Checker helps teams track which government websites need to be checked d | Credential Encryption | Fernet (PBKDF2-HMAC-SHA256, 100k iterations) | | Password Hashing | bcrypt (12 rounds) | | AI Analysis | Groq REST API (llama-3.3-70b-versatile) | +| Markdown Rendering | marked.js (CDN, client-side) | | Frontend | Vanilla JS + DM Sans/DM Mono (Google Fonts) | | OS | Ubuntu 22.04 / 24.04 LTS | @@ -31,21 +32,28 @@ Website Checker helps teams track which government websites need to be checked d ## Features ### All Users -- **Shift Checklist** — Sites grouped by Daily / Weekly frequency, collapsible groups, live progress bar, bulk-check, search filter +- **Shift Checklist** — Sites grouped by Daily / Weekly frequency, collapsible groups, live progress bar, bulk-check, search filter, server-side health dot per site - **Site Credentials** — View stored credentials (password masked, copy-to-clipboard) - **Per-site Notes** — Add or update notes per check, displayed in a dedicated row under the site card -- **AI Document Analysis** — Upload PDF / DOCX / XLSX / TXT solicitation files; AI extracts solicitation number, scope of work, due dates, driving distance from office, and evaluates against criteria -- **Bid Tracker** — Split-pane view of all tracked opportunities; post updates, filter by status, search by title / source / solicitation number +- **AI Document Analysis** — Upload PDF / DOCX / XLSX / TXT solicitation files; AI extracts solicitation number, scope of work, due dates, driving distance from office, and evaluates against active criteria. All documents are analyzed as one combined source. Output rendered as formatted markdown. +- **AI History** — Filter past analyses by verdict (PURSUE / PASS / UNCLEAR / No verdict), view criteria snapshot used, delete own records +- **Bid Tracker** — Split-pane view of all tracked opportunities; post updates, filter by status, search by title / source / solicitation number, urgency badges for bids due within 7 days / overdue, load-more pagination, export current view to CSV +- **My Profile** — Update own full name and email address +- **Password Reset** — Self-service reset via emailed link (1-hour token); "Forgot password?" on login page +- **Relative Timestamps** — Activity log timestamps shown as "2 hours ago" / "3 days ago" +- **Mobile Sidebar** — Hamburger toggle collapses sidebar on small screens ### Admins Only -- **Dashboard** — KPI cards (users, active today, total sites) and per-user completion progress -- **User Management** — Create, edit, activate/deactivate users; role assignment (admin / user) -- **Website Management** — CRUD for monitored sites with credentials, check type (daily/weekly), and user assignment -- **Shift Management** — Define shifts with days-of-week, time windows, assigned users and sites +- **Dashboard** — KPI cards (users, active today, total sites, open bids, bids due this week, AI analyses past 30 days); per-user completion progress; missed-shift warning +- **User Management** — Create, edit, activate/deactivate users; role assignment; send password reset link directly to any user with an email address +- **Website Management** — CRUD for monitored sites with credentials, check type (daily/weekly), user assignment; inline search and type filter +- **Shift Management** — Define shifts with days-of-week, time windows, assigned users and sites; weekly calendar view with today's column highlighted - **Reports** — Shift detail, unchecked sites, and summary reports with date/user/site filters; CSV export - **Logs** — Activity log (user actions) and application log (system events) with search and purge -- **Settings** — SMTP email configuration and Groq AI API key / model selection +- **Settings** — SMTP email configuration (host, port, security, username, password, from address, recipients); Groq AI API key / model selection; one-click **Test** button for both SMTP and Groq to verify config without saving - **AI Criteria Management** — Create, edit, reorder, and deactivate evaluation criteria used by AI analysis +- **Bid Email Reminders** — One-click digest email to all admin addresses listing bids due within 7 days +- **Missed Shift Alerting** — Warning card on the admin dashboard showing any user/shift pair with 0 sites checked today --- @@ -53,30 +61,34 @@ Website Checker helps teams track which government websites need to be checked d ``` webchecker_web/ -├── app.py # Flask application factory, blueprint registration -├── config.py # DB connection pool, schema DDL, get/set settings -├── models.py # All data-access functions (no ORM) -├── wsgi.py # Gunicorn entry point +├── app.py # Flask application factory, blueprint registration +├── config.py # DB connection pool, schema DDL, get/set settings +├── models.py # All data-access functions (no ORM) +├── wsgi.py # Gunicorn entry point ├── requirements.txt -├── .env.example # Environment variable template -├── DEPLOY.md # Full production deployment guide +├── .env.example # Environment variable template +├── CLAUDE.md # AI developer context +├── DEPLOY.md # Full production deployment guide │ ├── routes/ -│ ├── auth.py # Login, logout, change password -│ ├── admin_dashboard.py # /admin/ -│ ├── admin_users.py # /admin/users/ -│ ├── admin_websites.py # /admin/websites/ -│ ├── admin_shifts.py # /admin/shifts/ -│ ├── admin_logs.py # /admin/logs/ -│ ├── admin_reports.py # /admin/reports/ -│ ├── admin_settings.py # /admin/settings/ -│ ├── user_dashboard.py # /dashboard/ -│ ├── ai_summary.py # /ai-summary/ -│ └── bid_tracker.py # /bids/ +│ ├── auth.py # Login, logout, change password, profile, password reset +│ ├── admin_dashboard.py # /admin/ +│ ├── admin_users.py # /admin/users/ +│ ├── admin_websites.py # /admin/websites/ +│ ├── admin_shifts.py # /admin/shifts/ +│ ├── admin_logs.py # /admin/logs/ +│ ├── admin_reports.py # /admin/reports/ +│ ├── admin_settings.py # /admin/settings/ (incl. test-email, test-groq) +│ ├── user_dashboard.py # /dashboard/ (incl. server-side health check) +│ ├── ai_summary.py # /ai-summary/ (incl. history delete) +│ └── bid_tracker.py # /bids/ (incl. CSV export, email reminders) │ ├── templates/ -│ ├── base.html # Sidebar layout, flash messages, nav -│ ├── login.html +│ ├── base.html # Sidebar layout, flash messages, nav, mobile toggle +│ ├── login.html # Standalone (no base.html) +│ ├── forgot_password.html # Standalone password-reset request +│ ├── reset_password.html # Standalone new-password form +│ ├── profile.html # User profile editor │ ├── change_password.html │ ├── ai_summary.html │ ├── bid_tracker.html @@ -92,12 +104,13 @@ webchecker_web/ │ └── dashboard.html │ ├── static/ -│ ├── css/style.css # Full light-theme design system -│ └── js/app.js # Modal helpers, tabs, session timeout warning +│ ├── css/style.css # Full light-theme design system +│ └── js/app.js # Modal helpers, tabs, timeAgo(), mobile sidebar, session timeout │ └── utils/ - ├── crypto.py # Fernet encryption (DB-compatible with desktop app) - └── decorators.py # @login_required, @admin_required + ├── crypto.py # Fernet encryption (DB-compatible with desktop app) + ├── decorators.py # @login_required, @admin_required + └── email.py # SMTP helper: send_email(to, subject, body) ``` --- @@ -109,17 +122,26 @@ webchecker_web/ | `/` | Root redirect (role-based) | Any | | `/login` | Login page | Public | | `/logout` | Session clear + redirect | Authenticated | +| `/ping` | Session keepalive (returns 204) | Authenticated | +| `/profile` | Edit own full name and email | Authenticated | | `/change-password` | Change own password | Authenticated | -| `/admin/` | Admin dashboard | Admin | -| `/admin/users/` | User CRUD | Admin | +| `/forgot-password` | Request password reset email | Public | +| `/reset-password/` | Set new password via token | Public | +| `/admin/dashboard` | Admin dashboard with KPIs | Admin | +| `/admin/users/` | User CRUD + send reset link | Admin | | `/admin/websites/` | Website CRUD + credentials | Admin | -| `/admin/shifts/` | Shift CRUD + assignments | Admin | +| `/admin/shifts/` | Shift CRUD + calendar view | Admin | | `/admin/logs/` | Activity & app logs | Admin | | `/admin/reports/` | Shift reports + CSV export | Admin | -| `/admin/settings/` | Email + Groq AI settings | Admin | +| `/admin/settings/` | Email + Groq AI settings + connection tests | Admin | | `/dashboard/` | User shift checklist | User | -| `/ai-summary/` | AI document analysis | Any | +| `/dashboard/health/` | Server-side site health check | User | +| `/ai-summary/` | AI document analysis + history | Any | +| `/ai-summary/history/` | Analysis detail (JSON) | Any | +| `/ai-summary/history//delete` | Delete an analysis record | Any | | `/bids/` | Bid / opportunity tracker | Any | +| `/bids/export.csv` | Export bid list as CSV | Any | +| `/bids/remind` | Send bid deadline reminder email | Admin | --- @@ -134,12 +156,21 @@ DB_PORT=3306 DB_NAME=webchecker DB_USER=webchecker_user DB_PASSWORD= -CRYPTO_SECRET= # Fernet key — generate with: python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())" -GROQ_API_KEY= # Optional — used by AI Summary feature + +GROQ_API_KEY= # Optional — can also be set via Admin → Settings +GROQ_MODEL=llama-3.3-70b-versatile # Optional fallback; overridden by DB setting + +# SMTP email — optional fallback; values are stored in and overridden by Admin → Settings +SMTP_HOST= +SMTP_PORT=587 +SMTP_USER= +SMTP_PASSWORD= +SMTP_FROM= + FLASK_ENV=production ``` -> **Important:** `CRYPTO_SECRET` must match the key used by the desktop app if you are sharing the database. The salt for encryption is stored in the `app_settings` table under `crypto.salt`. +> **Important:** `SECRET_KEY` must remain stable across restarts — changing it invalidates all active sessions and CSRF tokens across Gunicorn workers. --- @@ -149,7 +180,8 @@ This web application shares the MySQL database with the desktop application. Key - **Encryption** — `utils/crypto.py` uses the exact same key derivation as the desktop (`APP_SECRET = b"WebsiteChecker-v1-CredentialKey"`, 100,000 PBKDF2 iterations, base64 salt, `enc:` prefix). Credentials are interchangeable between apps. - **`activity_log`** — The live database column is `logged_at` (desktop schema). Queries use `al.logged_at AS created_at` for template compatibility. -- **`app_settings`** — On startup, `initialize_database()` seeds any settings keys that are blank in the DB from the corresponding `.env` variables (e.g. `GROQ_API_KEY → groq.api_key`), using `ON DUPLICATE KEY UPDATE … IF(value = '', …)` so admin-saved values are never overwritten. +- **`app_settings`** — On startup, `initialize_database()` seeds any settings keys that are blank in the DB from the corresponding `.env` variables, using `ON DUPLICATE KEY UPDATE … IF(value = '', …)` so admin-saved values are never overwritten. +- **`password_reset_tokens`** — Web-only table; created by the app's migration-safe DDL on first startup. Tokens expire after 1 hour and are consumed on use. Expired tokens are purged on every successful login. - **`login_attempts.ip_address`** — The only additive column the web app creates. Added via `ALTER TABLE … ADD COLUMN IF NOT EXISTS` on startup. --- @@ -178,25 +210,36 @@ python app.py # Open http://localhost:5000 ``` -### Restart the production service +### Deploy changes to production ```bash -sudo systemctl restart webchecker -sudo journalctl -u webchecker -f +# After copying Python files to the server: +sudo systemctl reload webchecker + +# Templates and static files take effect immediately (no reload needed) + +# Check for errors after reload: +sudo journalctl -u webchecker -n 30 --no-pager ``` --- -## Security Notes +## Security - Passwords are hashed with bcrypt (12 rounds); legacy SHA-256 hashes from the desktop app are automatically rehashed on next login. -- Sessions expire after **30 minutes** of inactivity. A client-side warning fires 5 minutes before expiry. +- Sessions expire after **30 minutes** of inactivity. A client-side warning fires 5 minutes before expiry with a keep-alive ping endpoint. - Login is rate-limited: 5 failed attempts locks the account for 15 minutes. -- Credentials stored in `website_credentials` are Fernet-encrypted at rest. Plaintext legacy values (without the `enc:` prefix) are returned as-is for backward compatibility. -- `SECRET_KEY` must remain stable across restarts — changing it invalidates all active sessions. +- Credentials stored in `website_credentials` are Fernet-encrypted at rest. +- File uploads for AI analysis are validated against magic bytes (PDF `%PDF`, DOCX/XLSX `PK\x03\x04`, DOC/XLS OLE header) and UTF-8 decodability for text files — rejecting disguised uploads without extra dependencies. +- Upload size is capped at 20 MB (`MAX_CONTENT_LENGTH`). +- HTTP security headers (`X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy`) are set on every response. +- CSRF protection (Flask-WTF) on all non-GET requests. +- Session fixation prevention: `session.clear()` is called immediately before setting the session on successful login. +- Password reset tokens are single-use, expire after 1 hour, and are purged on each login. +- Admins cannot demote or deactivate the last remaining active admin. --- ## Deployment -See [`DEPLOY.md`](DEPLOY.md) for the full step-by-step guide covering MySQL setup, Gunicorn systemd service, Nginx reverse proxy, Certbot SSL, firewall rules, log rotation, and database backup. \ No newline at end of file +See [`DEPLOY.md`](DEPLOY.md) for the full step-by-step guide covering MySQL setup, Gunicorn systemd service, Nginx reverse proxy, Certbot SSL, firewall rules, log rotation, and database backup. diff --git a/models.py b/models.py index 5ba4d4c..dcc67e5 100644 --- a/models.py +++ b/models.py @@ -250,6 +250,23 @@ def get_user_by_id(user_id: int): conn.close() +def update_user_profile(user_id: int, full_name: str, email: str): + """Allow a user to update their own full name and email address.""" + conn = None + try: + conn = get_connection() + cur = conn.cursor() + cur.execute( + "UPDATE users SET full_name=%s, email=%s WHERE id=%s", + (full_name.strip() or None, email.strip().lower() or None, user_id), + ) + conn.commit() + cur.close() + finally: + if conn: + conn.close() + + def create_user(admin_id, username, password, role, full_name, email=None): conn = None try: @@ -1092,12 +1109,28 @@ def get_admin_dashboard_stats(): total_users = cur.fetchone()["n"] cur.execute("SELECT COUNT(DISTINCT user_id) AS n FROM shift_checks WHERE DATE(checked_at)=CURDATE()") active_today = cur.fetchone()["n"] + cur.execute("SELECT COUNT(*) AS n FROM bid_tracker WHERE status IN ('open','monitoring')") + open_bids = cur.fetchone()["n"] + cur.execute( + "SELECT COUNT(*) AS n FROM bid_tracker" + " WHERE status IN ('open','monitoring')" + " AND due_date BETWEEN CURDATE() AND DATE_ADD(CURDATE(), INTERVAL 7 DAY)" + ) + bids_due_soon = cur.fetchone()["n"] + cur.execute( + "SELECT COUNT(*) AS n FROM ai_analysis_log" + " WHERE analyzed_at >= DATE_SUB(NOW(), INTERVAL 30 DAY)" + ) + ai_analyses_30d = cur.fetchone()["n"] cur.close() return { - "user_stats": user_stats, - "total_sites": total_sites, - "total_users": total_users, - "active_today": active_today, + "user_stats": user_stats, + "total_sites": total_sites, + "total_users": total_users, + "active_today": active_today, + "open_bids": open_bids, + "bids_due_soon": bids_due_soon, + "ai_analyses_30d": ai_analyses_30d, } finally: if conn: diff --git a/routes/admin_settings.py b/routes/admin_settings.py index b222151..27d5b42 100644 --- a/routes/admin_settings.py +++ b/routes/admin_settings.py @@ -30,7 +30,7 @@ def save_email(): fields = [ "email.enabled", "email.smtp_host", "email.smtp_port", "email.smtp_user", "email.smtp_password", "email.security", - "email.recipients", "email.send_time", + "email.smtp_from", "email.recipients", "email.send_time", ] for field in fields: key = field diff --git a/routes/auth.py b/routes/auth.py index 681a658..fb23990 100644 --- a/routes/auth.py +++ b/routes/auth.py @@ -8,7 +8,7 @@ from models import ( authenticate, check_login_allowed, change_password, log_action, get_user_by_email, create_password_reset_token, get_password_reset_user, consume_password_reset_token, - purge_expired_reset_tokens, + purge_expired_reset_tokens, update_user_profile, ) from utils.decorators import login_required from utils.email import send_email @@ -107,6 +107,28 @@ def ping(): return "", 204 +@auth_bp.route("/profile", methods=["GET", "POST"]) +@login_required +def profile(): + user = session["user"] + if request.method == "POST": + full_name = request.form.get("full_name", "").strip() + email = request.form.get("email", "").strip() + try: + update_user_profile(user["id"], full_name, email) + session["user"]["full_name"] = full_name or user["username"] + session["user"]["email"] = email + session.modified = True + log_action(user["id"], "UPDATE_PROFILE", "users", user["id"], + "User updated their own profile.") + flash("Profile updated successfully.", "success") + except Exception as e: + logger.error(f"Profile update error for user_id={user['id']}: {e}") + flash(f"Error: {e}", "danger") + return redirect(url_for("auth.profile")) + return render_template("profile.html") + + @auth_bp.route("/forgot-password", methods=["GET", "POST"]) def forgot_password(): if "user" in session: diff --git a/templates/admin/dashboard.html b/templates/admin/dashboard.html index 0ee6b79..b3838b9 100644 --- a/templates/admin/dashboard.html +++ b/templates/admin/dashboard.html @@ -20,6 +20,18 @@
{{ stats.total_sites }}
Total Sites
+
+
{{ stats.open_bids }}
+
Open Bids
+
+
+
{{ stats.bids_due_soon }}
+
Due This Week
+
+
+
{{ stats.ai_analyses_30d }}
+
AI Analyses (30d)
+
diff --git a/templates/admin/settings.html b/templates/admin/settings.html index 3849575..6830069 100644 --- a/templates/admin/settings.html +++ b/templates/admin/settings.html @@ -42,6 +42,12 @@ +
+ + +
diff --git a/templates/admin/websites.html b/templates/admin/websites.html index 4ae9127..b8e41d0 100644 --- a/templates/admin/websites.html +++ b/templates/admin/websites.html @@ -3,7 +3,18 @@ {% block content %}

Website Link Management

- +
+
+ 🔍 + +
+ + +
@@ -13,7 +24,9 @@ {% for w in websites %} - + {{ w.id }} {{ w.name }} {{ w.check_type }} @@ -172,5 +185,21 @@ function addCredRow(btn) { } function removeRow(btn) { btn.closest('.cred-row').remove(); } function esc(s) { return (s||'').replace(/&/g,'&').replace(/"/g,'"').replace(/ {% endblock %} diff --git a/templates/base.html b/templates/base.html index 95b056b..3fb781f 100644 --- a/templates/base.html +++ b/templates/base.html @@ -46,6 +46,7 @@
{{ current_user.full_name or current_user.username }}
{{ current_user.role | capitalize }}
+ 👤 My Profile 🔑 Change Password ⇠ Sign Out {% endif %} diff --git a/templates/profile.html b/templates/profile.html new file mode 100644 index 0000000..a1bfb58 --- /dev/null +++ b/templates/profile.html @@ -0,0 +1,39 @@ +{% extends "base.html" %} +{% block title %}My Profile — Website Checker{% endblock %} +{% block content %} +
+

My Profile

+
+ +
+
+
Account Details
+
+
+
+ + + Username cannot be changed. Contact an admin if needed. +
+
+ + +
+
+ + + Used for password resets and notifications. +
+
+
+ 🔑 Change Password + +
+
+
+
+{% endblock %}