nngo/WebChecker--Web-app-
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

05/24 Enhance functionalities 2

Browse Source
This commit is contained in:
nngo
2026-05-24 23:04:55 -04:00
parent cb98c7e222
commit 13f8ba124d
11 changed files with 309 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+45 -19
View File
@@ -108,6 +108,7 @@ Without this, sessions and CSRF tokens are broken across the 4 Gunicorn workers.
| `templates/login.html` | Standalone login page | Does NOT extend `base.html`; has its own `<head>` and no `app.js`; CSRF token must be a direct hidden input | | `templates/login.html` | Standalone login page | Does NOT extend `base.html`; has its own `<head>` and no `app.js`; CSRF token must be a direct hidden input |
| `templates/forgot_password.html` | Standalone forgot-password page | Same constraints as `login.html` — standalone, direct CSRF hidden input, no `app.js` | | `templates/forgot_password.html` | Standalone forgot-password page | Same constraints as `login.html` — standalone, direct CSRF hidden input, no `app.js` |
| `templates/reset_password.html` | Standalone reset-password page | Same constraints as `login.html` — standalone, direct CSRF hidden input, no `app.js` | | `templates/reset_password.html` | Standalone reset-password page | Same constraints as `login.html` — standalone, direct CSRF hidden input, no `app.js` |
| `templates/profile.html` | User profile editor | Extends `base.html`; users can update `full_name` and `email`; username/role are read-only |
--- ---
@@ -121,6 +122,8 @@ Without this, sessions and CSRF tokens are broken across the 4 Gunicorn workers.
- `@admin_required` — redirects to login or 403 if not admin - `@admin_required` — redirects to login or 403 if not admin
- **Session fixation prevention** — `session.clear()` is called in `auth.login` immediately before setting `session["user"]` on successful authentication - **Session fixation prevention** — `session.clear()` is called in `auth.login` immediately before setting `session["user"]` on successful authentication
- **Keep-alive endpoint** — `GET /ping` (`routes/auth.py`, `@login_required`) touches `session.modified = True` and returns 204; called by the session-timeout warning in `app.js` - **Keep-alive endpoint** — `GET /ping` (`routes/auth.py`, `@login_required`) touches `session.modified = True` and returns 204; called by the session-timeout warning in `app.js`
- **Password reset** — `POST /forgot-password` creates a 1-hour `password_reset_tokens` token and emails a link. `POST /reset-password/<token>` validates, sets the new password, and deletes the token. Expired tokens are purged on every successful login via `purge_expired_reset_tokens()`.
- **Profile editing** — `GET/POST /profile` lets any authenticated user update their `full_name` and `email`. Changes are immediately reflected in `session["user"]` so the sidebar name refreshes without re-login.
--- ---
@@ -149,14 +152,19 @@ On startup, `initialize_database()` seeds blank DB rows from `.env` using `ON DU
Route: `/ai-summary/` (`routes/ai_summary.py`) Route: `/ai-summary/` (`routes/ai_summary.py`)
- **Stage 1 prompt** (`_EXTRACTION_PROMPT`) — extracts 12 fields per document including driving distance from `2815 Hartland Road, Falls Church, VA 22043` to the work site - **Stage 1 prompt** (`_EXTRACTION_PROMPT`) — extracts 12 fields; treats all uploaded documents as one combined source (do NOT analyze per-file); driving distance from `2815 Hartland Road, Falls Church, VA 22043`
- **Stage 2 prompt** (`_CRITERIA_PROMPT_SUFFIX`) — appended only when active criteria exist; produces a machine-readable `RECOMMENDATION: PURSUE|PASS|UNCLEAR` line - **Stage 2 prompt** (`_CRITERIA_PROMPT_SUFFIX`) — appended only when active criteria exist; produces a machine-readable `RECOMMENDATION: PURSUE|PASS|UNCLEAR` line
- **File extraction** — `pypdf` for PDF, `python-docx` (imported as `docx`) for DOCX/DOC, `openpyxl` for XLSX, plain decode for TXT/CSV/MD - **File extraction** — `pypdf` for PDF, `python-docx` (imported as `docx`) for DOCX/DOC, `openpyxl` for XLSX, plain decode for TXT/CSV/MD
- **Magic-byte validation** — `_validate_magic()` rejects files whose bytes don't match their declared extension (disguised uploads)
- **API call** — direct `requests.post()` to `https://api.groq.com/openai/v1/chat/completions` — the `groq` Python SDK is **not** installed - **API call** — direct `requests.post()` to `https://api.groq.com/openai/v1/chat/completions` — the `groq` Python SDK is **not** installed
- **Model** — `claude-sonnet-4-20250514` should NOT be used here; use `llama-3.3-70b-versatile` (Groq) - **Model** — `claude-sonnet-4-20250514` should NOT be used here; use `llama-3.3-70b-versatile` (Groq)
- **Max tokens** — 4,096; **temperature** — 0.2 - **Max tokens** — 4,096; **temperature** — 0.2
- **Text limit** — combined document text is truncated to 14,000 characters before being sent; the JSON response includes `"truncated": true` when this occurs so the UI can warn the user - **Text limit** — combined document text is truncated to 14,000 characters before being sent; the JSON response includes `"truncated": true` when this occurs
- **`sort_order` input** — always wrap `int(request.form.get("sort_order", 0) or 0)` in `try/except (ValueError, TypeError)` — bad input must degrade gracefully to 0, not raise a 500 - **`sort_order` input** — always wrap `int(request.form.get("sort_order", 0) or 0)` in `try/except (ValueError, TypeError)` — bad input must degrade gracefully to 0, not raise a 500
- **Output rendering** — `marked.js` (CDN) renders the AI summary as formatted markdown in both the live output and history modal
- **History detail** — `GET /ai-summary/history/<id>` returns `criteria_snapshot` (JSON string of `[{title, description}]`); the history modal parses and displays the criteria titles used
- **Delete history** — `POST /ai-summary/history/<id>/delete` enforces ownership (user can delete own; admin can delete any); logs the action
- **Verdict filter** — client-side pill buttons (All / PURSUE / PASS / UNCLEAR / No verdict) above the history table toggle row visibility
--- ---
@@ -188,7 +196,7 @@ Route: `/dashboard/` (`routes/user_dashboard.py`)
- Each site card has two rows: `.sc-row1` (main flex row) and `.sc-row2` (note row, only if `user_note` is set) - Each site card has two rows: `.sc-row1` (main flex row) and `.sc-row2` (note row, only if `user_note` is set)
- `.sc-actions` uses `margin-left:auto` to push buttons to the right edge of `.sc-row1` - `.sc-actions` uses `margin-left:auto` to push buttons to the right edge of `.sc-row1`
- `.site-card` is `display:block` — **not flex** — so `.sc-row1` and `.sc-row2` stack vertically - `.site-card` is `display:block` — **not flex** — so `.sc-row1` and `.sc-row2` stack vertically
- Health dots probe site reachability using Google's favicon service (cross-origin safe) - **Health dots** — `GET /dashboard/health/<id>` does a server-side HEAD request (6s timeout); returns `{status, ms}` JSON. Colors: green <3000ms, orange slow/timeout, red unreachable, gray error
- The `hhmm` Jinja filter (`app.py`) handles MySQL `TIME` columns returned as `datetime.timedelta` - The `hhmm` Jinja filter (`app.py`) handles MySQL `TIME` columns returned as `datetime.timedelta`
--- ---
@@ -320,25 +328,43 @@ mysql -u webchecker_user -p webchecker -e "SELECT key_name, value FROM app_setti
--- ---
## Pending Improvements (To-Do) ## Completed Improvements
All planned items have been implemented. The list below serves as a record and cross-reference.
### Security ### Security
- [x] **File upload size cap** — `MAX_CONTENT_LENGTH = 20 MB` set in `create_app()`; `RequestEntityTooLarge` handler returns JSON for `/ai-summary/` paths and flash+redirect for form routes - [x] File upload size cap (`MAX_CONTENT_LENGTH = 20 MB`); JSON error for `/ai-summary/`, flash+redirect elsewhere
- [x] **HTTP security headers** — `@app.after_request` in `app.py` sets `X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` on every response - [x] HTTP security headers — `X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` on every response
- [x] **Prevent last-admin demotion/deactivation** — `update_user()` in `models.py` checks that demoting or deactivating an admin won't leave zero active admins; raises `ValueError` surfaced as a flash message - [x] Prevent last-admin demotion/deactivation — `update_user()` raises `ValueError` if zero active admins would remain
- [x] **File MIME type validation** — `_validate_magic()` in `ai_summary.py` checks magic bytes for PDF (`%PDF`), DOCX/XLSX (`PK\x03\x04`), DOC/XLS (OLE header), and UTF-8 decodability for text types; no extra dependency needed - [x] File MIME type validation — `_validate_magic()` checks magic bytes and UTF-8 decodability; no extra dependency
- [x] Session fixation prevention — `session.clear()` before setting session on login
- [x] Password reset tokens — single-use, 1-hour expiry, purged on every successful login
### UI / UX ### UI / UX
- [ ] **Mobile-responsive sidebar** — No media queries exist; the sidebar + main-content layout is unusable on phones; implement a hamburger toggle that collapses the sidebar on small screens - [x] Mobile-responsive sidebar — hamburger toggle (`sidebar-toggle` button in `base.html`), CSS `transform:translateX`, backdrop overlay; IIFE in `app.js`
- [ ] **Bid due-date urgency badges** — Client-side only: highlight bids due within 7 days with a warning badge and bids past due with a red "Overdue" pill in the bid list - [x] Bid due-date urgency badges — "Overdue" (red) and "Due Soon" (yellow) client-side pills in bid list
- [ ] **Relative timestamps in Activity Log** — Format raw `YYYY-MM-DD HH:MM:SS` timestamps as "2 hours ago" / "3 days ago" using a small JS formatter - [x] Relative timestamps in Activity Log — `timeAgo()` in `app.js`; `[data-ts]` attribute on timestamp `<span>` elements in `logs.html`
- [ ] **Empty state for AI analysis history** — When no analyses exist, show a call-to-action ("Upload your first document ↑") instead of a blank panel - [x] Empty state for AI analysis history — robot emoji + "Start Analyzing" CTA when history is empty
- [ ] **Paginate bid list** — `get_all_bids()` fetches all rows with no limit; add `LIMIT`/`OFFSET` to the model query and a "load more" button in the split-pane list - [x] Paginate bid list — `LIMIT`/`OFFSET` in `get_all_bids()`; "Load more" button in bid list; `_bidOffset` / `_PAGE_SIZE` state vars
- [x] AI history verdict filter — pill buttons (All / PURSUE / PASS / UNCLEAR / No verdict) filter rows client-side
- [x] Highlight today in shift calendar — JS IIFE maps `getDay()` → MySQL DAYOFWEEK → column index; adds `.cal-today` CSS class
- [x] Website list search + type filter — inline search input + dropdown filter client-side in `admin/websites.html`
### Functionality ### Functionality
- [x] **Missed-shift alerting** — `get_missed_shifts_today()` in `models.py` queries shift+user pairs scheduled today with 0 checks; surfaced as a warning card on the admin dashboard - [x] Missed-shift alerting — `get_missed_shifts_today()` in `models.py`; warning card on admin dashboard
- [x] **Bid deadline email reminders** — `get_bids_due_soon()` + `get_admin_emails()` in `models.py`; `send_reminders` route in `bid_tracker.py`; "📧 Remind" button in bid tracker toolbar (admin only); uses `utils/email.py` - [x] Bid deadline email reminders — `send_reminders` route; "📧 Remind" button (admin only) in bid toolbar
- [x] **Server-side health checks** — `/dashboard/health/<id>` in `user_dashboard.py` does a server-side HEAD request; user dashboard JS updated to call this instead of the Google favicon proxy - [x] Server-side health checks — `/dashboard/health/<id>` HEAD request with 6s timeout; green/orange/red/gray dots
- [x] **Password reset via email** — `password_reset_tokens` table (migration-safe DDL in `config.py`); model functions in `models.py`; `/forgot-password` and `/reset-password/<token>` routes in `auth.py`; standalone templates `forgot_password.html` and `reset_password.html`; "Forgot password?" link on login page; uses `utils/email.py` - [x] Password reset via email — `password_reset_tokens` table; `/forgot-password` + `/reset-password/<token>` routes; standalone templates; "Forgot password?" on login page
- [x] **"Copy password" button in Credentials modal** — Already implemented in the original code via `data-copy` attribute and delegated click handler in `user/dashboard.html` - [x] Shift calendar view — weekly grid tab; Mon–Sun columns; active shifts as rows; `day_map` server-rendered
- [x] **Shift calendar view** — Weekly grid tab added to admin shifts page; Mon–Sun columns, active shifts as rows; server-side rendered with Jinja2 using existing `day_map` data - [x] Delete AI analysis history — `POST /ai-summary/history/<id>/delete`; ownership check; delete button in history modal
- [x] Criteria snapshot in history modal — `analysis_detail` returns `criteria_snapshot`; modal shows titles of criteria used
- [x] Purge expired password reset tokens — `purge_expired_reset_tokens()` called on every successful login (wrapped in try/except)
- [x] Test SMTP + Groq buttons — `POST /admin/settings/test-email` and `/test-groq`; inline AJAX result in settings page
- [x] Admin send password reset link — `POST /admin/users/<id>/send-reset`; generates token + sends email; "✉ Reset" button per user row
- [x] Export bids to CSV — `GET /bids/export.csv`; respects active status filter; "⬇ CSV" button in toolbar
- [x] Admin dashboard bid KPIs — Open Bids, Due This Week (red when >0), AI Analyses (30d) added to `get_admin_dashboard_stats()`
- [x] User profile page — `GET/POST /profile`; users edit own `full_name` and `email`; session updated immediately; "👤 My Profile" in sidebar
- [x] SMTP From Address field — `email.smtp_from` exposed in Admin → Settings form and saved alongside other email fields
### Pending
- No known pending items. Add new items here as they are identified.
DEPLOY.md
+23 -2
View File
@@ -141,8 +141,18 @@ DB_PORT=3306
DB_NAME=webchecker DB_NAME=webchecker
DB_USER=webchecker_user DB_USER=webchecker_user
DB_PASSWORD=YourStrongPasswordHere DB_PASSWORD=YourStrongPasswordHere
CRYPTO_SECRET=<output of: python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())">
FLASK_ENV=production FLASK_ENV=production
# AI analysis (can also be set via Admin → Settings after first login)
GROQ_API_KEY=
GROQ_MODEL=llama-3.3-70b-versatile
# SMTP email for password reset and bid reminders (can also be set via Admin → Settings)
SMTP_HOST=
SMTP_PORT=587
SMTP_USER=
SMTP_PASSWORD=
SMTP_FROM=noreply@yourdomain.com
``` ```
Secure the file so only the application user can read it: Secure the file so only the application user can read it:
@@ -392,7 +402,15 @@ Open a browser and navigate to `https://your-domain.com`. You should see the log
### 9.4 First login ### 9.4 First login
Log in with the default admin credentials set during `initialize_database()`. **Change the password immediately** via Admin → Change Password. Log in with the default admin credentials set during `initialize_database()`. **Change the password immediately** via Admin → Users (edit your user) or via My Profile → Change Password.
### 9.5 Configure email and AI settings
Navigate to **Admin → Settings** and fill in:
- **Email** — SMTP host, port, security mode, username, password, from address, and recipient list. Click **📬 Test** to verify without leaving the page.
- **Groq AI** — paste your API key and select a model. Click **🔌 Test** to confirm the key works.
These values are stored in the database and take precedence over `.env` fallbacks.
--- ---
@@ -472,6 +490,9 @@ Create `/etc/logrotate.d/webchecker`:
| Sessions expiring immediately | `SECRET_KEY` changed between restarts | Keep `SECRET_KEY` stable in `.env`; never regenerate it on a live system | | Sessions expiring immediately | `SECRET_KEY` changed between restarts | Keep `SECRET_KEY` stable in `.env`; never regenerate it on a live system |
| `413 Request Entity Too Large` | File upload exceeds `client_max_body_size` | Increase `client_max_body_size` in the Nginx config | | `413 Request Entity Too Large` | File upload exceeds `client_max_body_size` | Increase `client_max_body_size` in the Nginx config |
| Static files returning 404 | Wrong `alias` path in Nginx | Confirm `/opt/webchecker/static/` exists and the `alias` directive ends with `/` | | Static files returning 404 | Wrong `alias` path in Nginx | Confirm `/opt/webchecker/static/` exists and the `alias` directive ends with `/` |
| Password reset emails not delivered | SMTP not configured or wrong credentials | Go to Admin → Settings, check SMTP fields, click **📬 Test** — the error message shows the SMTP failure reason |
| Groq AI analysis fails with 401 | Invalid or missing API key | Go to Admin → Settings → Groq, paste a valid key, click **🔌 Test** to confirm before saving |
| `BuildError` for a new route after deploy | Gunicorn still running old bytecode | Run `sudo systemctl reload webchecker` — templates go live immediately but Python changes require a reload |
--- ---
README.md
+91 -48
View File
@@ -23,6 +23,7 @@ Website Checker helps teams track which government websites need to be checked d
| Credential Encryption | Fernet (PBKDF2-HMAC-SHA256, 100k iterations) | | Credential Encryption | Fernet (PBKDF2-HMAC-SHA256, 100k iterations) |
| Password Hashing | bcrypt (12 rounds) | | Password Hashing | bcrypt (12 rounds) |
| AI Analysis | Groq REST API (llama-3.3-70b-versatile) | | AI Analysis | Groq REST API (llama-3.3-70b-versatile) |
| Markdown Rendering | marked.js (CDN, client-side) |
| Frontend | Vanilla JS + DM Sans/DM Mono (Google Fonts) | | Frontend | Vanilla JS + DM Sans/DM Mono (Google Fonts) |
| OS | Ubuntu 22.04 / 24.04 LTS | | OS | Ubuntu 22.04 / 24.04 LTS |
@@ -31,21 +32,28 @@ Website Checker helps teams track which government websites need to be checked d
## Features ## Features
### All Users ### All Users
- **Shift Checklist** — Sites grouped by Daily / Weekly frequency, collapsible groups, live progress bar, bulk-check, search filter - **Shift Checklist** — Sites grouped by Daily / Weekly frequency, collapsible groups, live progress bar, bulk-check, search filter, server-side health dot per site
- **Site Credentials** — View stored credentials (password masked, copy-to-clipboard) - **Site Credentials** — View stored credentials (password masked, copy-to-clipboard)
- **Per-site Notes** — Add or update notes per check, displayed in a dedicated row under the site card - **Per-site Notes** — Add or update notes per check, displayed in a dedicated row under the site card
- **AI Document Analysis** — Upload PDF / DOCX / XLSX / TXT solicitation files; AI extracts solicitation number, scope of work, due dates, driving distance from office, and evaluates against criteria - **AI Document Analysis** — Upload PDF / DOCX / XLSX / TXT solicitation files; AI extracts solicitation number, scope of work, due dates, driving distance from office, and evaluates against active criteria. All documents are analyzed as one combined source. Output rendered as formatted markdown.
- **Bid Tracker** — Split-pane view of all tracked opportunities; post updates, filter by status, search by title / source / solicitation number - **AI History** — Filter past analyses by verdict (PURSUE / PASS / UNCLEAR / No verdict), view criteria snapshot used, delete own records
- **Bid Tracker** — Split-pane view of all tracked opportunities; post updates, filter by status, search by title / source / solicitation number, urgency badges for bids due within 7 days / overdue, load-more pagination, export current view to CSV
- **My Profile** — Update own full name and email address
- **Password Reset** — Self-service reset via emailed link (1-hour token); "Forgot password?" on login page
- **Relative Timestamps** — Activity log timestamps shown as "2 hours ago" / "3 days ago"
- **Mobile Sidebar** — Hamburger toggle collapses sidebar on small screens
### Admins Only ### Admins Only
- **Dashboard** — KPI cards (users, active today, total sites) and per-user completion progress - **Dashboard** — KPI cards (users, active today, total sites, open bids, bids due this week, AI analyses past 30 days); per-user completion progress; missed-shift warning
- **User Management** — Create, edit, activate/deactivate users; role assignment (admin / user) - **User Management** — Create, edit, activate/deactivate users; role assignment; send password reset link directly to any user with an email address
- **Website Management** — CRUD for monitored sites with credentials, check type (daily/weekly), and user assignment - **Website Management** — CRUD for monitored sites with credentials, check type (daily/weekly), user assignment; inline search and type filter
- **Shift Management** — Define shifts with days-of-week, time windows, assigned users and sites - **Shift Management** — Define shifts with days-of-week, time windows, assigned users and sites; weekly calendar view with today's column highlighted
- **Reports** — Shift detail, unchecked sites, and summary reports with date/user/site filters; CSV export - **Reports** — Shift detail, unchecked sites, and summary reports with date/user/site filters; CSV export
- **Logs** — Activity log (user actions) and application log (system events) with search and purge - **Logs** — Activity log (user actions) and application log (system events) with search and purge
- **Settings** — SMTP email configuration and Groq AI API key / model selection - **Settings** — SMTP email configuration (host, port, security, username, password, from address, recipients); Groq AI API key / model selection; one-click **Test** button for both SMTP and Groq to verify config without saving
- **AI Criteria Management** — Create, edit, reorder, and deactivate evaluation criteria used by AI analysis - **AI Criteria Management** — Create, edit, reorder, and deactivate evaluation criteria used by AI analysis
- **Bid Email Reminders** — One-click digest email to all admin addresses listing bids due within 7 days
- **Missed Shift Alerting** — Warning card on the admin dashboard showing any user/shift pair with 0 sites checked today
--- ---
@@ -53,30 +61,34 @@ Website Checker helps teams track which government websites need to be checked d
``` ```
webchecker_web/ webchecker_web/
├── app.py # Flask application factory, blueprint registration ├── app.py # Flask application factory, blueprint registration
├── config.py # DB connection pool, schema DDL, get/set settings ├── config.py # DB connection pool, schema DDL, get/set settings
├── models.py # All data-access functions (no ORM) ├── models.py # All data-access functions (no ORM)
├── wsgi.py # Gunicorn entry point ├── wsgi.py # Gunicorn entry point
├── requirements.txt ├── requirements.txt
├── .env.example # Environment variable template ├── .env.example # Environment variable template
├── DEPLOY.md # Full production deployment guide ├── CLAUDE.md # AI developer context
├── DEPLOY.md # Full production deployment guide
│ │
├── routes/ ├── routes/
│ ├── auth.py # Login, logout, change password │ ├── auth.py # Login, logout, change password, profile, password reset
│ ├── admin_dashboard.py # /admin/ │ ├── admin_dashboard.py # /admin/
│ ├── admin_users.py # /admin/users/ │ ├── admin_users.py # /admin/users/
│ ├── admin_websites.py # /admin/websites/ │ ├── admin_websites.py # /admin/websites/
│ ├── admin_shifts.py # /admin/shifts/ │ ├── admin_shifts.py # /admin/shifts/
│ ├── admin_logs.py # /admin/logs/ │ ├── admin_logs.py # /admin/logs/
│ ├── admin_reports.py # /admin/reports/ │ ├── admin_reports.py # /admin/reports/
│ ├── admin_settings.py # /admin/settings/ │ ├── admin_settings.py # /admin/settings/ (incl. test-email, test-groq)
│ ├── user_dashboard.py # /dashboard/ │ ├── user_dashboard.py # /dashboard/ (incl. server-side health check)
│ ├── ai_summary.py # /ai-summary/ │ ├── ai_summary.py # /ai-summary/ (incl. history delete)
│ └── bid_tracker.py # /bids/ │ └── bid_tracker.py # /bids/ (incl. CSV export, email reminders)
│ │
├── templates/ ├── templates/
│ ├── base.html # Sidebar layout, flash messages, nav │ ├── base.html # Sidebar layout, flash messages, nav, mobile toggle
│ ├── login.html │ ├── login.html # Standalone (no base.html)
│ ├── forgot_password.html # Standalone password-reset request
│ ├── reset_password.html # Standalone new-password form
│ ├── profile.html # User profile editor
│ ├── change_password.html │ ├── change_password.html
│ ├── ai_summary.html │ ├── ai_summary.html
│ ├── bid_tracker.html │ ├── bid_tracker.html
@@ -92,12 +104,13 @@ webchecker_web/
│ └── dashboard.html │ └── dashboard.html
│ │
├── static/ ├── static/
│ ├── css/style.css # Full light-theme design system │ ├── css/style.css # Full light-theme design system
│ └── js/app.js # Modal helpers, tabs, session timeout warning │ └── js/app.js # Modal helpers, tabs, timeAgo(), mobile sidebar, session timeout
│ │
└── utils/ └── utils/
├── crypto.py # Fernet encryption (DB-compatible with desktop app) ├── crypto.py # Fernet encryption (DB-compatible with desktop app)
└── decorators.py # @login_required, @admin_required ├── decorators.py # @login_required, @admin_required
└── email.py # SMTP helper: send_email(to, subject, body)
``` ```
--- ---
@@ -109,17 +122,26 @@ webchecker_web/
| `/` | Root redirect (role-based) | Any | | `/` | Root redirect (role-based) | Any |
| `/login` | Login page | Public | | `/login` | Login page | Public |
| `/logout` | Session clear + redirect | Authenticated | | `/logout` | Session clear + redirect | Authenticated |
| `/ping` | Session keepalive (returns 204) | Authenticated |
| `/profile` | Edit own full name and email | Authenticated |
| `/change-password` | Change own password | Authenticated | | `/change-password` | Change own password | Authenticated |
| `/admin/` | Admin dashboard | Admin | | `/forgot-password` | Request password reset email | Public |
| `/admin/users/` | User CRUD | Admin | | `/reset-password/<token>` | Set new password via token | Public |
| `/admin/dashboard` | Admin dashboard with KPIs | Admin |
| `/admin/users/` | User CRUD + send reset link | Admin |
| `/admin/websites/` | Website CRUD + credentials | Admin | | `/admin/websites/` | Website CRUD + credentials | Admin |
| `/admin/shifts/` | Shift CRUD + assignments | Admin | | `/admin/shifts/` | Shift CRUD + calendar view | Admin |
| `/admin/logs/` | Activity & app logs | Admin | | `/admin/logs/` | Activity & app logs | Admin |
| `/admin/reports/` | Shift reports + CSV export | Admin | | `/admin/reports/` | Shift reports + CSV export | Admin |
| `/admin/settings/` | Email + Groq AI settings | Admin | | `/admin/settings/` | Email + Groq AI settings + connection tests | Admin |
| `/dashboard/` | User shift checklist | User | | `/dashboard/` | User shift checklist | User |
| `/ai-summary/` | AI document analysis | Any | | `/dashboard/health/<id>` | Server-side site health check | User |
| `/ai-summary/` | AI document analysis + history | Any |
| `/ai-summary/history/<id>` | Analysis detail (JSON) | Any |
| `/ai-summary/history/<id>/delete` | Delete an analysis record | Any |
| `/bids/` | Bid / opportunity tracker | Any | | `/bids/` | Bid / opportunity tracker | Any |
| `/bids/export.csv` | Export bid list as CSV | Any |
| `/bids/remind` | Send bid deadline reminder email | Admin |
--- ---
@@ -134,12 +156,21 @@ DB_PORT=3306
DB_NAME=webchecker DB_NAME=webchecker
DB_USER=webchecker_user DB_USER=webchecker_user
DB_PASSWORD= DB_PASSWORD=
CRYPTO_SECRET= # Fernet key — generate with: python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
GROQ_API_KEY= # Optional — used by AI Summary feature GROQ_API_KEY= # Optional — can also be set via Admin → Settings
GROQ_MODEL=llama-3.3-70b-versatile # Optional fallback; overridden by DB setting
# SMTP email — optional fallback; values are stored in and overridden by Admin → Settings
SMTP_HOST=
SMTP_PORT=587
SMTP_USER=
SMTP_PASSWORD=
SMTP_FROM=
FLASK_ENV=production FLASK_ENV=production
``` ```
> **Important:** `CRYPTO_SECRET` must match the key used by the desktop app if you are sharing the database. The salt for encryption is stored in the `app_settings` table under `crypto.salt`. > **Important:** `SECRET_KEY` must remain stable across restarts — changing it invalidates all active sessions and CSRF tokens across Gunicorn workers.
--- ---
@@ -149,7 +180,8 @@ This web application shares the MySQL database with the desktop application. Key
- **Encryption** — `utils/crypto.py` uses the exact same key derivation as the desktop (`APP_SECRET = b"WebsiteChecker-v1-CredentialKey"`, 100,000 PBKDF2 iterations, base64 salt, `enc:` prefix). Credentials are interchangeable between apps. - **Encryption** — `utils/crypto.py` uses the exact same key derivation as the desktop (`APP_SECRET = b"WebsiteChecker-v1-CredentialKey"`, 100,000 PBKDF2 iterations, base64 salt, `enc:` prefix). Credentials are interchangeable between apps.
- **`activity_log`** — The live database column is `logged_at` (desktop schema). Queries use `al.logged_at AS created_at` for template compatibility. - **`activity_log`** — The live database column is `logged_at` (desktop schema). Queries use `al.logged_at AS created_at` for template compatibility.
- **`app_settings`** — On startup, `initialize_database()` seeds any settings keys that are blank in the DB from the corresponding `.env` variables (e.g. `GROQ_API_KEY → groq.api_key`), using `ON DUPLICATE KEY UPDATE … IF(value = '', …)` so admin-saved values are never overwritten. - **`app_settings`** — On startup, `initialize_database()` seeds any settings keys that are blank in the DB from the corresponding `.env` variables, using `ON DUPLICATE KEY UPDATE … IF(value = '', …)` so admin-saved values are never overwritten.
- **`password_reset_tokens`** — Web-only table; created by the app's migration-safe DDL on first startup. Tokens expire after 1 hour and are consumed on use. Expired tokens are purged on every successful login.
- **`login_attempts.ip_address`** — The only additive column the web app creates. Added via `ALTER TABLE … ADD COLUMN IF NOT EXISTS` on startup. - **`login_attempts.ip_address`** — The only additive column the web app creates. Added via `ALTER TABLE … ADD COLUMN IF NOT EXISTS` on startup.
--- ---
@@ -178,25 +210,36 @@ python app.py
# Open http://localhost:5000 # Open http://localhost:5000
``` ```
### Restart the production service ### Deploy changes to production
```bash ```bash
sudo systemctl restart webchecker # After copying Python files to the server:
sudo journalctl -u webchecker -f sudo systemctl reload webchecker
# Templates and static files take effect immediately (no reload needed)
# Check for errors after reload:
sudo journalctl -u webchecker -n 30 --no-pager
``` ```
--- ---
## Security Notes ## Security
- Passwords are hashed with bcrypt (12 rounds); legacy SHA-256 hashes from the desktop app are automatically rehashed on next login. - Passwords are hashed with bcrypt (12 rounds); legacy SHA-256 hashes from the desktop app are automatically rehashed on next login.
- Sessions expire after **30 minutes** of inactivity. A client-side warning fires 5 minutes before expiry. - Sessions expire after **30 minutes** of inactivity. A client-side warning fires 5 minutes before expiry with a keep-alive ping endpoint.
- Login is rate-limited: 5 failed attempts locks the account for 15 minutes. - Login is rate-limited: 5 failed attempts locks the account for 15 minutes.
- Credentials stored in `website_credentials` are Fernet-encrypted at rest. Plaintext legacy values (without the `enc:` prefix) are returned as-is for backward compatibility. - Credentials stored in `website_credentials` are Fernet-encrypted at rest.
- `SECRET_KEY` must remain stable across restarts — changing it invalidates all active sessions. - File uploads for AI analysis are validated against magic bytes (PDF `%PDF`, DOCX/XLSX `PK\x03\x04`, DOC/XLS OLE header) and UTF-8 decodability for text files — rejecting disguised uploads without extra dependencies.
- Upload size is capped at 20 MB (`MAX_CONTENT_LENGTH`).
- HTTP security headers (`X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy`) are set on every response.
- CSRF protection (Flask-WTF) on all non-GET requests.
- Session fixation prevention: `session.clear()` is called immediately before setting the session on successful login.
- Password reset tokens are single-use, expire after 1 hour, and are purged on each login.
- Admins cannot demote or deactivate the last remaining active admin.
--- ---
## Deployment ## Deployment
See [`DEPLOY.md`](DEPLOY.md) for the full step-by-step guide covering MySQL setup, Gunicorn systemd service, Nginx reverse proxy, Certbot SSL, firewall rules, log rotation, and database backup. See [`DEPLOY.md`](DEPLOY.md) for the full step-by-step guide covering MySQL setup, Gunicorn systemd service, Nginx reverse proxy, Certbot SSL, firewall rules, log rotation, and database backup.
models.py
+37 -4
View File
@@ -250,6 +250,23 @@ def get_user_by_id(user_id: int):
conn.close() conn.close()
def update_user_profile(user_id: int, full_name: str, email: str):
"""Allow a user to update their own full name and email address."""
conn = None
try:
conn = get_connection()
cur = conn.cursor()
cur.execute(
"UPDATE users SET full_name=%s, email=%s WHERE id=%s",
(full_name.strip() or None, email.strip().lower() or None, user_id),
)
conn.commit()
cur.close()
finally:
if conn:
conn.close()
def create_user(admin_id, username, password, role, full_name, email=None): def create_user(admin_id, username, password, role, full_name, email=None):
conn = None conn = None
try: try:
@@ -1092,12 +1109,28 @@ def get_admin_dashboard_stats():
total_users = cur.fetchone()["n"] total_users = cur.fetchone()["n"]
cur.execute("SELECT COUNT(DISTINCT user_id) AS n FROM shift_checks WHERE DATE(checked_at)=CURDATE()") cur.execute("SELECT COUNT(DISTINCT user_id) AS n FROM shift_checks WHERE DATE(checked_at)=CURDATE()")
active_today = cur.fetchone()["n"] active_today = cur.fetchone()["n"]
cur.execute("SELECT COUNT(*) AS n FROM bid_tracker WHERE status IN ('open','monitoring')")
open_bids = cur.fetchone()["n"]
cur.execute(
"SELECT COUNT(*) AS n FROM bid_tracker"
" WHERE status IN ('open','monitoring')"
" AND due_date BETWEEN CURDATE() AND DATE_ADD(CURDATE(), INTERVAL 7 DAY)"
)
bids_due_soon = cur.fetchone()["n"]
cur.execute(
"SELECT COUNT(*) AS n FROM ai_analysis_log"
" WHERE analyzed_at >= DATE_SUB(NOW(), INTERVAL 30 DAY)"
)
ai_analyses_30d = cur.fetchone()["n"]
cur.close() cur.close()
return { return {
"user_stats": user_stats, "user_stats": user_stats,
"total_sites": total_sites, "total_sites": total_sites,
"total_users": total_users, "total_users": total_users,
"active_today": active_today, "active_today": active_today,
"open_bids": open_bids,
"bids_due_soon": bids_due_soon,
"ai_analyses_30d": ai_analyses_30d,
} }
finally: finally:
if conn: if conn:
routes/admin_settings.py
+1 -1
View File
@@ -30,7 +30,7 @@ def save_email():
fields = [ fields = [
"email.enabled", "email.smtp_host", "email.smtp_port", "email.enabled", "email.smtp_host", "email.smtp_port",
"email.smtp_user", "email.smtp_password", "email.security", "email.smtp_user", "email.smtp_password", "email.security",
"email.recipients", "email.send_time", "email.smtp_from", "email.recipients", "email.send_time",
] ]
for field in fields: for field in fields:
key = field key = field
routes/auth.py
+23 -1
View File
@@ -8,7 +8,7 @@ from models import (
authenticate, check_login_allowed, change_password, log_action, authenticate, check_login_allowed, change_password, log_action,
get_user_by_email, create_password_reset_token, get_user_by_email, create_password_reset_token,
get_password_reset_user, consume_password_reset_token, get_password_reset_user, consume_password_reset_token,
purge_expired_reset_tokens, purge_expired_reset_tokens, update_user_profile,
) )
from utils.decorators import login_required from utils.decorators import login_required
from utils.email import send_email from utils.email import send_email
@@ -107,6 +107,28 @@ def ping():
return "", 204 return "", 204
@auth_bp.route("/profile", methods=["GET", "POST"])
@login_required
def profile():
user = session["user"]
if request.method == "POST":
full_name = request.form.get("full_name", "").strip()
email = request.form.get("email", "").strip()
try:
update_user_profile(user["id"], full_name, email)
session["user"]["full_name"] = full_name or user["username"]
session["user"]["email"] = email
session.modified = True
log_action(user["id"], "UPDATE_PROFILE", "users", user["id"],
"User updated their own profile.")
flash("Profile updated successfully.", "success")
except Exception as e:
logger.error(f"Profile update error for user_id={user['id']}: {e}")
flash(f"Error: {e}", "danger")
return redirect(url_for("auth.profile"))
return render_template("profile.html")
@auth_bp.route("/forgot-password", methods=["GET", "POST"]) @auth_bp.route("/forgot-password", methods=["GET", "POST"])
def forgot_password(): def forgot_password():
if "user" in session: if "user" in session:
templates/admin/dashboard.html
+12
View File
@@ -20,6 +20,18 @@
<div class="kpi-value">{{ stats.total_sites }}</div> <div class="kpi-value">{{ stats.total_sites }}</div>
<div class="kpi-label">Total Sites</div> <div class="kpi-label">Total Sites</div>
</div> </div>
<div class="kpi-card {{ 'kpi-warning' if stats.open_bids else '' }}">
<div class="kpi-value">{{ stats.open_bids }}</div>
<div class="kpi-label">Open Bids</div>
</div>
<div class="kpi-card {{ 'kpi-danger' if stats.bids_due_soon else '' }}">
<div class="kpi-value">{{ stats.bids_due_soon }}</div>
<div class="kpi-label">Due This Week</div>
</div>
<div class="kpi-card">
<div class="kpi-value">{{ stats.ai_analyses_30d }}</div>
<div class="kpi-label">AI Analyses (30d)</div>
</div>
</div> </div>
<!-- Missed shifts alert --> <!-- Missed shifts alert -->
templates/admin/settings.html
+6
View File
@@ -42,6 +42,12 @@
<label class="form-label">SMTP Password</label> <label class="form-label">SMTP Password</label>
<input class="form-control" type="password" name="email_smtp_password" placeholder="(unchanged)"> <input class="form-control" type="password" name="email_smtp_password" placeholder="(unchanged)">
</div> </div>
<div class="form-group">
<label class="form-label">From Address</label>
<input class="form-control" type="email" name="email_smtp_from"
value="{{ email.get('email.smtp_from','') }}"
placeholder="noreply@yourdomain.com">
</div>
<div class="form-group"> <div class="form-group">
<label class="form-label">Recipients (comma-separated)</label> <label class="form-label">Recipients (comma-separated)</label>
<input class="form-control" name="email_recipients" value="{{ email.get('email.recipients','') }}"> <input class="form-control" name="email_recipients" value="{{ email.get('email.recipients','') }}">
templates/admin/websites.html
+31 -2
View File
@@ -3,7 +3,18 @@
{% block content %} {% block content %}
<div class="page-header"> <div class="page-header">
<h1 class="page-title">Website Link Management</h1> <h1 class="page-title">Website Link Management</h1>
<button class="btn btn-primary" onclick="openModal('modal-create-site')">＋ Add Website</button> <div style="display:flex;gap:.5rem;align-items:center">
<div class="search-wrap" style="max-width:220px">
<span class="search-icon">🔍</span>
<input type="text" id="site-search" placeholder="Search name or URL…" autocomplete="off">
</div>
<select id="site-type-filter" class="form-control" style="width:auto">
<option value="">All types</option>
<option value="daily">Daily</option>
<option value="weekly">Weekly</option>
</select>
<button class="btn btn-primary" onclick="openModal('modal-create-site')">＋ Add Website</button>
</div>
</div> </div>
<div class="card"> <div class="card">
@@ -13,7 +24,9 @@
</thead> </thead>
<tbody> <tbody>
{% for w in websites %} {% for w in websites %}
<tr class="{{ 'row-weekly' if w.check_type == 'weekly' else '' }}"> <tr class="{{ 'row-weekly' if w.check_type == 'weekly' else '' }} site-row"
data-name="{{ w.name | lower }}" data-url="{{ w.url | lower }}"
data-type="{{ w.check_type }}">
<td>{{ w.id }}</td> <td>{{ w.id }}</td>
<td><strong>{{ w.name }}</strong></td> <td><strong>{{ w.name }}</strong></td>
<td><span class="badge badge-{{ 'warn' if w.check_type == 'weekly' else 'neutral' }}">{{ w.check_type }}</span></td> <td><span class="badge badge-{{ 'warn' if w.check_type == 'weekly' else 'neutral' }}">{{ w.check_type }}</span></td>
@@ -172,5 +185,21 @@ function addCredRow(btn) {
} }
function removeRow(btn) { btn.closest('.cred-row').remove(); } function removeRow(btn) { btn.closest('.cred-row').remove(); }
function esc(s) { return (s||'').replace(/&/g,'&amp;').replace(/"/g,'&quot;').replace(/</g,'&lt;'); } function esc(s) { return (s||'').replace(/&/g,'&amp;').replace(/"/g,'&quot;').replace(/</g,'&lt;'); }
(function() {
var searchEl = document.getElementById('site-search');
var typeEl = document.getElementById('site-type-filter');
function filterSites() {
var q = searchEl.value.trim().toLowerCase();
var type = typeEl.value;
document.querySelectorAll('.site-row').forEach(function(row) {
var nameMatch = !q || row.dataset.name.includes(q) || row.dataset.url.includes(q);
var typeMatch = !type || row.dataset.type === type;
row.style.display = (nameMatch && typeMatch) ? '' : 'none';
});
}
searchEl.addEventListener('input', filterSites);
typeEl.addEventListener('change', filterSites);
})();
</script> </script>
{% endblock %} {% endblock %}
templates/base.html
+1
View File
@@ -46,6 +46,7 @@
<div class="user-name">{{ current_user.full_name or current_user.username }}</div> <div class="user-name">{{ current_user.full_name or current_user.username }}</div>
<div class="user-role">{{ current_user.role | capitalize }}</div> <div class="user-role">{{ current_user.role | capitalize }}</div>
</div> </div>
<a href="{{ url_for('auth.profile') }}" class="sidebar-btn btn-ghost">👤 My Profile</a>
<a href="{{ url_for('auth.change_password_view') }}" class="sidebar-btn btn-ghost">🔑 Change Password</a> <a href="{{ url_for('auth.change_password_view') }}" class="sidebar-btn btn-ghost">🔑 Change Password</a>
<a href="{{ url_for('auth.logout') }}" class="sidebar-btn btn-danger">⇠ Sign Out</a> <a href="{{ url_for('auth.logout') }}" class="sidebar-btn btn-danger">⇠ Sign Out</a>
{% endif %} {% endif %}
templates/profile.html
+39
View File
@@ -0,0 +1,39 @@
{% extends "base.html" %}
{% block title %}My Profile — Website Checker{% endblock %}
{% block content %}
<div class="page-header">
<h1 class="page-title">My Profile</h1>
</div>
<div style="max-width:480px">
<div class="card">
<div class="card-header"><span class="card-title">Account Details</span></div>
<form method="post" action="{{ url_for('auth.profile') }}">
<div class="modal-body">
<div class="form-group">
<label class="form-label">Username</label>
<input class="form-control" value="{{ session['user']['username'] }}" disabled>
<small class="text-muted">Username cannot be changed. Contact an admin if needed.</small>
</div>
<div class="form-group">
<label class="form-label">Full Name</label>
<input class="form-control" name="full_name"
value="{{ session['user']['full_name'] or '' }}"
placeholder="Your full name">
</div>
<div class="form-group">
<label class="form-label">Email Address</label>
<input class="form-control" type="email" name="email"
value="{{ session['user']['email'] or '' }}"
placeholder="your@email.com">
<small class="text-muted">Used for password resets and notifications.</small>
</div>
</div>
<div class="modal-footer">
<a href="{{ url_for('auth.change_password_view') }}" class="btn btn-ghost">🔑 Change Password</a>
<button type="submit" class="btn btn-primary">Save Changes</button>
</div>
</form>
</div>
</div>
{% endblock %}