05/24 Enhance functionalities 2
This commit is contained in:
@@ -108,6 +108,7 @@ Without this, sessions and CSRF tokens are broken across the 4 Gunicorn workers.
|
||||
| `templates/login.html` | Standalone login page | Does NOT extend `base.html`; has its own `<head>` and no `app.js`; CSRF token must be a direct hidden input |
|
||||
| `templates/forgot_password.html` | Standalone forgot-password page | Same constraints as `login.html` — standalone, direct CSRF hidden input, no `app.js` |
|
||||
| `templates/reset_password.html` | Standalone reset-password page | Same constraints as `login.html` — standalone, direct CSRF hidden input, no `app.js` |
|
||||
| `templates/profile.html` | User profile editor | Extends `base.html`; users can update `full_name` and `email`; username/role are read-only |
|
||||
|
||||
---
|
||||
|
||||
@@ -121,6 +122,8 @@ Without this, sessions and CSRF tokens are broken across the 4 Gunicorn workers.
|
||||
- `@admin_required` — redirects to login or 403 if not admin
|
||||
- **Session fixation prevention** — `session.clear()` is called in `auth.login` immediately before setting `session["user"]` on successful authentication
|
||||
- **Keep-alive endpoint** — `GET /ping` (`routes/auth.py`, `@login_required`) touches `session.modified = True` and returns 204; called by the session-timeout warning in `app.js`
|
||||
- **Password reset** — `POST /forgot-password` creates a 1-hour `password_reset_tokens` token and emails a link. `POST /reset-password/<token>` validates, sets the new password, and deletes the token. Expired tokens are purged on every successful login via `purge_expired_reset_tokens()`.
|
||||
- **Profile editing** — `GET/POST /profile` lets any authenticated user update their `full_name` and `email`. Changes are immediately reflected in `session["user"]` so the sidebar name refreshes without re-login.
|
||||
|
||||
---
|
||||
|
||||
@@ -149,14 +152,19 @@ On startup, `initialize_database()` seeds blank DB rows from `.env` using `ON DU
|
||||
|
||||
Route: `/ai-summary/` (`routes/ai_summary.py`)
|
||||
|
||||
- **Stage 1 prompt** (`_EXTRACTION_PROMPT`) — extracts 12 fields per document including driving distance from `2815 Hartland Road, Falls Church, VA 22043` to the work site
|
||||
- **Stage 1 prompt** (`_EXTRACTION_PROMPT`) — extracts 12 fields; treats all uploaded documents as one combined source (do NOT analyze per-file); driving distance from `2815 Hartland Road, Falls Church, VA 22043`
|
||||
- **Stage 2 prompt** (`_CRITERIA_PROMPT_SUFFIX`) — appended only when active criteria exist; produces a machine-readable `RECOMMENDATION: PURSUE|PASS|UNCLEAR` line
|
||||
- **File extraction** — `pypdf` for PDF, `python-docx` (imported as `docx`) for DOCX/DOC, `openpyxl` for XLSX, plain decode for TXT/CSV/MD
|
||||
- **Magic-byte validation** — `_validate_magic()` rejects files whose bytes don't match their declared extension (disguised uploads)
|
||||
- **API call** — direct `requests.post()` to `https://api.groq.com/openai/v1/chat/completions` — the `groq` Python SDK is **not** installed
|
||||
- **Model** — `claude-sonnet-4-20250514` should NOT be used here; use `llama-3.3-70b-versatile` (Groq)
|
||||
- **Max tokens** — 4,096; **temperature** — 0.2
|
||||
- **Text limit** — combined document text is truncated to 14,000 characters before being sent; the JSON response includes `"truncated": true` when this occurs so the UI can warn the user
|
||||
- **Text limit** — combined document text is truncated to 14,000 characters before being sent; the JSON response includes `"truncated": true` when this occurs
|
||||
- **`sort_order` input** — always wrap `int(request.form.get("sort_order", 0) or 0)` in `try/except (ValueError, TypeError)` — bad input must degrade gracefully to 0, not raise a 500
|
||||
- **Output rendering** — `marked.js` (CDN) renders the AI summary as formatted markdown in both the live output and history modal
|
||||
- **History detail** — `GET /ai-summary/history/<id>` returns `criteria_snapshot` (JSON string of `[{title, description}]`); the history modal parses and displays the criteria titles used
|
||||
- **Delete history** — `POST /ai-summary/history/<id>/delete` enforces ownership (user can delete own; admin can delete any); logs the action
|
||||
- **Verdict filter** — client-side pill buttons (All / PURSUE / PASS / UNCLEAR / No verdict) above the history table toggle row visibility
|
||||
|
||||
---
|
||||
|
||||
@@ -188,7 +196,7 @@ Route: `/dashboard/` (`routes/user_dashboard.py`)
|
||||
- Each site card has two rows: `.sc-row1` (main flex row) and `.sc-row2` (note row, only if `user_note` is set)
|
||||
- `.sc-actions` uses `margin-left:auto` to push buttons to the right edge of `.sc-row1`
|
||||
- `.site-card` is `display:block` — **not flex** — so `.sc-row1` and `.sc-row2` stack vertically
|
||||
- Health dots probe site reachability using Google's favicon service (cross-origin safe)
|
||||
- **Health dots** — `GET /dashboard/health/<id>` does a server-side HEAD request (6s timeout); returns `{status, ms}` JSON. Colors: green <3000ms, orange slow/timeout, red unreachable, gray error
|
||||
- The `hhmm` Jinja filter (`app.py`) handles MySQL `TIME` columns returned as `datetime.timedelta`
|
||||
|
||||
---
|
||||
@@ -320,25 +328,43 @@ mysql -u webchecker_user -p webchecker -e "SELECT key_name, value FROM app_setti
|
||||
|
||||
---
|
||||
|
||||
## Pending Improvements (To-Do)
|
||||
## Completed Improvements
|
||||
|
||||
All planned items have been implemented. The list below serves as a record and cross-reference.
|
||||
|
||||
### Security
|
||||
- [x] **File upload size cap** — `MAX_CONTENT_LENGTH = 20 MB` set in `create_app()`; `RequestEntityTooLarge` handler returns JSON for `/ai-summary/` paths and flash+redirect for form routes
|
||||
- [x] **HTTP security headers** — `@app.after_request` in `app.py` sets `X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` on every response
|
||||
- [x] **Prevent last-admin demotion/deactivation** — `update_user()` in `models.py` checks that demoting or deactivating an admin won't leave zero active admins; raises `ValueError` surfaced as a flash message
|
||||
- [x] **File MIME type validation** — `_validate_magic()` in `ai_summary.py` checks magic bytes for PDF (`%PDF`), DOCX/XLSX (`PK\x03\x04`), DOC/XLS (OLE header), and UTF-8 decodability for text types; no extra dependency needed
|
||||
- [x] File upload size cap (`MAX_CONTENT_LENGTH = 20 MB`); JSON error for `/ai-summary/`, flash+redirect elsewhere
|
||||
- [x] HTTP security headers — `X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` on every response
|
||||
- [x] Prevent last-admin demotion/deactivation — `update_user()` raises `ValueError` if zero active admins would remain
|
||||
- [x] File MIME type validation — `_validate_magic()` checks magic bytes and UTF-8 decodability; no extra dependency
|
||||
- [x] Session fixation prevention — `session.clear()` before setting session on login
|
||||
- [x] Password reset tokens — single-use, 1-hour expiry, purged on every successful login
|
||||
|
||||
### UI / UX
|
||||
- [ ] **Mobile-responsive sidebar** — No media queries exist; the sidebar + main-content layout is unusable on phones; implement a hamburger toggle that collapses the sidebar on small screens
|
||||
- [ ] **Bid due-date urgency badges** — Client-side only: highlight bids due within 7 days with a warning badge and bids past due with a red "Overdue" pill in the bid list
|
||||
- [ ] **Relative timestamps in Activity Log** — Format raw `YYYY-MM-DD HH:MM:SS` timestamps as "2 hours ago" / "3 days ago" using a small JS formatter
|
||||
- [ ] **Empty state for AI analysis history** — When no analyses exist, show a call-to-action ("Upload your first document ↑") instead of a blank panel
|
||||
- [ ] **Paginate bid list** — `get_all_bids()` fetches all rows with no limit; add `LIMIT`/`OFFSET` to the model query and a "load more" button in the split-pane list
|
||||
- [x] Mobile-responsive sidebar — hamburger toggle (`sidebar-toggle` button in `base.html`), CSS `transform:translateX`, backdrop overlay; IIFE in `app.js`
|
||||
- [x] Bid due-date urgency badges — "Overdue" (red) and "Due Soon" (yellow) client-side pills in bid list
|
||||
- [x] Relative timestamps in Activity Log — `timeAgo()` in `app.js`; `[data-ts]` attribute on timestamp `<span>` elements in `logs.html`
|
||||
- [x] Empty state for AI analysis history — robot emoji + "Start Analyzing" CTA when history is empty
|
||||
- [x] Paginate bid list — `LIMIT`/`OFFSET` in `get_all_bids()`; "Load more" button in bid list; `_bidOffset` / `_PAGE_SIZE` state vars
|
||||
- [x] AI history verdict filter — pill buttons (All / PURSUE / PASS / UNCLEAR / No verdict) filter rows client-side
|
||||
- [x] Highlight today in shift calendar — JS IIFE maps `getDay()` → MySQL DAYOFWEEK → column index; adds `.cal-today` CSS class
|
||||
- [x] Website list search + type filter — inline search input + dropdown filter client-side in `admin/websites.html`
|
||||
|
||||
### Functionality
|
||||
- [x] **Missed-shift alerting** — `get_missed_shifts_today()` in `models.py` queries shift+user pairs scheduled today with 0 checks; surfaced as a warning card on the admin dashboard
|
||||
- [x] **Bid deadline email reminders** — `get_bids_due_soon()` + `get_admin_emails()` in `models.py`; `send_reminders` route in `bid_tracker.py`; "📧 Remind" button in bid tracker toolbar (admin only); uses `utils/email.py`
|
||||
- [x] **Server-side health checks** — `/dashboard/health/<id>` in `user_dashboard.py` does a server-side HEAD request; user dashboard JS updated to call this instead of the Google favicon proxy
|
||||
- [x] **Password reset via email** — `password_reset_tokens` table (migration-safe DDL in `config.py`); model functions in `models.py`; `/forgot-password` and `/reset-password/<token>` routes in `auth.py`; standalone templates `forgot_password.html` and `reset_password.html`; "Forgot password?" link on login page; uses `utils/email.py`
|
||||
- [x] **"Copy password" button in Credentials modal** — Already implemented in the original code via `data-copy` attribute and delegated click handler in `user/dashboard.html`
|
||||
- [x] **Shift calendar view** — Weekly grid tab added to admin shifts page; Mon–Sun columns, active shifts as rows; server-side rendered with Jinja2 using existing `day_map` data
|
||||
- [x] Missed-shift alerting — `get_missed_shifts_today()` in `models.py`; warning card on admin dashboard
|
||||
- [x] Bid deadline email reminders — `send_reminders` route; "📧 Remind" button (admin only) in bid toolbar
|
||||
- [x] Server-side health checks — `/dashboard/health/<id>` HEAD request with 6s timeout; green/orange/red/gray dots
|
||||
- [x] Password reset via email — `password_reset_tokens` table; `/forgot-password` + `/reset-password/<token>` routes; standalone templates; "Forgot password?" on login page
|
||||
- [x] Shift calendar view — weekly grid tab; Mon–Sun columns; active shifts as rows; `day_map` server-rendered
|
||||
- [x] Delete AI analysis history — `POST /ai-summary/history/<id>/delete`; ownership check; delete button in history modal
|
||||
- [x] Criteria snapshot in history modal — `analysis_detail` returns `criteria_snapshot`; modal shows titles of criteria used
|
||||
- [x] Purge expired password reset tokens — `purge_expired_reset_tokens()` called on every successful login (wrapped in try/except)
|
||||
- [x] Test SMTP + Groq buttons — `POST /admin/settings/test-email` and `/test-groq`; inline AJAX result in settings page
|
||||
- [x] Admin send password reset link — `POST /admin/users/<id>/send-reset`; generates token + sends email; "✉ Reset" button per user row
|
||||
- [x] Export bids to CSV — `GET /bids/export.csv`; respects active status filter; "⬇ CSV" button in toolbar
|
||||
- [x] Admin dashboard bid KPIs — Open Bids, Due This Week (red when >0), AI Analyses (30d) added to `get_admin_dashboard_stats()`
|
||||
- [x] User profile page — `GET/POST /profile`; users edit own `full_name` and `email`; session updated immediately; "👤 My Profile" in sidebar
|
||||
- [x] SMTP From Address field — `email.smtp_from` exposed in Admin → Settings form and saved alongside other email fields
|
||||
|
||||
### Pending
|
||||
- No known pending items. Add new items here as they are identified.
|
||||
|
||||
@@ -141,8 +141,18 @@ DB_PORT=3306
|
||||
DB_NAME=webchecker
|
||||
DB_USER=webchecker_user
|
||||
DB_PASSWORD=YourStrongPasswordHere
|
||||
CRYPTO_SECRET=<output of: python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())">
|
||||
FLASK_ENV=production
|
||||
|
||||
# AI analysis (can also be set via Admin → Settings after first login)
|
||||
GROQ_API_KEY=
|
||||
GROQ_MODEL=llama-3.3-70b-versatile
|
||||
|
||||
# SMTP email for password reset and bid reminders (can also be set via Admin → Settings)
|
||||
SMTP_HOST=
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=
|
||||
SMTP_PASSWORD=
|
||||
SMTP_FROM=noreply@yourdomain.com
|
||||
```
|
||||
|
||||
Secure the file so only the application user can read it:
|
||||
@@ -392,7 +402,15 @@ Open a browser and navigate to `https://your-domain.com`. You should see the log
|
||||
|
||||
### 9.4 First login
|
||||
|
||||
Log in with the default admin credentials set during `initialize_database()`. **Change the password immediately** via Admin → Change Password.
|
||||
Log in with the default admin credentials set during `initialize_database()`. **Change the password immediately** via Admin → Users (edit your user) or via My Profile → Change Password.
|
||||
|
||||
### 9.5 Configure email and AI settings
|
||||
|
||||
Navigate to **Admin → Settings** and fill in:
|
||||
- **Email** — SMTP host, port, security mode, username, password, from address, and recipient list. Click **📬 Test** to verify without leaving the page.
|
||||
- **Groq AI** — paste your API key and select a model. Click **🔌 Test** to confirm the key works.
|
||||
|
||||
These values are stored in the database and take precedence over `.env` fallbacks.
|
||||
|
||||
---
|
||||
|
||||
@@ -472,6 +490,9 @@ Create `/etc/logrotate.d/webchecker`:
|
||||
| Sessions expiring immediately | `SECRET_KEY` changed between restarts | Keep `SECRET_KEY` stable in `.env`; never regenerate it on a live system |
|
||||
| `413 Request Entity Too Large` | File upload exceeds `client_max_body_size` | Increase `client_max_body_size` in the Nginx config |
|
||||
| Static files returning 404 | Wrong `alias` path in Nginx | Confirm `/opt/webchecker/static/` exists and the `alias` directive ends with `/` |
|
||||
| Password reset emails not delivered | SMTP not configured or wrong credentials | Go to Admin → Settings, check SMTP fields, click **📬 Test** — the error message shows the SMTP failure reason |
|
||||
| Groq AI analysis fails with 401 | Invalid or missing API key | Go to Admin → Settings → Groq, paste a valid key, click **🔌 Test** to confirm before saving |
|
||||
| `BuildError` for a new route after deploy | Gunicorn still running old bytecode | Run `sudo systemctl reload webchecker` — templates go live immediately but Python changes require a reload |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@ Website Checker helps teams track which government websites need to be checked d
|
||||
| Credential Encryption | Fernet (PBKDF2-HMAC-SHA256, 100k iterations) |
|
||||
| Password Hashing | bcrypt (12 rounds) |
|
||||
| AI Analysis | Groq REST API (llama-3.3-70b-versatile) |
|
||||
| Markdown Rendering | marked.js (CDN, client-side) |
|
||||
| Frontend | Vanilla JS + DM Sans/DM Mono (Google Fonts) |
|
||||
| OS | Ubuntu 22.04 / 24.04 LTS |
|
||||
|
||||
@@ -31,21 +32,28 @@ Website Checker helps teams track which government websites need to be checked d
|
||||
## Features
|
||||
|
||||
### All Users
|
||||
- **Shift Checklist** — Sites grouped by Daily / Weekly frequency, collapsible groups, live progress bar, bulk-check, search filter
|
||||
- **Shift Checklist** — Sites grouped by Daily / Weekly frequency, collapsible groups, live progress bar, bulk-check, search filter, server-side health dot per site
|
||||
- **Site Credentials** — View stored credentials (password masked, copy-to-clipboard)
|
||||
- **Per-site Notes** — Add or update notes per check, displayed in a dedicated row under the site card
|
||||
- **AI Document Analysis** — Upload PDF / DOCX / XLSX / TXT solicitation files; AI extracts solicitation number, scope of work, due dates, driving distance from office, and evaluates against criteria
|
||||
- **Bid Tracker** — Split-pane view of all tracked opportunities; post updates, filter by status, search by title / source / solicitation number
|
||||
- **AI Document Analysis** — Upload PDF / DOCX / XLSX / TXT solicitation files; AI extracts solicitation number, scope of work, due dates, driving distance from office, and evaluates against active criteria. All documents are analyzed as one combined source. Output rendered as formatted markdown.
|
||||
- **AI History** — Filter past analyses by verdict (PURSUE / PASS / UNCLEAR / No verdict), view criteria snapshot used, delete own records
|
||||
- **Bid Tracker** — Split-pane view of all tracked opportunities; post updates, filter by status, search by title / source / solicitation number, urgency badges for bids due within 7 days / overdue, load-more pagination, export current view to CSV
|
||||
- **My Profile** — Update own full name and email address
|
||||
- **Password Reset** — Self-service reset via emailed link (1-hour token); "Forgot password?" on login page
|
||||
- **Relative Timestamps** — Activity log timestamps shown as "2 hours ago" / "3 days ago"
|
||||
- **Mobile Sidebar** — Hamburger toggle collapses sidebar on small screens
|
||||
|
||||
### Admins Only
|
||||
- **Dashboard** — KPI cards (users, active today, total sites) and per-user completion progress
|
||||
- **User Management** — Create, edit, activate/deactivate users; role assignment (admin / user)
|
||||
- **Website Management** — CRUD for monitored sites with credentials, check type (daily/weekly), and user assignment
|
||||
- **Shift Management** — Define shifts with days-of-week, time windows, assigned users and sites
|
||||
- **Dashboard** — KPI cards (users, active today, total sites, open bids, bids due this week, AI analyses past 30 days); per-user completion progress; missed-shift warning
|
||||
- **User Management** — Create, edit, activate/deactivate users; role assignment; send password reset link directly to any user with an email address
|
||||
- **Website Management** — CRUD for monitored sites with credentials, check type (daily/weekly), user assignment; inline search and type filter
|
||||
- **Shift Management** — Define shifts with days-of-week, time windows, assigned users and sites; weekly calendar view with today's column highlighted
|
||||
- **Reports** — Shift detail, unchecked sites, and summary reports with date/user/site filters; CSV export
|
||||
- **Logs** — Activity log (user actions) and application log (system events) with search and purge
|
||||
- **Settings** — SMTP email configuration and Groq AI API key / model selection
|
||||
- **Settings** — SMTP email configuration (host, port, security, username, password, from address, recipients); Groq AI API key / model selection; one-click **Test** button for both SMTP and Groq to verify config without saving
|
||||
- **AI Criteria Management** — Create, edit, reorder, and deactivate evaluation criteria used by AI analysis
|
||||
- **Bid Email Reminders** — One-click digest email to all admin addresses listing bids due within 7 days
|
||||
- **Missed Shift Alerting** — Warning card on the admin dashboard showing any user/shift pair with 0 sites checked today
|
||||
|
||||
---
|
||||
|
||||
@@ -53,30 +61,34 @@ Website Checker helps teams track which government websites need to be checked d
|
||||
|
||||
```
|
||||
webchecker_web/
|
||||
├── app.py # Flask application factory, blueprint registration
|
||||
├── config.py # DB connection pool, schema DDL, get/set settings
|
||||
├── models.py # All data-access functions (no ORM)
|
||||
├── wsgi.py # Gunicorn entry point
|
||||
├── app.py # Flask application factory, blueprint registration
|
||||
├── config.py # DB connection pool, schema DDL, get/set settings
|
||||
├── models.py # All data-access functions (no ORM)
|
||||
├── wsgi.py # Gunicorn entry point
|
||||
├── requirements.txt
|
||||
├── .env.example # Environment variable template
|
||||
├── DEPLOY.md # Full production deployment guide
|
||||
├── .env.example # Environment variable template
|
||||
├── CLAUDE.md # AI developer context
|
||||
├── DEPLOY.md # Full production deployment guide
|
||||
│
|
||||
├── routes/
|
||||
│ ├── auth.py # Login, logout, change password
|
||||
│ ├── admin_dashboard.py # /admin/
|
||||
│ ├── admin_users.py # /admin/users/
|
||||
│ ├── admin_websites.py # /admin/websites/
|
||||
│ ├── admin_shifts.py # /admin/shifts/
|
||||
│ ├── admin_logs.py # /admin/logs/
|
||||
│ ├── admin_reports.py # /admin/reports/
|
||||
│ ├── admin_settings.py # /admin/settings/
|
||||
│ ├── user_dashboard.py # /dashboard/
|
||||
│ ├── ai_summary.py # /ai-summary/
|
||||
│ └── bid_tracker.py # /bids/
|
||||
│ ├── auth.py # Login, logout, change password, profile, password reset
|
||||
│ ├── admin_dashboard.py # /admin/
|
||||
│ ├── admin_users.py # /admin/users/
|
||||
│ ├── admin_websites.py # /admin/websites/
|
||||
│ ├── admin_shifts.py # /admin/shifts/
|
||||
│ ├── admin_logs.py # /admin/logs/
|
||||
│ ├── admin_reports.py # /admin/reports/
|
||||
│ ├── admin_settings.py # /admin/settings/ (incl. test-email, test-groq)
|
||||
│ ├── user_dashboard.py # /dashboard/ (incl. server-side health check)
|
||||
│ ├── ai_summary.py # /ai-summary/ (incl. history delete)
|
||||
│ └── bid_tracker.py # /bids/ (incl. CSV export, email reminders)
|
||||
│
|
||||
├── templates/
|
||||
│ ├── base.html # Sidebar layout, flash messages, nav
|
||||
│ ├── login.html
|
||||
│ ├── base.html # Sidebar layout, flash messages, nav, mobile toggle
|
||||
│ ├── login.html # Standalone (no base.html)
|
||||
│ ├── forgot_password.html # Standalone password-reset request
|
||||
│ ├── reset_password.html # Standalone new-password form
|
||||
│ ├── profile.html # User profile editor
|
||||
│ ├── change_password.html
|
||||
│ ├── ai_summary.html
|
||||
│ ├── bid_tracker.html
|
||||
@@ -92,12 +104,13 @@ webchecker_web/
|
||||
│ └── dashboard.html
|
||||
│
|
||||
├── static/
|
||||
│ ├── css/style.css # Full light-theme design system
|
||||
│ └── js/app.js # Modal helpers, tabs, session timeout warning
|
||||
│ ├── css/style.css # Full light-theme design system
|
||||
│ └── js/app.js # Modal helpers, tabs, timeAgo(), mobile sidebar, session timeout
|
||||
│
|
||||
└── utils/
|
||||
├── crypto.py # Fernet encryption (DB-compatible with desktop app)
|
||||
└── decorators.py # @login_required, @admin_required
|
||||
├── crypto.py # Fernet encryption (DB-compatible with desktop app)
|
||||
├── decorators.py # @login_required, @admin_required
|
||||
└── email.py # SMTP helper: send_email(to, subject, body)
|
||||
```
|
||||
|
||||
---
|
||||
@@ -109,17 +122,26 @@ webchecker_web/
|
||||
| `/` | Root redirect (role-based) | Any |
|
||||
| `/login` | Login page | Public |
|
||||
| `/logout` | Session clear + redirect | Authenticated |
|
||||
| `/ping` | Session keepalive (returns 204) | Authenticated |
|
||||
| `/profile` | Edit own full name and email | Authenticated |
|
||||
| `/change-password` | Change own password | Authenticated |
|
||||
| `/admin/` | Admin dashboard | Admin |
|
||||
| `/admin/users/` | User CRUD | Admin |
|
||||
| `/forgot-password` | Request password reset email | Public |
|
||||
| `/reset-password/<token>` | Set new password via token | Public |
|
||||
| `/admin/dashboard` | Admin dashboard with KPIs | Admin |
|
||||
| `/admin/users/` | User CRUD + send reset link | Admin |
|
||||
| `/admin/websites/` | Website CRUD + credentials | Admin |
|
||||
| `/admin/shifts/` | Shift CRUD + assignments | Admin |
|
||||
| `/admin/shifts/` | Shift CRUD + calendar view | Admin |
|
||||
| `/admin/logs/` | Activity & app logs | Admin |
|
||||
| `/admin/reports/` | Shift reports + CSV export | Admin |
|
||||
| `/admin/settings/` | Email + Groq AI settings | Admin |
|
||||
| `/admin/settings/` | Email + Groq AI settings + connection tests | Admin |
|
||||
| `/dashboard/` | User shift checklist | User |
|
||||
| `/ai-summary/` | AI document analysis | Any |
|
||||
| `/dashboard/health/<id>` | Server-side site health check | User |
|
||||
| `/ai-summary/` | AI document analysis + history | Any |
|
||||
| `/ai-summary/history/<id>` | Analysis detail (JSON) | Any |
|
||||
| `/ai-summary/history/<id>/delete` | Delete an analysis record | Any |
|
||||
| `/bids/` | Bid / opportunity tracker | Any |
|
||||
| `/bids/export.csv` | Export bid list as CSV | Any |
|
||||
| `/bids/remind` | Send bid deadline reminder email | Admin |
|
||||
|
||||
---
|
||||
|
||||
@@ -134,12 +156,21 @@ DB_PORT=3306
|
||||
DB_NAME=webchecker
|
||||
DB_USER=webchecker_user
|
||||
DB_PASSWORD=
|
||||
CRYPTO_SECRET= # Fernet key — generate with: python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
|
||||
GROQ_API_KEY= # Optional — used by AI Summary feature
|
||||
|
||||
GROQ_API_KEY= # Optional — can also be set via Admin → Settings
|
||||
GROQ_MODEL=llama-3.3-70b-versatile # Optional fallback; overridden by DB setting
|
||||
|
||||
# SMTP email — optional fallback; values are stored in and overridden by Admin → Settings
|
||||
SMTP_HOST=
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=
|
||||
SMTP_PASSWORD=
|
||||
SMTP_FROM=
|
||||
|
||||
FLASK_ENV=production
|
||||
```
|
||||
|
||||
> **Important:** `CRYPTO_SECRET` must match the key used by the desktop app if you are sharing the database. The salt for encryption is stored in the `app_settings` table under `crypto.salt`.
|
||||
> **Important:** `SECRET_KEY` must remain stable across restarts — changing it invalidates all active sessions and CSRF tokens across Gunicorn workers.
|
||||
|
||||
---
|
||||
|
||||
@@ -149,7 +180,8 @@ This web application shares the MySQL database with the desktop application. Key
|
||||
|
||||
- **Encryption** — `utils/crypto.py` uses the exact same key derivation as the desktop (`APP_SECRET = b"WebsiteChecker-v1-CredentialKey"`, 100,000 PBKDF2 iterations, base64 salt, `enc:` prefix). Credentials are interchangeable between apps.
|
||||
- **`activity_log`** — The live database column is `logged_at` (desktop schema). Queries use `al.logged_at AS created_at` for template compatibility.
|
||||
- **`app_settings`** — On startup, `initialize_database()` seeds any settings keys that are blank in the DB from the corresponding `.env` variables (e.g. `GROQ_API_KEY → groq.api_key`), using `ON DUPLICATE KEY UPDATE … IF(value = '', …)` so admin-saved values are never overwritten.
|
||||
- **`app_settings`** — On startup, `initialize_database()` seeds any settings keys that are blank in the DB from the corresponding `.env` variables, using `ON DUPLICATE KEY UPDATE … IF(value = '', …)` so admin-saved values are never overwritten.
|
||||
- **`password_reset_tokens`** — Web-only table; created by the app's migration-safe DDL on first startup. Tokens expire after 1 hour and are consumed on use. Expired tokens are purged on every successful login.
|
||||
- **`login_attempts.ip_address`** — The only additive column the web app creates. Added via `ALTER TABLE … ADD COLUMN IF NOT EXISTS` on startup.
|
||||
|
||||
---
|
||||
@@ -178,25 +210,36 @@ python app.py
|
||||
# Open http://localhost:5000
|
||||
```
|
||||
|
||||
### Restart the production service
|
||||
### Deploy changes to production
|
||||
|
||||
```bash
|
||||
sudo systemctl restart webchecker
|
||||
sudo journalctl -u webchecker -f
|
||||
# After copying Python files to the server:
|
||||
sudo systemctl reload webchecker
|
||||
|
||||
# Templates and static files take effect immediately (no reload needed)
|
||||
|
||||
# Check for errors after reload:
|
||||
sudo journalctl -u webchecker -n 30 --no-pager
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Notes
|
||||
## Security
|
||||
|
||||
- Passwords are hashed with bcrypt (12 rounds); legacy SHA-256 hashes from the desktop app are automatically rehashed on next login.
|
||||
- Sessions expire after **30 minutes** of inactivity. A client-side warning fires 5 minutes before expiry.
|
||||
- Sessions expire after **30 minutes** of inactivity. A client-side warning fires 5 minutes before expiry with a keep-alive ping endpoint.
|
||||
- Login is rate-limited: 5 failed attempts locks the account for 15 minutes.
|
||||
- Credentials stored in `website_credentials` are Fernet-encrypted at rest. Plaintext legacy values (without the `enc:` prefix) are returned as-is for backward compatibility.
|
||||
- `SECRET_KEY` must remain stable across restarts — changing it invalidates all active sessions.
|
||||
- Credentials stored in `website_credentials` are Fernet-encrypted at rest.
|
||||
- File uploads for AI analysis are validated against magic bytes (PDF `%PDF`, DOCX/XLSX `PK\x03\x04`, DOC/XLS OLE header) and UTF-8 decodability for text files — rejecting disguised uploads without extra dependencies.
|
||||
- Upload size is capped at 20 MB (`MAX_CONTENT_LENGTH`).
|
||||
- HTTP security headers (`X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy`) are set on every response.
|
||||
- CSRF protection (Flask-WTF) on all non-GET requests.
|
||||
- Session fixation prevention: `session.clear()` is called immediately before setting the session on successful login.
|
||||
- Password reset tokens are single-use, expire after 1 hour, and are purged on each login.
|
||||
- Admins cannot demote or deactivate the last remaining active admin.
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
See [`DEPLOY.md`](DEPLOY.md) for the full step-by-step guide covering MySQL setup, Gunicorn systemd service, Nginx reverse proxy, Certbot SSL, firewall rules, log rotation, and database backup.
|
||||
See [`DEPLOY.md`](DEPLOY.md) for the full step-by-step guide covering MySQL setup, Gunicorn systemd service, Nginx reverse proxy, Certbot SSL, firewall rules, log rotation, and database backup.
|
||||
|
||||
@@ -250,6 +250,23 @@ def get_user_by_id(user_id: int):
|
||||
conn.close()
|
||||
|
||||
|
||||
def update_user_profile(user_id: int, full_name: str, email: str):
|
||||
"""Allow a user to update their own full name and email address."""
|
||||
conn = None
|
||||
try:
|
||||
conn = get_connection()
|
||||
cur = conn.cursor()
|
||||
cur.execute(
|
||||
"UPDATE users SET full_name=%s, email=%s WHERE id=%s",
|
||||
(full_name.strip() or None, email.strip().lower() or None, user_id),
|
||||
)
|
||||
conn.commit()
|
||||
cur.close()
|
||||
finally:
|
||||
if conn:
|
||||
conn.close()
|
||||
|
||||
|
||||
def create_user(admin_id, username, password, role, full_name, email=None):
|
||||
conn = None
|
||||
try:
|
||||
@@ -1092,12 +1109,28 @@ def get_admin_dashboard_stats():
|
||||
total_users = cur.fetchone()["n"]
|
||||
cur.execute("SELECT COUNT(DISTINCT user_id) AS n FROM shift_checks WHERE DATE(checked_at)=CURDATE()")
|
||||
active_today = cur.fetchone()["n"]
|
||||
cur.execute("SELECT COUNT(*) AS n FROM bid_tracker WHERE status IN ('open','monitoring')")
|
||||
open_bids = cur.fetchone()["n"]
|
||||
cur.execute(
|
||||
"SELECT COUNT(*) AS n FROM bid_tracker"
|
||||
" WHERE status IN ('open','monitoring')"
|
||||
" AND due_date BETWEEN CURDATE() AND DATE_ADD(CURDATE(), INTERVAL 7 DAY)"
|
||||
)
|
||||
bids_due_soon = cur.fetchone()["n"]
|
||||
cur.execute(
|
||||
"SELECT COUNT(*) AS n FROM ai_analysis_log"
|
||||
" WHERE analyzed_at >= DATE_SUB(NOW(), INTERVAL 30 DAY)"
|
||||
)
|
||||
ai_analyses_30d = cur.fetchone()["n"]
|
||||
cur.close()
|
||||
return {
|
||||
"user_stats": user_stats,
|
||||
"total_sites": total_sites,
|
||||
"total_users": total_users,
|
||||
"active_today": active_today,
|
||||
"user_stats": user_stats,
|
||||
"total_sites": total_sites,
|
||||
"total_users": total_users,
|
||||
"active_today": active_today,
|
||||
"open_bids": open_bids,
|
||||
"bids_due_soon": bids_due_soon,
|
||||
"ai_analyses_30d": ai_analyses_30d,
|
||||
}
|
||||
finally:
|
||||
if conn:
|
||||
|
||||
@@ -30,7 +30,7 @@ def save_email():
|
||||
fields = [
|
||||
"email.enabled", "email.smtp_host", "email.smtp_port",
|
||||
"email.smtp_user", "email.smtp_password", "email.security",
|
||||
"email.recipients", "email.send_time",
|
||||
"email.smtp_from", "email.recipients", "email.send_time",
|
||||
]
|
||||
for field in fields:
|
||||
key = field
|
||||
|
||||
+23
-1
@@ -8,7 +8,7 @@ from models import (
|
||||
authenticate, check_login_allowed, change_password, log_action,
|
||||
get_user_by_email, create_password_reset_token,
|
||||
get_password_reset_user, consume_password_reset_token,
|
||||
purge_expired_reset_tokens,
|
||||
purge_expired_reset_tokens, update_user_profile,
|
||||
)
|
||||
from utils.decorators import login_required
|
||||
from utils.email import send_email
|
||||
@@ -107,6 +107,28 @@ def ping():
|
||||
return "", 204
|
||||
|
||||
|
||||
@auth_bp.route("/profile", methods=["GET", "POST"])
|
||||
@login_required
|
||||
def profile():
|
||||
user = session["user"]
|
||||
if request.method == "POST":
|
||||
full_name = request.form.get("full_name", "").strip()
|
||||
email = request.form.get("email", "").strip()
|
||||
try:
|
||||
update_user_profile(user["id"], full_name, email)
|
||||
session["user"]["full_name"] = full_name or user["username"]
|
||||
session["user"]["email"] = email
|
||||
session.modified = True
|
||||
log_action(user["id"], "UPDATE_PROFILE", "users", user["id"],
|
||||
"User updated their own profile.")
|
||||
flash("Profile updated successfully.", "success")
|
||||
except Exception as e:
|
||||
logger.error(f"Profile update error for user_id={user['id']}: {e}")
|
||||
flash(f"Error: {e}", "danger")
|
||||
return redirect(url_for("auth.profile"))
|
||||
return render_template("profile.html")
|
||||
|
||||
|
||||
@auth_bp.route("/forgot-password", methods=["GET", "POST"])
|
||||
def forgot_password():
|
||||
if "user" in session:
|
||||
|
||||
@@ -20,6 +20,18 @@
|
||||
<div class="kpi-value">{{ stats.total_sites }}</div>
|
||||
<div class="kpi-label">Total Sites</div>
|
||||
</div>
|
||||
<div class="kpi-card {{ 'kpi-warning' if stats.open_bids else '' }}">
|
||||
<div class="kpi-value">{{ stats.open_bids }}</div>
|
||||
<div class="kpi-label">Open Bids</div>
|
||||
</div>
|
||||
<div class="kpi-card {{ 'kpi-danger' if stats.bids_due_soon else '' }}">
|
||||
<div class="kpi-value">{{ stats.bids_due_soon }}</div>
|
||||
<div class="kpi-label">Due This Week</div>
|
||||
</div>
|
||||
<div class="kpi-card">
|
||||
<div class="kpi-value">{{ stats.ai_analyses_30d }}</div>
|
||||
<div class="kpi-label">AI Analyses (30d)</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Missed shifts alert -->
|
||||
|
||||
@@ -42,6 +42,12 @@
|
||||
<label class="form-label">SMTP Password</label>
|
||||
<input class="form-control" type="password" name="email_smtp_password" placeholder="(unchanged)">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="form-label">From Address</label>
|
||||
<input class="form-control" type="email" name="email_smtp_from"
|
||||
value="{{ email.get('email.smtp_from','') }}"
|
||||
placeholder="noreply@yourdomain.com">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="form-label">Recipients (comma-separated)</label>
|
||||
<input class="form-control" name="email_recipients" value="{{ email.get('email.recipients','') }}">
|
||||
|
||||
@@ -3,7 +3,18 @@
|
||||
{% block content %}
|
||||
<div class="page-header">
|
||||
<h1 class="page-title">Website Link Management</h1>
|
||||
<button class="btn btn-primary" onclick="openModal('modal-create-site')">＋ Add Website</button>
|
||||
<div style="display:flex;gap:.5rem;align-items:center">
|
||||
<div class="search-wrap" style="max-width:220px">
|
||||
<span class="search-icon">🔍</span>
|
||||
<input type="text" id="site-search" placeholder="Search name or URL…" autocomplete="off">
|
||||
</div>
|
||||
<select id="site-type-filter" class="form-control" style="width:auto">
|
||||
<option value="">All types</option>
|
||||
<option value="daily">Daily</option>
|
||||
<option value="weekly">Weekly</option>
|
||||
</select>
|
||||
<button class="btn btn-primary" onclick="openModal('modal-create-site')">＋ Add Website</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="card">
|
||||
@@ -13,7 +24,9 @@
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for w in websites %}
|
||||
<tr class="{{ 'row-weekly' if w.check_type == 'weekly' else '' }}">
|
||||
<tr class="{{ 'row-weekly' if w.check_type == 'weekly' else '' }} site-row"
|
||||
data-name="{{ w.name | lower }}" data-url="{{ w.url | lower }}"
|
||||
data-type="{{ w.check_type }}">
|
||||
<td>{{ w.id }}</td>
|
||||
<td><strong>{{ w.name }}</strong></td>
|
||||
<td><span class="badge badge-{{ 'warn' if w.check_type == 'weekly' else 'neutral' }}">{{ w.check_type }}</span></td>
|
||||
@@ -172,5 +185,21 @@ function addCredRow(btn) {
|
||||
}
|
||||
function removeRow(btn) { btn.closest('.cred-row').remove(); }
|
||||
function esc(s) { return (s||'').replace(/&/g,'&').replace(/"/g,'"').replace(/</g,'<'); }
|
||||
|
||||
(function() {
|
||||
var searchEl = document.getElementById('site-search');
|
||||
var typeEl = document.getElementById('site-type-filter');
|
||||
function filterSites() {
|
||||
var q = searchEl.value.trim().toLowerCase();
|
||||
var type = typeEl.value;
|
||||
document.querySelectorAll('.site-row').forEach(function(row) {
|
||||
var nameMatch = !q || row.dataset.name.includes(q) || row.dataset.url.includes(q);
|
||||
var typeMatch = !type || row.dataset.type === type;
|
||||
row.style.display = (nameMatch && typeMatch) ? '' : 'none';
|
||||
});
|
||||
}
|
||||
searchEl.addEventListener('input', filterSites);
|
||||
typeEl.addEventListener('change', filterSites);
|
||||
})();
|
||||
</script>
|
||||
{% endblock %}
|
||||
|
||||
@@ -46,6 +46,7 @@
|
||||
<div class="user-name">{{ current_user.full_name or current_user.username }}</div>
|
||||
<div class="user-role">{{ current_user.role | capitalize }}</div>
|
||||
</div>
|
||||
<a href="{{ url_for('auth.profile') }}" class="sidebar-btn btn-ghost">👤 My Profile</a>
|
||||
<a href="{{ url_for('auth.change_password_view') }}" class="sidebar-btn btn-ghost">🔑 Change Password</a>
|
||||
<a href="{{ url_for('auth.logout') }}" class="sidebar-btn btn-danger">⇠ Sign Out</a>
|
||||
{% endif %}
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
{% extends "base.html" %}
|
||||
{% block title %}My Profile — Website Checker{% endblock %}
|
||||
{% block content %}
|
||||
<div class="page-header">
|
||||
<h1 class="page-title">My Profile</h1>
|
||||
</div>
|
||||
|
||||
<div style="max-width:480px">
|
||||
<div class="card">
|
||||
<div class="card-header"><span class="card-title">Account Details</span></div>
|
||||
<form method="post" action="{{ url_for('auth.profile') }}">
|
||||
<div class="modal-body">
|
||||
<div class="form-group">
|
||||
<label class="form-label">Username</label>
|
||||
<input class="form-control" value="{{ session['user']['username'] }}" disabled>
|
||||
<small class="text-muted">Username cannot be changed. Contact an admin if needed.</small>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="form-label">Full Name</label>
|
||||
<input class="form-control" name="full_name"
|
||||
value="{{ session['user']['full_name'] or '' }}"
|
||||
placeholder="Your full name">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="form-label">Email Address</label>
|
||||
<input class="form-control" type="email" name="email"
|
||||
value="{{ session['user']['email'] or '' }}"
|
||||
placeholder="your@email.com">
|
||||
<small class="text-muted">Used for password resets and notifications.</small>
|
||||
</div>
|
||||
</div>
|
||||
<div class="modal-footer">
|
||||
<a href="{{ url_for('auth.change_password_view') }}" class="btn btn-ghost">🔑 Change Password</a>
|
||||
<button type="submit" class="btn btn-primary">Save Changes</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
Reference in New Issue
Block a user