04/30 Web Checker web app ver. 1.0

2026-04-30 17:15:56 -04:00
parent feb8e5051c
commit cd63d5a020
39 changed files with 8031 additions and 1 deletions
@@ -1,2 +1,202 @@
-# WebChecker--Web-app-
+# Website Checker — Web Application

+A full-stack web application for monitoring government procurement websites across team shifts. Converted from a Tkinter desktop application, sharing the same MySQL database.
+
+---
+
+## Overview
+
+Website Checker helps teams track which government websites need to be checked during each shift, manage bid/procurement opportunities, and run AI-powered document analysis on solicitation files. Administrators manage users, websites, shifts, and view reports; regular users work through their daily or weekly checklists.
+
+---
+
+## Technology Stack
+
+| Layer | Technology |
+|---|---|
+| Language | Python 3.10+ |
+| Web Framework | Flask 3.0 |
+| Database | MySQL 8.x (shared with desktop app) |
+| Application Server | Gunicorn |
+| Reverse Proxy | Nginx |
+| Process Manager | systemd |
+| Credential Encryption | Fernet (PBKDF2-HMAC-SHA256, 100k iterations) |
+| Password Hashing | bcrypt (12 rounds) |
+| AI Analysis | Groq REST API (llama-3.3-70b-versatile) |
+| Frontend | Vanilla JS + DM Sans/DM Mono (Google Fonts) |
+| OS | Ubuntu 22.04 / 24.04 LTS |
+
+---
+
+## Features
+
+### All Users
+- **Shift Checklist** — Sites grouped by Daily / Weekly frequency, collapsible groups, live progress bar, bulk-check, search filter
+- **Site Credentials** — View stored credentials (password masked, copy-to-clipboard)
+- **Per-site Notes** — Add or update notes per check, displayed in a dedicated row under the site card
+- **AI Document Analysis** — Upload PDF / DOCX / XLSX / TXT solicitation files; AI extracts solicitation number, scope of work, due dates, driving distance from office, and evaluates against criteria
+- **Bid Tracker** — Split-pane view of all tracked opportunities; post updates, filter by status, search by title / source / solicitation number
+
+### Admins Only
+- **Dashboard** — KPI cards (users, active today, total sites) and per-user completion progress
+- **User Management** — Create, edit, activate/deactivate users; role assignment (admin / user)
+- **Website Management** — CRUD for monitored sites with credentials, check type (daily/weekly), and user assignment
+- **Shift Management** — Define shifts with days-of-week, time windows, assigned users and sites
+- **Reports** — Shift detail, unchecked sites, and summary reports with date/user/site filters; CSV export
+- **Logs** — Activity log (user actions) and application log (system events) with search and purge
+- **Settings** — SMTP email configuration and Groq AI API key / model selection
+- **AI Criteria Management** — Create, edit, reorder, and deactivate evaluation criteria used by AI analysis
+
+---
+
+## Project Structure
+
+```
+webchecker_web/
+├── app.py                  # Flask application factory, blueprint registration
+├── config.py               # DB connection pool, schema DDL, get/set settings
+├── models.py               # All data-access functions (no ORM)
+├── wsgi.py                 # Gunicorn entry point
+├── requirements.txt
+├── .env.example            # Environment variable template
+├── DEPLOY.md               # Full production deployment guide
+│
+├── routes/
+│   ├── auth.py             # Login, logout, change password
+│   ├── admin_dashboard.py  # /admin/
+│   ├── admin_users.py      # /admin/users/
+│   ├── admin_websites.py   # /admin/websites/
+│   ├── admin_shifts.py     # /admin/shifts/
+│   ├── admin_logs.py       # /admin/logs/
+│   ├── admin_reports.py    # /admin/reports/
+│   ├── admin_settings.py   # /admin/settings/
+│   ├── user_dashboard.py   # /dashboard/
+│   ├── ai_summary.py       # /ai-summary/
+│   └── bid_tracker.py      # /bids/
+│
+├── templates/
+│   ├── base.html           # Sidebar layout, flash messages, nav
+│   ├── login.html
+│   ├── change_password.html
+│   ├── ai_summary.html
+│   ├── bid_tracker.html
+│   ├── admin/
+│   │   ├── dashboard.html
+│   │   ├── users.html
+│   │   ├── websites.html
+│   │   ├── shifts.html
+│   │   ├── logs.html
+│   │   ├── reports.html
+│   │   └── settings.html
+│   └── user/
+│       └── dashboard.html
+│
+├── static/
+│   ├── css/style.css       # Full light-theme design system
+│   └── js/app.js           # Modal helpers, tabs, session timeout warning
+│
+└── utils/
+    ├── crypto.py           # Fernet encryption (DB-compatible with desktop app)
+    └── decorators.py       # @login_required, @admin_required
+```
+
+---
+
+## URL Routes
+
+| Prefix | Description | Access |
+|---|---|---|
+| `/` | Root redirect (role-based) | Any |
+| `/login` | Login page | Public |
+| `/logout` | Session clear + redirect | Authenticated |
+| `/change-password` | Change own password | Authenticated |
+| `/admin/` | Admin dashboard | Admin |
+| `/admin/users/` | User CRUD | Admin |
+| `/admin/websites/` | Website CRUD + credentials | Admin |
+| `/admin/shifts/` | Shift CRUD + assignments | Admin |
+| `/admin/logs/` | Activity & app logs | Admin |
+| `/admin/reports/` | Shift reports + CSV export | Admin |
+| `/admin/settings/` | Email + Groq AI settings | Admin |
+| `/dashboard/` | User shift checklist | User |
+| `/ai-summary/` | AI document analysis | Any |
+| `/bids/` | Bid / opportunity tracker | Any |
+
+---
+
+## Environment Variables
+
+Copy `.env.example` to `.env` and fill in all values:
+
+```dotenv
+SECRET_KEY=          # Flask session secret — generate with: python3 -c "import secrets; print(secrets.token_hex(32))"
+DB_HOST=127.0.0.1
+DB_PORT=3306
+DB_NAME=webchecker
+DB_USER=webchecker_user
+DB_PASSWORD=
+CRYPTO_SECRET=       # Fernet key — generate with: python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
+GROQ_API_KEY=        # Optional — used by AI Summary feature
+FLASK_ENV=production
+```
+
+> **Important:** `CRYPTO_SECRET` must match the key used by the desktop app if you are sharing the database. The salt for encryption is stored in the `app_settings` table under `crypto.salt`.
+
+---
+
+## Database Compatibility
+
+This web application shares the MySQL database with the desktop application. Key compatibility notes:
+
+- **Encryption** — `utils/crypto.py` uses the exact same key derivation as the desktop (`APP_SECRET = b"WebsiteChecker-v1-CredentialKey"`, 100,000 PBKDF2 iterations, base64 salt, `enc:` prefix). Credentials are interchangeable between apps.
+- **`activity_log`** — The live database column is `logged_at` (desktop schema). Queries use `al.logged_at AS created_at` for template compatibility.
+- **`app_settings`** — On startup, `initialize_database()` seeds any settings keys that are blank in the DB from the corresponding `.env` variables (e.g. `GROQ_API_KEY → groq.api_key`), using `ON DUPLICATE KEY UPDATE … IF(value = '', …)` so admin-saved values are never overwritten.
+- **`login_attempts.ip_address`** — The only additive column the web app creates. Added via `ALTER TABLE … ADD COLUMN IF NOT EXISTS` on startup.
+
+---
+
+## Development
+
+### Run locally
+
+```bash
+# 1. Clone and enter the project
+cd webchecker_web
+
+# 2. Create and activate a virtual environment
+python3 -m venv venv
+source venv/bin/activate
+
+# 3. Install dependencies
+pip install -r requirements.txt
+
+# 4. Configure environment
+cp .env.example .env
+# Edit .env with your local MySQL credentials
+
+# 5. Run the development server
+python app.py
+# Open http://localhost:5000
+```
+
+### Restart the production service
+
+```bash
+sudo systemctl restart webchecker
+sudo journalctl -u webchecker -f
+```
+
+---
+
+## Security Notes
+
+- Passwords are hashed with bcrypt (12 rounds); legacy SHA-256 hashes from the desktop app are automatically rehashed on next login.
+- Sessions expire after **30 minutes** of inactivity. A client-side warning fires 5 minutes before expiry.
+- Login is rate-limited: 5 failed attempts locks the account for 15 minutes.
+- Credentials stored in `website_credentials` are Fernet-encrypted at rest. Plaintext legacy values (without the `enc:` prefix) are returned as-is for backward compatibility.
+- `SECRET_KEY` must remain stable across restarts — changing it invalidates all active sessions.
+
+---
+
+## Deployment
+
+See [`DEPLOY.md`](DEPLOY.md) for the full step-by-step guide covering MySQL setup, Gunicorn systemd service, Nginx reverse proxy, Certbot SSL, firewall rules, log rotation, and database backup.