04/30 Web Checker web app ver. 1.0
This commit is contained in:
@@ -0,0 +1,254 @@
|
||||
# CLAUDE.md — AI Developer Context
|
||||
|
||||
This file gives Claude (or any AI assistant) the context needed to continue development on this project without re-reading the entire codebase from scratch.
|
||||
|
||||
---
|
||||
|
||||
## What This Project Is
|
||||
|
||||
**Website Checker Web** is a Flask web application that is a direct port of a Tkinter desktop application. Both apps share the **same MySQL database** — this is the single most important constraint. Any schema change, query, or encryption logic must remain compatible with what the desktop app writes and reads.
|
||||
|
||||
The app helps a small team at LT Services Inc. (Falls Church, VA) track government procurement websites across scheduled shifts, manage bid/opportunity follow-up, and run AI-powered solicitation document analysis.
|
||||
|
||||
---
|
||||
|
||||
## Architecture at a Glance
|
||||
|
||||
```
|
||||
Browser → Nginx (reverse proxy) → Gunicorn (4 workers) → Flask app
|
||||
│
|
||||
MySQL (shared with desktop)
|
||||
```
|
||||
|
||||
- **Entry point:** `wsgi.py` → `app.py::create_app()`
|
||||
- **No ORM** — all DB access is raw SQL via `mysql-connector-python` in `models.py`
|
||||
- **No frontend framework** — vanilla JS, no build step, no npm
|
||||
- **Blueprints:** one file per feature area in `routes/`
|
||||
- **Templates:** Jinja2, all extend `base.html`
|
||||
- **Static assets:** single `style.css` + `app.js` — no preprocessor
|
||||
|
||||
---
|
||||
|
||||
## Critical Constraints
|
||||
|
||||
### 1. Shared Database
|
||||
Never rename columns, drop tables, or change column types without verifying the desktop app still works. Key schema facts:
|
||||
- `activity_log` timestamp column is **`logged_at`** (not `created_at`)
|
||||
- `app_log` timestamp column is **`logged_at`**
|
||||
- All other tables generally use `created_at`
|
||||
- The desktop app writes `bid_tracker`, `bid_updates`, `ai_analysis_log`, `ai_criteria`, `app_settings`, `users`, `websites`, `website_credentials`, `shifts`, `shift_users`, `shift_websites`, `shift_checks`, `login_attempts`
|
||||
|
||||
### 2. Credential Encryption
|
||||
`utils/crypto.py` must stay byte-for-byte compatible with the desktop's `utils/crypto.py`:
|
||||
- `_APP_SECRET = b"WebsiteChecker-v1-CredentialKey"` — never change
|
||||
- `_ITERATIONS = 100_000` — never change
|
||||
- Salt stored as **base64** in `app_settings` under key `"crypto.salt"`
|
||||
- Ciphertext has **`enc:`** prefix; values without this prefix are legacy plaintext and returned as-is
|
||||
- Calling `reset_fernet()` forces key reload if the salt changes
|
||||
|
||||
### 3. No Inline JS with Dynamic Jinja Values
|
||||
All button `onclick` handlers that need dynamic data (site ID, site name, note text) **must use `data-*` attributes** on the HTML element and read them in a delegated event listener. Direct `onclick="fn({{ value }})"` breaks when the value contains quotes, apostrophes, or backslashes.
|
||||
|
||||
Example of the correct pattern:
|
||||
```html
|
||||
<button class="js-check" data-id="{{ site.id }}" data-name="{{ site.name }}">Check</button>
|
||||
```
|
||||
```js
|
||||
document.getElementById('list').addEventListener('click', function(e) {
|
||||
const btn = e.target.closest('.js-check');
|
||||
if (btn) openCheckModal(btn.dataset.id, btn.dataset.name);
|
||||
});
|
||||
```
|
||||
|
||||
### 4. Jinja Macros and `{% extends %}`
|
||||
Jinja macros defined in the same file as `{% extends "base.html" %}` cannot be called with `{{ macro_name(...) }}` before the macro definition is reached. **Do not use macros in child templates.** Inline the HTML directly or use JS to build dynamic content.
|
||||
|
||||
### 5. CSS Specificity — Local `<style>` vs `style.css`
|
||||
The global `static/css/style.css` is loaded in `base.html` and has equal or higher specificity than local `<style>` blocks in child templates. Always fix layout issues in `style.css` — do not rely on local `<style>` overrides.
|
||||
|
||||
---
|
||||
|
||||
## Key Files Reference
|
||||
|
||||
| File | Purpose | Notes |
|
||||
|------|---------|-------|
|
||||
| `app.py` | Flask factory | Registers all 11 blueprints; defines `hhmm` template filter for MySQL TIME columns |
|
||||
| `config.py` | DB config, DDL, settings | Calls `load_dotenv()` at top of file — **must be before `DB_CONFIG` dict** |
|
||||
| `models.py` | All DB queries | ~1,430 lines; no ORM; every function opens/closes its own connection |
|
||||
| `utils/crypto.py` | Fernet encryption | Must match desktop exactly |
|
||||
| `utils/decorators.py` | `@login_required`, `@admin_required` | Simple session checks |
|
||||
| `static/css/style.css` | Full design system | Light theme, DM Sans + DM Mono, CSS variables in `:root` |
|
||||
| `static/js/app.js` | Global JS utilities | `openModal()`, `closeModal()`, `copyToClipboard()`, session timeout warning |
|
||||
|
||||
---
|
||||
|
||||
## Session & Auth
|
||||
|
||||
- Session key: `session["user"]` — dict with `id`, `username`, `full_name`, `role`, `email`
|
||||
- Role values: `"admin"` or `"user"`
|
||||
- Session lifetime: 30 minutes (`app.permanent_session_lifetime`)
|
||||
- Login rate limit: 5 attempts → 15-minute lockout (enforced in `models.check_login_allowed`)
|
||||
- `@login_required` — redirects to `/login` if no session
|
||||
- `@admin_required` — redirects to login or 403 if not admin
|
||||
|
||||
---
|
||||
|
||||
## Settings System
|
||||
|
||||
`app_settings` table stores key-value pairs for runtime configuration.
|
||||
|
||||
| `key_name` | `.env` fallback | Used by |
|
||||
|---|---|---|
|
||||
| `groq.api_key` | `GROQ_API_KEY` | AI Summary route |
|
||||
| `groq.model` | `GROQ_MODEL` | AI Summary route |
|
||||
| `email.smtp_host` | `SMTP_HOST` | Email reports |
|
||||
| `email.smtp_port` | `SMTP_PORT` | Email reports |
|
||||
| `email.smtp_user` | `SMTP_USER` | Email reports |
|
||||
| `email.smtp_password` | `SMTP_PASSWORD` | Email reports |
|
||||
| `email.smtp_from` | `SMTP_FROM` | Email reports |
|
||||
| `crypto.salt` | *(generated)* | Fernet key derivation |
|
||||
|
||||
`get_setting(key, default)` reads DB first, then `.env` fallback, then `default`.
|
||||
`set_setting(key, value)` upserts into `app_settings`.
|
||||
On startup, `initialize_database()` seeds blank DB rows from `.env` using `ON DUPLICATE KEY UPDATE value = IF(value='', VALUES(value), value)`.
|
||||
|
||||
---
|
||||
|
||||
## AI Summary Feature
|
||||
|
||||
Route: `/ai-summary/` (`routes/ai_summary.py`)
|
||||
|
||||
- **Stage 1 prompt** (`_EXTRACTION_PROMPT`) — extracts 12 fields per document including driving distance from `2815 Hartland Road, Falls Church, VA 22043` to the work site
|
||||
- **Stage 2 prompt** (`_CRITERIA_PROMPT_SUFFIX`) — appended only when active criteria exist; produces a machine-readable `RECOMMENDATION: PURSUE|PASS|UNCLEAR` line
|
||||
- **File extraction** — `pypdf` for PDF, `python-docx` (imported as `docx`) for DOCX/DOC, `openpyxl` for XLSX, plain decode for TXT/CSV/MD
|
||||
- **API call** — direct `requests.post()` to `https://api.groq.com/openai/v1/chat/completions` — the `groq` Python SDK is **not** installed
|
||||
- **Model** — `claude-sonnet-4-20250514` should NOT be used here; use `llama-3.3-70b-versatile` (Groq)
|
||||
- **Max tokens** — 4,096; **temperature** — 0.2
|
||||
|
||||
---
|
||||
|
||||
## Bid Tracker Feature
|
||||
|
||||
Route: `/bids/` (`routes/bid_tracker.py`)
|
||||
|
||||
Split-pane UI — left: filterable bid list, right: detail panel with update timeline.
|
||||
|
||||
Key AJAX endpoints (all return JSON, no page reload):
|
||||
- `GET /bids/list/json?status=` — paginated bid list
|
||||
- `GET /bids/<id>/json` — bid detail + updates + `can_edit` flag
|
||||
- `POST /bids/<id>/updates/json` — post new update
|
||||
- `POST /bids/updates/<id>/delete/json` — delete update
|
||||
|
||||
Ownership rule: users can only edit/delete their own bids. Admins can edit/delete any bid. This is enforced in both the route (`can_edit = is_admin or bid.added_by == user.id`) and the rendered detail panel.
|
||||
|
||||
---
|
||||
|
||||
## User Dashboard
|
||||
|
||||
Route: `/dashboard/` (`routes/user_dashboard.py`)
|
||||
|
||||
- Sites are grouped by `check_type` (`daily` / `weekly` / other) and rendered as collapsible groups
|
||||
- Each site card has two rows: `.sc-row1` (main flex row) and `.sc-row2` (note row, only if `user_note` is set)
|
||||
- `.sc-actions` uses `margin-left:auto` to push buttons to the right edge of `.sc-row1`
|
||||
- `.site-card` is `display:block` — **not flex** — so `.sc-row1` and `.sc-row2` stack vertically
|
||||
- Health dots probe site reachability using Google's favicon service (cross-origin safe)
|
||||
- The `hhmm` Jinja filter (`app.py`) handles MySQL `TIME` columns returned as `datetime.timedelta`
|
||||
|
||||
---
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Adding a new admin page
|
||||
1. Create `routes/admin_mypage.py` with a Blueprint at `/admin/mypage`
|
||||
2. Register it in `app.py::create_app()`
|
||||
3. Add a nav link in `templates/base.html` inside the admin nav section
|
||||
4. Create `templates/admin/mypage.html` extending `base.html`
|
||||
5. Add model functions to `models.py`
|
||||
6. Add any new CSS classes to `static/css/style.css`
|
||||
|
||||
### Adding a new modal
|
||||
```html
|
||||
<div class="modal-overlay" id="modal-mymodal">
|
||||
<div class="modal"> <!-- or modal-dialog for admin pages -->
|
||||
<div class="modal-header">
|
||||
<span class="modal-title">Title</span>
|
||||
<button class="modal-close" onclick="closeModal('modal-mymodal')">✕</button>
|
||||
</div>
|
||||
<div class="modal-body">…</div>
|
||||
<div class="modal-footer">
|
||||
<button class="btn btn-secondary" onclick="closeModal('modal-mymodal')">Cancel</button>
|
||||
<button class="btn btn-primary">Save</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
```
|
||||
Open with `openModal('modal-mymodal')` from `app.js`.
|
||||
|
||||
### Serialising DB rows to JSON
|
||||
MySQL connector returns `datetime`, `date`, `timedelta` objects which are not JSON-serialisable. Always convert:
|
||||
```python
|
||||
def ser(row):
|
||||
return {k: v.isoformat() if hasattr(v, 'isoformat') else v for k, v in row.items()}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Known Gotchas
|
||||
|
||||
| Gotcha | Detail |
|
||||
|--------|--------|
|
||||
| `activity_log.logged_at` | Column is `logged_at` in the production DB, not `created_at`. Queried as `al.logged_at AS created_at` |
|
||||
| `TIME` columns → `timedelta` | MySQL returns `TIME` as `datetime.timedelta`. Use the `\|hhmm` filter in templates |
|
||||
| `GROQ_API_KEY` in `.env` | Read directly via `os.environ.get("GROQ_API_KEY")` in the analyze route — not via `get_setting()` alone — because `get_setting()` depends on the DB row being populated |
|
||||
| `div.modal` vs `.modal-overlay` | All modal backdrops must use `class="modal-overlay"`. Inner dialog boxes use `class="modal"` or `class="modal-dialog"`. Never use `div.modal` as a backdrop |
|
||||
| Jinja macros in child templates | Cannot call `{{ macro_name() }}` before the `{% macro %}` definition is parsed. Inline the HTML instead |
|
||||
| `INSERT IGNORE` for settings seed | Use `ON DUPLICATE KEY UPDATE value = IF(value='', VALUES(value), value)` — `INSERT IGNORE` silently skips rows that already exist, even with empty values |
|
||||
| `bcrypt` not in original requirements | Added as `bcrypt==4.1.3`. Required by `models.py` for password hashing |
|
||||
|
||||
---
|
||||
|
||||
## Development Workflow
|
||||
|
||||
```bash
|
||||
# Activate venv
|
||||
source /home/webchecker/venv/bin/activate
|
||||
|
||||
# After changing Python files — reload Gunicorn (zero-downtime)
|
||||
sudo systemctl reload webchecker
|
||||
|
||||
# After changing templates or static files — no restart needed (served live)
|
||||
|
||||
# View live logs
|
||||
sudo journalctl -u webchecker -f
|
||||
|
||||
# View Nginx errors
|
||||
sudo tail -f /var/log/nginx/webchecker_error.log
|
||||
|
||||
# Run a quick DB query
|
||||
mysql -u webchecker_user -p webchecker -e "SELECT key_name, value FROM app_settings;"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File Placement on Server
|
||||
|
||||
```
|
||||
/opt/webchecker/ ← project root (or /home/webchecker/app/)
|
||||
/opt/webchecker/.env ← secrets (chmod 600, owned by webchecker)
|
||||
/opt/webchecker/venv/ ← Python virtual environment
|
||||
/var/log/webchecker/ ← Gunicorn access.log + error.log
|
||||
/run/webchecker/ ← Gunicorn UNIX socket (webchecker.sock)
|
||||
/etc/systemd/system/webchecker.service
|
||||
/etc/nginx/sites-available/webchecker
|
||||
```
|
||||
|
||||
## Change Philosophy
|
||||
|
||||
1. **Surgical, additive patches** — smallest possible change to achieve the goal
|
||||
2. **Preserve all routes, function names, variable names** unless explicitly directed otherwise
|
||||
3. **Never remove existing functionality** unless explicitly directed
|
||||
4. **Log all create/update/delete actions** via `log_action()`
|
||||
5. **Migration existence checks** — all migrations safe to re-run
|
||||
6. **Full file contents for 1–3 file changes**; deployment map for larger changesets
|
||||
7. **Explicit deploy instructions** — migration steps separated from code steps
|
||||
8. **Root cause analysis** on errors — never apply temporary workarounds
|
||||
Reference in New Issue
Block a user