Files

T

nngo 58c6218c14 04/23 Enhance app functionalities 2

2026-04-23 17:32:44 -04:00

20 KiB

Raw Blame History

CLAUDE.md — Website Checker: Complete Technical Knowledge Base

This document captures the full architecture, design decisions, database schema, module inventory, feature set, and operational notes for the Website Checker desktop application.

1. Application Overview

Website Checker is a Python/Tkinter desktop application for shift-based website monitoring. Regular users log in, view websites assigned to their shift, open each site, and mark it checked. Administrators manage users, websites, shifts, credentials, and run reports.

UI: Python tkinter + ttk
Database: Remote MySQL 5.7+ / MariaDB 10.3+ via mysql-connector-python
Entry point: app.py → class App(tk.Tk)
Default window: 1100×700, minimum 960×620
Default theme: Light (toggleable to Dark)

2. Project File Structure

website_checker/
├── app.py                          Entry point, shell, navigation, session management
├── config.py                       DB config, connection pool, schema DDL, migrations
├── models.py                       All database access (data layer)
├── requirements.txt
├── CLAUDE.md                       This file
├── utils/
│   ├── config_crypto.py            Windows DPAPI encryption for config.ini credentials
│   ├── crypto.py                   Fernet credential encryption (website passwords)
│   ├── export.py                   CSV + Excel export
│   ├── scheduler.py                Daily email report background daemon
│   └── ui_helpers.py               ThemeManager, COLOURS proxy, DateEntry, widgets
└── views/
    ├── admin_dashboard_view.py     Admin home — KPI cards + per-user progress table
    ├── admin_log_view.py           Activity log treeview
    ├── admin_shifts_view.py        Shift CRUD + PDF export
    ├── admin_users_view.py         User CRUD
    ├── admin_websites_view.py      Website CRUD + visibility + credentials (collapsible)
    ├── ai_summary_view.py          AI document analysis via Groq API (NEW)
    ├── change_password_view.py     Self-service password change (all roles)
    ├── email_settings_view.py      SMTP / scheduled report configuration
    ├── login_view.py               Login form with rate-limiting countdown
    ├── reports_view.py             Shift Detail / Unchecked / Summary / Chart tabs
    ├── settings_view.py            DB connection settings dialog
    └── user_dashboard_view.py      User checklist — search, bulk check, notifications

3. Runtime Files

`config.ini`

Created on first launch. Contains [database], [email], [crypto], and [groq] sections. Sensitive fields are DPAPI-encrypted (see §6 Security). Encrypted values have a dpapi: prefix.

[database]
host=your-mysql-host
port=3306
database=website_checker
user=dpapi:<base64-blob>        ; encrypted
password=dpapi:<base64-blob>    ; encrypted

[email]
enabled=false
smtp_host=smtp.example.com
smtp_port=587
smtp_user=sender@example.com
smtp_password=dpapi:<base64-blob>   ; encrypted
use_tls=true
recipients=admin@example.com
send_time=18:00

[groq]
api_key=dpapi:<base64-blob>     ; encrypted
model=llama-3.3-70b-versatile

[crypto]
salt=<base64 32-byte salt — auto-generated>

`app.log`

UTF-8 log file in working directory. All INFO/WARNING/ERROR from every module.

4. Database Schema

All tables use InnoDB/utf8mb4. Created by initialize_database() on first launch. Safe ALTER TABLE migrations run automatically for columns added post-deployment.

users

Column	Type	Notes
id	INT PK AUTO
username	VARCHAR(100) UNIQUE
password	VARCHAR(255)	bcrypt hash. Legacy SHA-256 (64 hex) auto-migrated on login
role	ENUM('admin','user')
full_name	VARCHAR(200)
is_active	TINYINT(1)
failed_attempts	TINYINT UNSIGNED	Incremented on bad login; reset on success
locked_until	DATETIME NULL	Set when failed_attempts >= MAX_FAILED_ATTEMPTS (5)
created_at / updated_at	DATETIME

websites

Column	Type	Notes
id	INT PK AUTO
name	VARCHAR(200)
url	TEXT
check_type	ENUM('daily','weekly')	weekly = once per ISO week
visibility	ENUM('all','assigned')	assigned = only users in website_users table
note	TEXT	Shown on user dashboard card
is_active	TINYINT(1)	Soft-delete
created_by	INT FK users SET NULL

website_credentials

Column	Notes
website_id FK
username	Plaintext
password	Fernet-encrypted with `enc:` prefix
label	e.g. "Admin", "Read-only"

shift_checks

One row per user per site per day (upserted). user_note optional.

activity_log

action codes: LOGIN, LOGOUT, SESSION_TIMEOUT, ACCOUNT_LOCKED, CREATE/UPDATE/DELETE_USER, CHANGE_PASSWORD, CHANGE_PASSWORD_FAIL, RESET_PASSWORD, CREATE/UPDATE/DELETE_WEBSITE, ADD/REMOVE_CREDENTIAL, CREATE/UPDATE/DELETE_SHIFT, CHECK_WEBSITE, UPDATE_NOTE, EXPORT_CSV, EXPORT_EXCEL, EXPORT_SHIFT_PDF, UPDATE_EMAIL_SETTINGS, CREATE/UPDATE/DELETE_AI_CRITERION.

shifts

days_of_week is a digit string using MySQL DAYOFWEEK: 1=Sun 2=Mon 3=Tue 4=Wed 5=Thu 6=Fri 7=Sat. E.g. "23456" = Mon–Fri. Queried with LOCATE(DAYOFWEEK(CURDATE()), days_of_week) > 0.

shift_users (junction): (shift_id, user_id) PK, both CASCADE

shift_websites (junction): (shift_id, website_id) PK + sort_order

login_attempts: username, attempted_at, ip_address (indexed)

website_users (junction): (website_id, user_id) PK — for visibility='assigned'

ai_criteria

Stores evaluation criteria used by the AI to assess opportunity alignment.

Column	Type	Notes
id	INT PK AUTO
title	VARCHAR(200)	Short label
description	TEXT	Full criterion text sent to the AI
is_active	TINYINT(1)	Inactive criteria excluded from AI prompt
sort_order	INT	Controls display and prompt ordering
created_by	INT FK users SET NULL

ai_analysis_log

Persists every AI analysis result for history and audit purposes.

Column	Type	Notes
id	INT PK AUTO
user_id	INT FK users SET NULL
file_names	TEXT	Comma-separated filenames analyzed
model	VARCHAR(100)	Groq model used
verdict	ENUM('PURSUE','PASS','UNCLEAR') NULL	NULL when no criteria active
criteria_snapshot	TEXT NULL	Active criteria text at time of analysis
summary_text	MEDIUMTEXT	Full AI response
analyzed_at	DATETIME

5. Module Details

config.py

DB_CONFIG loaded from config.ini at import; placeholders if file missing
get_connection() → pool (pool_size=5)
initialize_database() → idempotent DDL + ALTER TABLE migrations
load_config() / save_config() → decrypt/encrypt DPAPI fields transparently
migrate_plaintext_config() → one-time migration; encrypts any plain-text creds on startup
config_exists() → True if host, database, user are set
reload_db_config() → re-reads config.ini, resets pool

utils/config_crypto.py (NEW)

Windows DPAPI-based encryption for config.ini sensitive values.

encrypt_value(plaintext) → "dpapi:" string
decrypt_value(stored) → plaintext; plain-text pass-through for legacy values
is_encrypted(value) → True if value starts with "dpapi:"
Tied to the current Windows user account — blob is unreadable on any other machine/account
Graceful fallback: if pywin32 not available, values stored/returned as plain text

models.py

Each function opens+closes its own connection. All writes call log_action().

Password helpers:

_hash_password(pw) → bcrypt rounds=12
_verify_password(pw, stored) → handles bcrypt + legacy SHA-256
_needs_rehash(stored) → True for 64-char hex (SHA-256)

Rate-limit constants: MAX_FAILED_ATTEMPTS=5, LOCKOUT_MINUTES=15 Password strength: PW_MIN_LENGTH=8, requires upper + digit + special char

utils/crypto.py

PBKDF2-HMAC-SHA256, 100,000 iterations, 32-byte salt from config.ini [crypto]
encrypt(plaintext) → "enc:" + base64(Fernet token)
decrypt(ciphertext) → plaintext; legacy plaintext (no enc: prefix) passes through unchanged
reset_fernet() → force key reload after config.ini replacement

utils/ui_helpers.py

ThemeManager singleton — initial="light". toggle(rebuild_callback) flips theme.
COLOURS = _ColourProxy(dict) — always delegates to ThemeManager.get(). Import once, always current.
THEMES["dark"] and THEMES["light"] each have 18 colour keys + 4 cal_* keys
DateEntry — Frame subclass; .get() → "YYYY-MM-DD", .set(str). Calendar popup with prev/next month+year.
make_scrollable_frame() — mousewheel scoped to Enter/Leave to prevent stale-widget crashes
scrolled_text(parent, height, width) → (frame, tk.Text)

utils/scheduler.py

Daemon thread; polls every 60 seconds
Sends HTML email once per day when now >= send_time
smtp_password is DPAPI-encrypted on save (encrypt_value) / decrypted on load (decrypt_value)
last_sent_date in-memory (resets on restart)
start() / stop() called from app.py on login/logout (admin only)

views/ai_summary_view.py

AI-powered document analysis panel. Accessible from the sidebar for both admin and user roles.

Key internals:

Supported file types: .txt, .md, .csv, .pdf, .docx, .doc, .xlsx, .xls
Groq API key and model selection are visible only to admins (user role sees neither)
Regular users use the stored API key transparently
Background thread for AI calls (prevents UI freeze)
Extraction prompt focused on procurement/solicitation fields: Solicitation Number/Type, Set-Aside, Description, Work Site, Pre-Proposal Conference, POC, Square Footage, Driving Distance from office (2815 Hartland Rd, Falls Church VA), Last Day for Questions, Due Date
Overall summary covers: Scope of Work, Contract Period, Proposal Submission Requirements, Key Deadlines & Action Items
Groq API key stored DPAPI-encrypted in config.ini [groq] section
max_tokens=4096; temperature=0.2; MAX_CHARS_PER_FILE=14,000
Evaluation Criteria panel (collapsible): admin can add/edit/delete criteria; users see read-only list. Active criteria are appended to the AI prompt as an alignment evaluation section. The AI outputs RECOMMENDATION: PURSUE/PASS/UNCLEAR which is parsed and displayed as a colour-coded verdict banner.
History tab: every analysis is saved to ai_analysis_log (with verdict + criteria snapshot). History treeview is filterable by verdict. Double-click restores full output in the Analyze tab. Admins see all users; regular users see only their own.
CriterionDialog has a live character counter with 500-char soft limit guidance.
URL open and health-check probe both guarded by _validate_and_normalise_url.

.doc reading (legacy binary Word format) — 3-tier fallback:

win32com (Word COM automation — requires MS Word installed)
docx2txt (pure Python)
Raw ASCII scrape from binary

views/admin_websites_view.py

Website CRUD dialog now has a collapsible credentials section (hidden by default)
"🔑 Show Credentials" / "🔒 Hide Credentials" toggle button
Auto-expands when editing a site that already has saved credentials
"＋ Add Credential" auto-expands the section if collapsed
Bug fixed: _on_visibility_change previously used before=self.creds_container which crosses a widget parent boundary causing a TclError; corrected to before=self._cred_body

views/user_dashboard_view.py

Key internals:

_health_cache: {website_id: ("ok"|"slow"|"down", ms)} — daemon threads, HEAD request, 6s timeout
Dot colours: green (<=3000ms), amber (>3000ms), red (error)
Mousewheel: Enter/Leave scoped; unbind_all in destroy()
Keyboard shortcuts: self.bind() stored in _shortcut_ids; unbound in destroy()
Notifications: after(60_000) loop; plyer first, fallback to borderless Toplevel toast
Credentials popup no longer opens automatically on link click
"🔑 Credentials" button appears on each site card only if the site has saved credentials
CredentialsPopup: 📋 copy button for username (2s flash); 📋 copy button for password (2s flash + clipboard auto-cleared after 15s for security); 👁 toggle to reveal password

views/login_view.py

check_login_allowed(username) called before authenticate()
Locked: form disabled; 1-second countdown; re-enables when timer reaches 0
Not-yet-locked failed attempt: shows "N attempt(s) remaining before lockout"

views/admin_shifts_view.py

_export_pdf(): reportlab A4 document; one section per active shift; user+website tables

views/reports_view.py

All date fields use DateEntry (no manual text entry)
Chart: matplotlib Agg + FigureCanvasTkAgg; bars green>=100% / amber>=50% / red<50%

6. Authentication & Security

Concern	Implementation
Password hashing	bcrypt rounds=12; SHA-256 auto-rehashed on next login
Website credential encryption	Fernet (AES-128-CBC + HMAC) via cryptography library
Config credential protection	Windows DPAPI (CryptProtectData) — user-account-scoped
Encrypted fields	DB user, DB password, SMTP password, Groq API key
Login rate limiting	5 attempts → 15-min lockout in DB
Session timeout	30-min idle; 1-min warning; any mouse/key event resets
Password policy	8+ chars, uppercase, digit, special character
Self-service change	Requires current password; strength meter; same-as-current guard
Clipboard security	Password copy auto-clears clipboard after 15 seconds

DPAPI Migration

migrate_plaintext_config() is called automatically on every startup (in _init_db). It is a no-op if all sensitive fields are already encrypted (dpapi: prefix present). On first run after this feature was added, it encrypts any existing plain-text values in-place.

7. Key Business Logic

check_type

daily: shown every shift day
weekly: hidden once checked this week (YEARWEEK ISO); reappears Monday

visibility

all: every user in shift sees it
assigned: only users in website_users table see it

get_today_checks resolution

Check if user has active shifts today (DAYOFWEEK match)
If yes: union of shift websites filtered by check_type + visibility
If no: all active websites filtered by visibility + check_type (legacy fallback)

Soft deletes

Websites and Shifts: is_active=0. Users: hard-delete (admin can deactivate first).

8. Configuration Constants (edit in source)

File	Constant	Default
app.py	IDLE_TIMEOUT_MS	1,800,000 (30 min)
app.py	IDLE_WARNING_MS	60,000 (1 min)
app.py	APP_VERSION	"1.0.0"
app.py	VERSION_CHECK_ENABLED	False
models.py	MAX_FAILED_ATTEMPTS	5
models.py	LOCKOUT_MINUTES	15
models.py	PW_MIN_LENGTH	8
user_dashboard_view.py	NOTIFY_MINUTES_BEFORE	15
admin_dashboard_view.py	REFRESH_INTERVAL_MS	60,000
utils/crypto.py	_ITERATIONS	100,000
ai_summary_view.py	MAX_CHARS_PER_FILE	14,000
ai_summary_view.py	_OFFICE_ADDRESS	"2815 Hartland Road, Falls Church, VA 22043, USA"
ai_summary_view.py	CriterionDialog._DESC_SOFT_LIMIT	500 (chars, soft guidance only)
app.py	App._IDLE_THROTTLE_S	5.0 (seconds between Motion-event timer resets)
admin_users_view.py	_PasswordResetDialog._AUTO_CLOSE_S	120 (seconds before auto-close)
admin_users_view.py	_PasswordResetDialog._CLIP_CLEAR_S	30 (seconds before clipboard wipe)

9. Critical Gotchas

bind_all("") is NEVER used at module level. Always Enter/Leave scoped. Every view's destroy() calls unbind_all("") and _unbind_shortcuts().
COLOURS is a live proxy — delegates to ThemeManager.get() on every access. Never snapshot it into a local variable at class-creation time.
bcrypt is slow by design (~200-400ms at rounds=12). Expected behaviour.
config.ini sensitive fields are DPAPI-encrypted (dpapi: prefix). Website credential passwords are Fernet-encrypted (enc: prefix). Non-sensitive fields (host, port, db name, smtp_host, recipients) remain plain text.
get_today_checks GROUP BY includes sc.id to prevent row collisions when a user belongs to multiple shifts sharing the same website.
Weekly site logic uses YEARWEEK(..., 1) (ISO week, Monday start).
The email scheduler last_sent_date is now persisted to config.ini [email] last_sent_date (ISO format). A restart after the configured send_time will not re-send the report on the same day.
All log messages use ASCII only (hyphens not em-dashes/arrows) to prevent cp1252 UnicodeEncodeError on Windows consoles.
DPAPI encrypted blobs are tied to the Windows user account that created them. If config.ini is copied to a different machine or user account, credentials cannot be decrypted. Users must re-enter credentials via the Settings dialog.
The AI summary view fetches credentials from the DB at card render time (not lazily). Avoid having hundreds of sites with credentials as this adds DB round-trips per render.

10. Dependencies

mysql-connector-python>=8.0.0
bcrypt>=4.0.0
openpyxl>=3.1.0
cryptography>=41.0.0
matplotlib>=3.7.0
plyer>=2.1.0
reportlab>=4.0.0
groq>=1.0.0
pypdf>=3.0.0
python-docx>=1.0.0
pywin32>=306
docx2txt>=0.8

stdlib used: tkinter, csv, smtplib, urllib.request, configparser, threading, calendar, datetime, base64, re, tempfile

11. First-Run Flow

_boot() → config_exists() → False → SettingsView (locked modal)
User enters DB creds → Test Connection → Save & Connect
reload_db_config() → initialize_database() → seeds admin/admin123 if users empty
migrate_plaintext_config() → encrypts any plain-text credentials in config.ini
Login screen shown
ThemeManager(root, initial="light") applied; shell built

Admin role

Dashboard · Websites · Users · Shifts · Reports · Activity Log · 🤖 AI Summary · Settings · Sign Out

User role

My Shift · 🤖 AI Summary · Change Password · Sign Out

13. Deployment Notes

Python 3.9+ required. 3.12 tested on Windows.
tkinter bundled on Windows/macOS; Linux: apt install python3-tk
Create DB first: CREATE DATABASE website_checker CHARACTER SET utf8mb4;
MySQL port 3306 must be reachable from client
Default admin: username=admin / password=admin123 — change immediately
pywin32 required for DPAPI encryption (Windows only). Install: pip install pywin32
Microsoft Word recommended for .doc file support in AI Summary (falls back to docx2txt)
Groq API key required for AI Summary feature — free at https://console.groq.com

14. AI Summary Feature — Setup & Prompt Details

Setup (Admin)

Obtain a free Groq API key at https://console.groq.com
Navigate to 🤖 AI Summary in the sidebar
Enter the API key and select a model
Click 💾 Save Settings — key is stored DPAPI-encrypted in config.ini

Supported Models

llama-3.3-70b-versatile (default, recommended)
llama-3.1-8b-instant
gemma2-9b-it
mixtral-8x7b-32768

Prompt Strategy

The prompt instructs the AI to extract 12 labeled fields per document: Solicitation Number, Type, Set-Aside, Description/Scope, Work Site, Pre-Proposal Conference, POC, Square Footage, Driving Distance from 2815 Hartland Rd Falls Church VA (calculated by AI), Last Day for Questions, Due Date, Other requirements.

Then produces an Overall Summary with 4 sections: A. Scope of Work · B. Contract Period · C. Proposal Submission Requirements · D. Key Deadlines

The driving distance/time is an AI estimate using major highways. Actual times may vary with traffic.

20 KiB Raw Blame History Unescape Escape