nngo/GOV_QR_Codes_Management
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

05/19 Update doc

Browse Source
This commit is contained in:
nngo
2026-05-19 15:16:30 -04:00
parent 74e0b71a11
commit 023b2fdf46
2 changed files with 949 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
-2
View File
@@ -207,5 +207,3 @@ cython_debug/
marimo/_static/ marimo/_static/
marimo/_lsp/ marimo/_lsp/
__marimo__/ __marimo__/
README.md
Claude.md
Claude.md
+949
View File
@@ -0,0 +1,949 @@
# CLAUDE.md — QR Code Attendance Management System
# Unified reference — LT Services, Inc. & GOV Services, Inc.
This file is the authoritative reference for Claude across all sessions.
Every session must treat this file as ground truth for architecture, conventions,
domain rules, and developer preferences. Read it fully before making any changes.
---
## 1. Project Overview
A Flask/Python web application for **employee attendance tracking via QR codes** with
GPS validation, Excel import/export, payroll calculations, role-based access control,
and per-QR photo verification toggle.
**Deployed for two companies on separate servers:**
- **LT Services, Inc.** — `THEME_NAME` empty (blue theme). Production: `test.ltservicesinc.com`. Gitea: `gitea.ngodanguyen.tech/nngo/LT_QR_Codes_Management`.
- **GOV Services, Inc.** — `THEME_NAME=gov` (green theme). Production: `qr.govservicesinc.com`. Key `.env` differences: `COMPANY_NAME=GOV. Services, Inc`, `THEME_NAME=gov`, `QR_BASE_URL=https://qr.govservicesinc.com`.
**Both codebases are structurally identical** in all Python files, routes, models, and templates.
They differ only in `.env` values and `static/css/theme-gov.css`.
**Every code fix must be applied to both instances.**
- **Stack:** Flask 3.1, SQLAlchemy 2.0 / MySQL (PyMySQL only — `mysql-connector-python` removed), openpyxl 3.1, pandas 2.2, Python 3.12
- **Frontend:** Jinja2 templates + vanilla JS + Font Awesome 6 icons
- **Auth guard:** Cloudflare Turnstile (optional, toggled via `.env`)
- **Production server:** Gunicorn + gevent workers, Nginx reverse proxy, Ubuntu Server
---
## 2. Folder Structure
```
QR_Code_Management/
│
├── app.py # Application factory (create_app) + startup entry point
├── config.py # All env-var reads — single source of truth
├── extensions.py # Shared singletons: db (SQLAlchemy) + logger_handler (AppLogger)
├── logger_handler.py # AppLogger class + log_user_activity / log_database_operations decorators
├── location_logging.py # Android GPS debug routes (/api/log-location-action, /api/location-debug-info)
├── turnstile_utils.py # Cloudflare Turnstile verification helper (TurnstileUtils class)
├── advanced_security_middleware.py # SecurityManager (CSRF token gen, rate limiting, suspicious IP tracking)
│ # Wired into create_app() — do not import separately in blueprints
├── working_hours_calculator.py # CANONICAL calculator for Excel exports (WorkingHoursCalculator)
├── single_checkin_calculator.py # Legacy calculator — NOT used for exports; never switch to this
├── time_attendance_import_service.py # Excel import pipeline with duplicate detection
├── qr_code_import_service.py # Bulk QR code import from Excel
├── address_normalization_fix.py # normalize_address() + addresses_are_similar() helpers
├── app_performance_middleware.py # PerformanceMonitor — dev-mode only; lazy-imported inside __main__
├── db_audit_tables.py # DB audit table helpers (standalone operational script)
├── db_health_check.py # DB connectivity health check (standalone)
├── db_maintenance.py # DB maintenance utilities (standalone)
├── db_performance_optimization.py # Index/query optimization helpers
├── employee_data_merger.py # Merge duplicate employee records (standalone)
├── employee_duplicate_remove.py # Remove employee duplicates (standalone)
├── employee_sync_scheduler.py # Scheduled employee sync (schedule library)
├── employee_table_sync.py # Employee table sync logic
├── requirements.txt # All pinned Python dependencies (single MySQL driver: PyMySQL)
│
├── models/
│ ├── __init__.py # set_db(db) → unpacks and returns all model classes
│ ├── base.py # Shared db reference (base.db); all models import from here
│ ├── user.py # User model (table: users)
│ ├── employee.py # Employee model (table: employee)
│ ├── attendance.py # AttendanceData model (table: attendance_data)
│ ├── time_attendance.py # TimeAttendance model (table: time_attendance)
│ ├── qrcode.py # QRCode, QRCodeStyle, QRCodeLocation models
│ ├── project.py # Project model (table: projects)
│ └── permissions.py # UserProjectPermission, UserLocationPermission models
│
├── routes/
│ ├── __init__.py # Empty (intentional)
│ ├── auth.py # Blueprint 'auth': /, /register (admin-only), /login, /logout, /profile
│ ├── dashboard.py # Blueprint 'dashboard': /dashboard, project QR views, stats APIs
│ ├── users.py # Blueprint 'users': /users/*, user management APIs
│ ├── admin.py # Blueprint 'admin': /admin/logs, /api/logs/*
│ ├── projects.py # Blueprint 'projects': /projects/*
│ ├── qr_codes.py # Blueprint 'qr_codes': /qr-codes/*, /qr/<url> check-in flow
│ ├── attendance.py # Blueprint 'attendance': /attendance report + API endpoints
│ ├── attendance_edit.py # Side-effect module: edit, add manual, save, delete routes
│ │ # Imports bp FROM attendance.py — does NOT define its own blueprint
│ ├── verification.py # Side-effect module: /verification-review/* routes
│ │ # Imports bp FROM attendance.py — does NOT define its own blueprint
│ ├── attendance_export.py # Side-effect module: /export-configuration, /generate-excel-export
│ │ # Imports bp FROM attendance.py — does NOT define its own blueprint
│ ├── statistics.py # Blueprint 'statistics': /statistics
│ ├── employees.py # Blueprint 'employees': /employees/*
│ ├── time_attendance.py # Blueprint 'time_attendance': /time-attendance/* (all TA routes)
│ └── time_attendance_export.py # Plain module (NO Blueprint): export helper functions only
│
├── utils/
│ ├── __init__.py
│ ├── helpers.py # Role decorators, QR generation, role/permission helpers
│ ├── geocoding.py # Haversine distance, Google Maps client, reverse geocode
│ └── template_helpers.py # Context processors: get_employee_name, format_hours, etc.
│
├── static/
│ ├── css/
│ │ ├── style.css # MASTER stylesheet — all CSS variables and layout
│ │ ├── theme-gov.css # GOV Services brand theme overrides (loaded when THEME_NAME=gov)
│ │ └── *.css # Page-specific stylesheets
│ └── js/
│ ├── script.js # Global JS — includes CSRF fetch wrapper
│ ├── qr_destination.js # Check-in page: GPS, bilingual UI, camera — CSRF-EXEMPT
│ ├── attendance_report.js # Attendance report: pagination, sorting, charts
│ ├── dashboard.js
│ ├── export_configuration.js
│ ├── users.js
│ ├── attendance_fullscreen.js
│ └── android_location_handler.js # Android-specific GPS workaround
│
├── templates/
│ ├── base.html # Base for unauthenticated pages — loads theme CSS conditionally
│ ├── base_authenticated.html # Base for protected pages — loads theme CSS conditionally
│ │ # Brand text uses {{ COMPANY_NAME }} — no hardcoded strings
│ ├── [all other page templates] # All POST forms include {{ csrf_token() }} hidden field
│ └── errors/
│ ├── 403.html / 404.html / 500.html
│
└── tools/
├── migration_photo_verification_toggle.py # Adds photo_verification_enabled to qr_codes
├── migration_PM_permissions.py # One-time: user_project/location_permissions tables
├── migration_dynamic_qr_locations.py # One-time: qr_type column + qr_code_locations table
└── optimize_time_attendance_db.py # Standalone DB index optimization script
```
---
## 3. Database Tables
| Table | Model | Purpose |
|---|---|---|
| `users` | `User` | System user accounts with RBAC |
| `employee` | `Employee` | Employee master data (firstName, lastName, title, contractId) |
| `attendance_data` | `AttendanceData` | QR check-in records with GPS, photo verification |
| `time_attendance` | `TimeAttendance` | Imported time-clock records from Excel |
| `qr_codes` | `QRCode` | QR code definitions (standard and dynamic types) |
| `qr_code_styles` | `QRCodeStyle` | Reusable QR code visual styles |
| `qr_code_locations` | `QRCodeLocation` | Selectable locations for dynamic QR codes |
| `projects` | `Project` | Projects that group QR codes and employees |
| `user_project_permissions` | `UserProjectPermission` | Project-level access for Project Managers |
| `user_location_permissions` | `UserLocationPermission` | Location-level access for Project Managers |
| `log_events` | (raw SQL) | Application event log (created by AppLogger) |
### `qr_codes` — key columns
| Column | Type | Notes |
|---|---|---|
| `qr_type` | VARCHAR(20) | `'standard'` or `'dynamic'` |
| `photo_verification_enabled` | TINYINT(1) DEFAULT 1 | Per-QR photo verification toggle (added May 2026) |
| `active_status` | BOOLEAN | |
| `project_id` | FK → projects | |
### `attendance_data` — key columns
| Column | Type | Notes |
|---|---|---|
| `employee_id` | VARCHAR(50) | Always stored uppercased |
| `location_name` | VARCHAR(100) | Resolved location — never stores `'Dynamic'` |
| `location_accuracy` | FLOAT | Haversine distance in miles |
| `is_dynamic_qr` | BOOLEAN | True when checked in via dynamic QR |
| `verification_photo` | TEXT | Base64 encoded image |
| `verification_required` | BOOLEAN | |
| `verification_status` | VARCHAR(20) | `pending` / `approved` / `rejected` |
**Key relationships:**
- `Employee.contractId` → `Project.id`
- `QRCode.project_id` → `Project.id`
- `AttendanceData.qr_code_id` → `QRCode.id` (CASCADE DELETE)
- `TimeAttendance.project_id` → `Project.id`
---
## 4. User Roles & Access Control
```python
VALID_ROLES = ['admin', 'staff', 'payroll', 'project_manager', 'accounting']
STAFF_LEVEL_ROLES = ['staff', 'payroll', 'project_manager', 'accounting']
```
| Role | Key Access |
|---|---|
| `admin` | Full access; only role that sees System Logs, Users, all export tools |
| `staff` | Create/edit QR codes; view dashboard and reports; no delete, no admin |
| `payroll` | Same as staff + Time Attendance section |
| `accounting` | Same as payroll (identical menu items) |
| `project_manager` | Reports only; scoped to assigned projects and locations via permission tables |
**Auth decorators** (in `utils/helpers.py`):
- `@login_required` — redirects to `/login` if no session
- `@admin_required` — 403 if role is not `admin`
- `@staff_or_admin_required` — 403 if not admin or staff-level
**`/register` is restricted to `@admin_required`** — public self-registration is disabled.
---
## 5. Application Factory & Initialization Order
```
1. load_dotenv()
2. from extensions import db, init_logger
3. create_app():
a. app.config.from_object(get_config())
b. SECRET_KEY guard — sys.exit(1) if default value in non-debug mode
c. db.init_app(app)
d. set_db(db) → unpacks all model classes
e. init_logger(app, db) → binds logger_handler in extensions.py
f. register_blueprints() — in canonical order (see §6)
+ side-effect import of attendance_edit, verification, attendance_export
g. create_location_logging_routes() ← from location_logging.py
h. SecurityManager.init_app() ← wires CSRF before_request + rate limiting
i. inject_csrf_token() context processor
j. inject_company_name() context processor (COMPANY_NAME, THEME_NAME, CURRENT_YEAR)
k. inject_logging_status(), inject_turnstile() context processors
l. template_filters (strftime, days_since, time_ago)
m. before_request: adjust_session_lifetime + g.start_time + suspicious UA scan
n. after_request: slow-query detection + error response logging
o. Error handlers: 403, 404, 500
p. Startup init: create_tables() + update_existing_qr_codes() ← runs under Gunicorn too
4. if __name__ == '__main__':
a. lazy import PerformanceMonitor (dev-mode only)
b. initialize_performance_optimizations()
c. app.run()
```
**Critical notes:**
- `create_tables()` and `update_existing_qr_codes()` run inside `with app.app_context()` inside `create_app()` — they execute under gunicorn, not only under `__main__`.
- `PerformanceMonitor` is imported lazily inside `__main__` only — gunicorn workers never load it.
- `update_existing_qr_codes()` prefers `QR_BASE_URL` from `.env`. Falls back to `FLASK_HOST`/`FLASK_PORT`. Never uses `request.url_root`.
---
## 6. Blueprint Registration Order
```python
auth_bp, dashboard_bp, users_bp, admin_bp, projects_bp,
qr_codes_bp, attendance_bp, statistics_bp,
employees_bp, time_attendance_bp
```
**Attendance blueprint split:**
`attendance.py` defines `bp = Blueprint('attendance', __name__)` once.
`attendance_edit.py`, `verification.py`, and `attendance_export.py` each do:
```python
from routes.attendance import bp # shared blueprint — do not redefine
```
`app.py` registers only `attendance_bp` once. Sub-modules are loaded as side-effect imports:
```python
import routes.attendance_edit # noqa: F401
import routes.verification # noqa: F401
import routes.attendance_export # noqa: F401
```
`time_attendance_export.py` is **not a Blueprint**. Plain module providing export helper functions. Never register it separately.
---
## 7. All Routes
### auth (Blueprint: `auth`)
| URL | Endpoint | Notes |
|---|---|---|
| `/` | `auth.index` | |
| `/register` | `auth.register` | `@admin_required` — not public |
| `/login` | `auth.login` | Rate-limited via SecurityManager |
| `/logout` | `auth.logout` | |
| `/profile` | `auth.profile` | |
### dashboard (Blueprint: `dashboard`)
| URL | Endpoint |
|---|---|
| `/dashboard` | `dashboard.dashboard` |
| `/project/<id>/qr-codes` | `dashboard.project_qr_codes` |
| `/dashboard/search` | `dashboard.search_qr_codes` |
| `/api/dashboard/stats` | `dashboard.dashboard_stats_api` |
| `/api/dashboard/realtime` | `dashboard.dashboard_realtime_api` |
### users (Blueprint: `users`)
| URL | Endpoint |
|---|---|
| `/users` | `users.users` |
| `/users/create` | `users.create_user` |
| `/users/<id>/edit` | `users.edit_user` |
| `/users/<id>/delete` | `users.delete_user` |
| `/users/<id>/reactivate` | `users.reactivate_user` |
| `/users/<id>/promote` | `users.promote_user` |
| `/users/<id>/demote` | `users.demote_user` |
| `/users/<id>/toggle-status` | `users.toggle_user_status` |
| `/users/<id>/activate` | `users.activate_user` |
| `/users/<id>/deactivate` | `users.deactivate_user` |
| `/users/<id>/permanently-delete` | `users.permanently_delete_user` |
| `/api/users/stats` | `users.user_stats_api` |
| `/api/locations-by-projects` | `users.get_locations_by_projects` |
| `/api/roles/permissions` | `users.role_permissions_api` |
| `/api/geocode` | `users.geocode_address_api` |
| `/api/reverse-geocode` | `users.reverse_geocode_api` |
### admin (Blueprint: `admin`)
| URL | Endpoint |
|---|---|
| `/admin/logs` | `admin.admin_logs` |
| `/admin/health/google-maps` | `admin.google_maps_health` |
| `/api/logs/recent` | `admin.api_recent_logs` |
| `/api/logs/stats` | `admin.api_log_stats` |
| `/api/logs/cleanup` | `admin.api_cleanup_logs` |
| `/api/logs/clear` | `admin.api_clear_logs` |
| `/api/logs/clear-old` | `admin.api_clear_old_logs` |
| `/api/logs/export` | `admin.api_export_logs` |
### projects (Blueprint: `projects`)
| URL | Endpoint |
|---|---|
| `/projects` | `projects.projects` |
| `/projects/create` | `projects.create_project` |
| `/projects/<id>/edit` | `projects.edit_project` |
| `/projects/<id>/toggle` | `projects.toggle_project` |
| `/api/projects/active` | `projects.api_active_projects` |
### qr_codes (Blueprint: `qr_codes`)
| URL | Endpoint | Notes |
|---|---|---|
| `/qr-codes/create` | `qr_codes.create_qr_code` | |
| `/qr-codes/bulk-import` | `qr_codes.import_bulk_qr_codes` | |
| `/qr-codes/bulk-import/template` | `qr_codes.download_qr_import_template` | |
| `/qr-codes/<id>/edit` | `qr_codes.edit_qr_code` | |
| `/qr-codes/<id>/delete` | `qr_codes.delete_qr_code` | |
| `/qr-codes/<id>/toggle-status` | `qr_codes.toggle_qr_status` | |
| `/qr-codes/<id>/activate` | `qr_codes.activate_qr_code` | |
| `/qr-codes/<id>/deactivate` | `qr_codes.deactivate_qr_code` | |
| `/qr-codes/<id>/copy-url` | `qr_codes.copy_qr_url` | |
| `/qr-codes/<id>/open-link` | `qr_codes.open_qr_link` | |
| `/qr/<url>` | `qr_codes.qr_destination` | |
| `/qr/<url>/checkin` | `qr_codes.qr_checkin` | **CSRF-exempt** — public unauthenticated |
| `/qr/<url>/locations` | `qr_codes.qr_get_locations` | |
### attendance (Blueprint: `attendance` — split across 4 files)
| URL | Endpoint | File |
|---|---|---|
| `/attendance` | `attendance.attendance_report` | `attendance.py` |
| `/api/attendance/locations` | `attendance.attendance_locations_api` | `attendance.py` |
| `/api/attendance/stats` | `attendance.attendance_stats_api` | `attendance.py` |
| `/api/search_employees` | `attendance.search_employees_api` | `attendance.py` |
| `/api/get_project_locations` | `attendance.get_project_locations_api` | `attendance.py` |
| `/api/time-attendance/locations` | `attendance.time_attendance_locations_api` | `attendance.py` |
| `/attendance/<id>/edit` | `attendance.edit_attendance` | `attendance_edit.py` |
| `/attendance/<id>/delete` | `attendance.delete_attendance` | `attendance_edit.py` |
| `/attendance/add` | `attendance.add_manual_attendance` | `attendance_edit.py` |
| `/attendance/save_manual` | `attendance.save_manual_attendance` | `attendance_edit.py` |
| `/verification-review` | `attendance.verification_review` | `verification.py` |
| `/verification-review/<id>` | `attendance.verification_review_detail` | `verification.py` |
| `/verification-review/<id>/update` | `attendance.update_verification_status` | `verification.py` |
| `/api/attendance/<id>/verification-details` | `attendance.get_verification_details` | `verification.py` |
| `/export-configuration` | `attendance.export_configuration` | `attendance_export.py` |
| `/generate-excel-export` | `attendance.generate_excel_export` | `attendance_export.py` |
### statistics (Blueprint: `statistics`)
| URL | Endpoint |
|---|---|
| `/statistics` | `statistics.qr_statistics` |
| `/api/statistics/export` | `statistics.export_statistics` |
### employees (Blueprint: `employees`)
| URL | Endpoint |
|---|---|
| `/employees` | `employees.employees` |
| `/employees/create` | `employees.create_employee` |
| `/employees/<idx>/edit` | `employees.edit_employee` |
| `/employees/<idx>/delete` | `employees.delete_employee` |
| `/employees/<idx>` | `employees.employee_detail` |
| `/api/employees/search` | `employees.api_employees_search` |
### time_attendance (Blueprint: `time_attendance`)
| URL | Endpoint |
|---|---|
| `/time-attendance` | `time_attendance.time_attendance_dashboard` |
| `/time-attendance/import` | `time_attendance.import_time_attendance` |
| `/time-attendance/import/analyze-duplicates` | `time_attendance.analyze_import_duplicates` |
| `/time-attendance/import/analyze-invalid` | `time_attendance.analyze_import_invalid` |
| `/time-attendance/import/start` | `time_attendance.start_import_job` |
| `/time-attendance/import/stream/<job_id>` | `time_attendance.stream_import_progress` |
| `/time-attendance/import/cancel-pending` | `time_attendance.cancel_pending_import` |
| `/time-attendance/import/validate` | `time_attendance.validate_import_file` |
| `/time-attendance/import/batch/<batch_id>` | `time_attendance.view_import_batch` |
| `/time-attendance/import/batch/<batch_id>/delete` | `time_attendance.delete_import_batch` |
| `/time-attendance/import/download-template` | `time_attendance.download_import_template` |
| `/time-attendance/export` | `time_attendance.export_time_attendance` |
| `/time-attendance/export/excel` | `time_attendance.excel_export_time_attendance` |
| `/time-attendance/export-by-building` | `time_attendance.export_time_attendance_by_building` |
| `/time-attendance/records` | `time_attendance.time_attendance_records` |
| `/time-attendance/record/<id>` | `time_attendance.time_attendance_record_detail` |
| `/time-attendance/delete/<id>` | `time_attendance.delete_time_attendance_record` |
| `/api/time-attendance/employee/<id>` | `time_attendance.api_time_attendance_by_employee` |
| `/api/time-attendance/location/<n>` | `time_attendance.api_time_attendance_by_location` |
### Location logging (registered directly on `app`, not a Blueprint)
| URL | Purpose |
|---|---|
| `/api/log-location-action` | Android GPS debug logging |
| `/api/location-debug-info` | Return GPS debug info |
### Security API (registered by SecurityManager on `app`)
| URL | Purpose |
|---|---|
| `/api/security/status` | Admin-only security dashboard stats |
| `/api/security/clear-blocks` | Admin-only: clear rate-limit blocks |
---
## 8. Template System
### Two Base Templates
- **`base.html`** — unauthenticated pages (login, register, QR scan, errors). Body class: `login-layout`.
- **`base_authenticated.html`** — all protected pages. Body class: `has-sidebar`. Fixed collapsible sidebar, top header with user/role badge, flash message rendering.
### Template Block Names
| Block | Purpose |
|---|---|
| `{% block title %}` | Page `<title>` text only — **no CSS or JS here** |
| `{% block page_title %}` | Top header `<h1>` (authenticated only) |
| `{% block extra_head %}` | Page-specific CSS — inside `<head>` |
| `{% block content %}` | Main page body |
| `{% block extra_scripts %}` | Page-specific JS — before `</body>` |
**Rule:** CSS always goes in `extra_head`. JS always goes in `extra_scripts`. **Never inject either into `{% block title %}`.**
### Theme System
Both `base.html` and `base_authenticated.html` load the theme override after `style.css`:
```html
<!-- Theme Override (set THEME_NAME in .env to activate, e.g. THEME_NAME=gov) -->
{% if THEME_NAME %}
<link rel="stylesheet" href="{{ url_for('static', filename='css/theme-' + THEME_NAME + '.css') }}" />
{% endif %}
```
Loaded **before** `{% block extra_head %}` so page-specific CSS loads after and can override.
**`theme-gov.css` overrides:** `--primary-color: #16a34a` / `--primary-hover: #15803d` (institutional green). No CSS text hacks — brand names handled via `{{ COMPANY_NAME }}`.
**Adding a third company theme:** Create `static/css/theme-{name}.css` and set `THEME_NAME={name}` in `.env`. No code changes required.
### Global Context Variables
| Variable | Source |
|---|---|
| `COMPANY_NAME` | `.env` (`COMPANY_NAME`) |
| `THEME_NAME` | `.env` (`THEME_NAME`) — empty string when not set |
| `CURRENT_YEAR` | `datetime.now().year` |
| `csrf_token` | `generate_csrf_token()` from `advanced_security_middleware` |
| `is_admin` | `bool` from session role |
| `turnstile_enabled`, `turnstile_site_key` | Turnstile config |
### CSRF in Templates
Every POST form:
```html
<input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
```
Every AJAX POST:
```js
'X-CSRF-Token': (window.qrConfig && window.qrConfig.csrfToken) || ''
```
Exception: `/qr/<url>/checkin` is CSRF-exempt (public, unauthenticated).
---
## 9. Design System
### CSS Custom Properties (defined in `style.css`)
```css
--primary-color: #2563eb /* Blue — LT default */
--primary-hover: #1d4ed8
--secondary-color: #64748b
--success-color: #10b981
--warning-color: #f59e0b
--danger-color: #ef4444
--info-color: #0891b2
--gray-50: #f8fafc /* Page background */
--gray-700: #334155 /* Body text */
--gray-900: #0f172a /* Headings */
```
**Light mode only** — `color-scheme: light only !important` enforced globally.
### Layout
- Sidebar: `280px` expanded, `64px` collapsed. Gradient `#2563eb → #1d4ed8` (LT). GOV theme: `#16a34a → #15803d`.
- Header height: `64px`.
- Z-index: dropdown 1000, modal 1050, sidebar 1100, overlay 1200.
### Excel Export Styling
- **Header rows:** white bold text on solid black fill (`000000`)
- **Miss-punch / amber:** `FFC000` fill
- **Font:** Aptos Narrow 11pt everywhere; 14pt report title, 12pt section summary
---
## 10. Security Architecture
### CSRF Protection
- `SecurityManager` from `advanced_security_middleware.py` wired via `init_app()` in `create_app()`
- `before_request` validates `csrf_token` form field or `X-CSRF-Token` header on every `POST/PUT/PATCH/DELETE`
- Token stored in `session['csrf_token']`; compared with `hmac.compare_digest()`
- **CSRF-exempt:** `auth.login`, `auth.register`, `qr_codes.qr_checkin`, `static`
### Rate Limiting
- 5 failed login attempts within 15 minutes → IP blocked 15 minutes (in-memory per worker)
- `SecurityManager.create_secure_session()` called on login success (clears failed-attempt counter)
### Session Security
- `session.clear()` before setting new keys on login (prevents session fixation)
- `before_request` hook `adjust_session_lifetime()`: `remember_me` → 30 days, default → 10 hours
- `SESSION_COOKIE_SECURE=True` requires HTTPS — HTTP-only deployments must set `false` or login loops
### validate_session_security() — DO NOT USE in before_request
In-memory dict per worker — breaks under multi-worker gunicorn (login worker A, next request hits worker B with empty dict → 401). Intentionally not called. Flask signed cookie + CSRF handles integrity.
### SQL Injection Prevention
All dynamic SQL uses SQLAlchemy parameterized queries:
```python
conditions, params = [], {}
conditions.append("ad.check_in_date >= :date_from")
params["date_from"] = date_from
db.session.execute(text("... WHERE 1=1 " + filter_clause), params)
```
---
## 11. QR Code System
### QR Types
- **`standard`** (default) — fixed single location
- **`dynamic`** — employee selects location at scan time from list auto-generated from all active standard QR codes. No manual management UI — always queried live via `SELECT DISTINCT location, location_address FROM qr_codes WHERE qr_type='standard'`.
### Photo Verification — Two-Layer Toggle
Both must be `True` for photo verification to trigger:
1. **Global:** `PHOTO_VERIFICATION_ENABLED` in `.env`
2. **Per-QR:** `qr_codes.photo_verification_enabled` (TINYINT(1) DEFAULT 1)
```python
qr_photo_verification = getattr(qr_code, 'photo_verification_enabled', True)
if qr_photo_verification and current_app.config.get('PHOTO_VERIFICATION_ENABLED', True) \
and location_accuracy > threshold:
# require photo
```
Toggle UI in `create_qr_code.html` and `edit_qr_code.html` — uses `addEventListener('change', ...)` in `extra_scripts` block.
### Check-In Flow
1. Employee scans QR → `qr_destination.html`
2. Enters ID; GPS captured by browser
3. 30-min interval guard (configurable via `TIME_INTERVAL`)
4. Dynamic QR: server rejects if `selected_location_name` is empty; `location_name` in record is always the resolved name, never `'Dynamic'`
5. Haversine distance calculated; photo required if beyond threshold (and both toggles enabled)
6. Server-side photo size check: rejects > `VERIFICATION_PHOTO_MAX_SIZE` with HTTP 413
7. Record saved to `attendance_data`
### Check-In Page Features
- Bilingual: English/Vietnamese toggle (`qr_destination.js`)
- Staff ID persistence: `localStorage` remembers last employee ID
- Android GPS: special handler (`android_location_handler.js`)
- QR URLs must be full `https://` absolute URLs — relative URLs parsed as search queries by phones
- Base URL constructed from `QR_BASE_URL` env var — never from `request.url_root`
---
## 12. Time Attendance Import Pipeline
### Excel File Requirements
- **Required columns:** `ID`, `Date`, `Time`, `Location Name`, `Action Description`
- **Optional columns:** `Name`, `Platform`, `Event Description`, `Recorded Address`, `Distance`
### Import Flow
1. Upload → `validate_import_file`
2. Duplicate analysis → `analyze_import_duplicates` → user reviews
3. Invalid record analysis → `analyze_import_invalid`
4. Start job → `start_import_job` (SSE streaming progress via `stream_import_progress`)
5. Result in `time_attendance_import_result.html`
### Special Handling
- `Recorded Address`: read via openpyxl directly (not pandas) to preserve HYPERLINK formulas
- Duplicate detection: hash of `employee_id + date + time + action_description`
- Import tracked by `import_batch_id` (UUID)
- `self.db` not `db` — in `TimeAttendanceImportService`, always access SQLAlchemy via `self.db`
- Validation `except` blocks must not silently swallow exceptions (fail-open prevention)
---
## 13. Time Attendance Excel Export
### Calculator — CRITICAL
**Always use `WorkingHoursCalculator`** from `working_hours_calculator.py`.
**Never switch to `SingleCheckInCalculator`** — legacy, causes silent calculation errors.
### Rounding Rules
- **Daily / Weekly / Grand Total + SP/PW/PT summary columns:** `_qtr()` — quarter-hour rounding
- **Individual Hours/Building entry cells:** raw `round(..., 2)` — no quarter rounding
- `_qtr()` pipeline: decimal hours → minutes → `round_time_to_quarter_hour()` → `convert_minutes_to_base100()` → `round_base100_hours()`
### Work Type Codes
- **SP** = Special Project, **PW** = Periodic Work, **PT** = Project Team (Part-Time)
- Parsed from `employee_id` via `parse_employee_id_for_work_type()`
### Overnight Shift Handling
**Rule 1 — Sort key:** early-morning OUTs (`hour <= 3`) use `_overnight_aware_sort_key()` which adds 86400 seconds — pushes them past midnight so they sort after same-day evening INs.
**Rule 2 — Detection threshold:** `hour >= 12` (noon). Any IN at or after 12:00 PM is an overnight IN candidate if no matching OUT exists same day AND early-morning OUT (`hour <= 3`) exists next calendar day.
**Orphan guard:** early-morning OUTs with `check_in_date <= current_day` are orphans — skip. Only OUTs with `check_in_date > current_day` (moved by overnight detection) pair with evening INs.
### Cross-Type SP Pairs
`is_cross_type = True` when SP IN pairs with non-SP OUT (or vice versa). Accumulates separately in `cross_type_sp/pw/pt_hours` to prevent double-counting in summary rows.
### Summary Rows (SP / PW / PT + Regular)
```
col8='SP' col9=_qtr(sp_hours) ← only if sp_hours > 0
col8='PW' col9=_qtr(pw_hours) ← only if pw_hours > 0
col8='PT' col9=_qtr(pt_hours) ← only if pt_hours > 0
col8='Regular' col9=_qtr(regular_only_hours) ← always when any special type exists
col7='GRAND TOTAL:' col9=_qtr(grand_regular) col10=_qtr(grand_ot)
```
### Export Date Range
`_resolve_date_range()` enforces a **14-day cap by default**. Both export functions accept `unlimited=False`; pass `unlimited=True` to bypass.
**UI:** "Unlimited" checkbox in `time_attendance_records.html` before export buttons.
### Employee Name Format
`"Lastname, Firstname"` — `f"{emp.lastName}, {emp.firstName}"`
### Export Date Iteration
Cap `sorted_dates` to `<= end_date` — prevents overnight buffer day from rendering as a display row.
### Cross-Building Pairing
Same-day IN at Building A + OUT at Building B → pair and label `"IN: Building A → OUT: Building B"`. Suppress spurious Missed Punch rows.
---
## 14. Logging System (`logger_handler.py`)
Single `AppLogger` instance in `extensions.py`. Import everywhere as:
```python
from extensions import logger_handler
logger_handler.logger.info("...")
logger_handler.logger.error("...", exc_info=True) # always pass exc_info=True in except blocks
```
### Log Destinations
- `logs/application.log` — rotating 10MB/5 backups
- `logs/errors.log` — rotating 5MB/10 backups
- `logs/security.log` — rotating 2MB/20 backups
- `log_events` DB table — admin dashboard at `/admin/logs`
### Log Every Action
All create, edit, and delete operations must include a log entry:
```python
logger_handler.logger.info(f"User {session['username']} created employee {new_employee.id}")
```
### traceback Convention
- Use `exc_info=True` on `logger.error()` — never `import traceback` inline
- `traceback.format_exc()` only acceptable when passing `stack_trace=` to `log_flask_error()`
---
## 15. Configuration (`config.py` + `.env`)
All env-var reads centralized in `config.py`. Blueprints use `current_app.config['KEY']`.
### Key `.env` Variables
```
DATABASE_URL # mysql+pymysql://user:pass@host/db — special chars in password OK
SECRET_KEY # MUST differ from default
COMPANY_NAME / CONTRACT_NAME
FLASK_HOST / FLASK_PORT / FLASK_ENV / DEBUG
SESSION_COOKIE_SECURE # MUST be 'false' for HTTP-only deployments
SESSION_COOKIE_HTTPONLY / SESSION_COOKIE_SAMESITE
TIME_INTERVAL # Check-in cooldown in minutes (default 30)
GOOGLE_MAPS_API_KEY # Optional; falls back to Haversine-only
TURNSTILE_ENABLED / TURNSTILE_SITE_KEY / TURNSTILE_SECRET_KEY
ENABLE_PHOTO_VERIFICATION # Global toggle (default 'true')
PHOTO_VERIFICATION_DISTANCE_THRESHOLD # Miles (default 0.3)
VERIFICATION_PHOTO_MAX_SIZE # Bytes (default 5MB)
UPLOAD_FOLDER # Temp path (default /tmp)
DEFAULT_ADMIN_PASSWORD # CHANGE IN PRODUCTION
QR_BASE_URL # Public-facing domain for QR links — required behind reverse proxy
THEME_NAME # Activates static/css/theme-{THEME_NAME}.css
SYNC_INTERVAL_MINUTES
SQLALCHEMY_ENGINE_OPTIONS_POOL_SIZE / POOL_TIMEOUT / POOL_RECYCLE / MAX_OVERFLOW
```
---
## 16. Distance Calculation
Uses **Haversine formula** only (straight-line geodesic distance). Google Maps Distance Matrix API removed.
`utils/geocoding.py` initializes `gmaps_client` for geocoding (address → coordinates) if `GOOGLE_MAPS_API_KEY` is set.
### Google Maps Guard — ALWAYS USE
```python
from utils.geocoding import is_gmaps_available, gmaps_client
if is_gmaps_available():
result = gmaps_client.geocode(address)
else:
# fall back to OpenStreetMap / Haversine
```
Never call `gmaps_client.method()` without a `None` guard.
### Address Normalization
Multi-strategy matching: core street extraction → component matching → fuzzy fallback. Handles geocoding drift for near-identical addresses.
---
## 17. Migration Scripts
All in `tools/`. Always use **`pymysql` directly** — never import the Flask app or SQLAlchemy ORM.
ORM loads models at import time; if target column doesn't exist yet, it crashes on startup.
**Pattern:**
```python
from dotenv import load_dotenv
load_dotenv()
import pymysql, re
def parse_db_url(url):
"""Use regex — urlparse breaks on special chars (@, :) in passwords."""
url = re.sub(r'^mysql\+pymysql://', '', url)
m = re.match(
r'^(?P<user>[^:]+):(?P<password>.+)@(?P<host>[^@:/]+)(?::(?P<port>\d+))?/(?P<db>[^?]+)',
url
)
return {'host': m.group('host'), 'port': int(m.group('port') or 3306),
'user': m.group('user'), 'password': m.group('password'), 'database': m.group('db')}
def run():
conn = pymysql.connect(**parse_db_url(os.environ['DATABASE_URL']), charset='utf8mb4', autocommit=False)
with conn.cursor() as cur:
cur.execute("SELECT COUNT(*) FROM information_schema.COLUMNS WHERE ...")
if cur.fetchone()[0] > 0:
print("[SKIP] already exists"); return
cur.execute("ALTER TABLE `table` ADD COLUMN `col` TINYINT(1) NOT NULL DEFAULT 1")
conn.commit()
```
### Completed Migrations
| Script | What it adds |
|---|---|
| `migration_photo_verification_toggle.py` | `qr_codes.photo_verification_enabled` TINYINT(1) DEFAULT 1 |
| `migration_dynamic_qr_locations.py` | `qr_type` column + `qr_code_locations` table |
| `migration_PM_permissions.py` | `user_project_permissions` + `user_location_permissions` tables |
---
## 18. Coding Conventions & Developer Preferences
### Change Philosophy
- **Minimal, additive changes only** — preserve all route names, function names, endpoint names, variable names, URL patterns
- New functionality added alongside existing code, not replacing it
- Never rename routes, functions, endpoints, or variables
- **Every code fix applies to both LT and GOV instances**
### File Delivery
- **≤ 4 files changed** → present each file individually
- **≥ 5 files changed** → deliver as a single zip archive
### File Editing Approach
- Pull latest code at start of each session
- CRLF normalization first if needed: `content.replace('\r\n', '\n')`
- Use surgical `str_replace` edits — never rewrite large blocks wholesale
- AST parse check after every Python edit: `python3 -c "import ast; ast.parse(open(f).read())"`
- Simulate logic before and after any calculator or pairing logic changes
### SQLAlchemy Patterns
```python
# Correct (SQLAlchemy 2.0):
record = db.session.get(Model, record_id)
if record is None:
abort(404)
# Deprecated — do not use:
record = Model.query.get_or_404(record_id)
record = Model.query.get(record_id)
# Raw SQL requires text():
db.session.execute(text("SELECT ..."), params)
# Never pass raw strings to conn.execute() — ObjectNotExecutableError in SQLAlchemy 2.0
```
### JavaScript Patterns
- Use `createElement` + `addEventListener` — never inline `onchange`, `onclick`, etc.
- Wrap in `DOMContentLoaded`
- CSS → `{% block extra_head %}`, JS → `{% block extra_scripts %}`, **never into `{% block title %}`**
- All AJAX POST: `'X-CSRF-Token': (window.qrConfig && window.qrConfig.csrfToken) || ''`
- No `localStorage` / `sessionStorage` in artifacts
### Error Handling Pattern
Every route `except` block must:
1. `db.session.rollback()`
2. `logger_handler.logger.error(f"...: {e}", exc_info=True)`
3. `flash(...)` a user-facing message
4. Return redirect or error response
No bare `except:` — always `except Exception as e:`.
### Employee Autocomplete (attendance and time-attendance)
- Visible text input + hidden `employee_id` field synced on numeric input
- Fetches `/api/search_employees` (includes unregistered IDs from `attendance_data`, not only Employee table)
- CSS inlined in `extra_head`; JS uses `createElement` + `addEventListener`
### Timestamp Convention
Use `datetime.now()` (local time) throughout — **not** `datetime.utcnow()`.
### Context Safety
- Use `has_request_context()` (not `if not request:`) to check Flask request context
- Capture `current_app._get_current_object()` in route body, not inside lazy generators
---
## 19. Attendance Report
Query fetches **1,001 rows**, trims to 1,000 if extra row present, sets `records_truncated = True`.
Template displays yellow banner when truncated:
```
Showing the most recent 1,000 records. Narrow the date range or apply additional filters.
```
---
## 20. Known Bugs Fixed — Do Not Reintroduce
### Set 1 — Critical
| File | Fix |
|---|---|
| `routes/time_attendance_export.py` | Removed stray second docstring |
| `app.py` | Error handlers wired to `render_template('errors/*.html')`; 403 added |
| `templates/errors/*.html` | `url_for('dashboard')` → `url_for('dashboard.dashboard')` |
| `routes/payroll.py` | Removed duplicate `get_employee_name`, `get_qr_code_checkin_count`, `@bp.context_processor` |
| `models/attendance.py` | `check_in_time` default: `datetime.now().time` → `lambda: datetime.now().time()` |
| `models/user.py` | Added `@staticmethod` to `has_export_permissions()` |
### Set 2 — Moderate / Minor
| File | Fix |
|---|---|
| `app.py` | Merged duplicate before/after request hooks |
| `app.py` | `update_existing_qr_codes()` uses `QR_BASE_URL` first, not `request.url_root` |
| `config.py` | `SQLALCHEMY_ENGINE_OPTIONS` pool wiring |
| `utils/geocoding.py` | `is_gmaps_available()` helper |
| `working_hours_calculator.py` | CRLF → LF |
### Set 3 — Structural
| File | Fix |
|---|---|
| `routes/time_attendance_export.py` | Removed 14 unused imports |
| `routes/dashboard.py`, `statistics.py`, `payroll.py` | `db.session.rollback()` in all except blocks |
| `models/employee.py` | `cls.id.like()` on BigInteger → `cast(cls.id, String).like()` |
### Set 4 — Export Pipeline
| File | Fix |
|---|---|
| `routes/time_attendance_export.py` | Overnight IN threshold: `>= 19` → `>= 12` |
| `routes/time_attendance_export.py` | SP/PW/PT summary rows + `Regular` row |
| `routes/time_attendance_export.py` | `unlimited=False` parameter throughout |
| `templates/time_attendance_records.html` | "Unlimited" checkbox |
### Set 5 — GOV Deployment
| File | Fix |
|---|---|
| `routes/qr_codes.py` | `_get_qr_base_url()` helper; replaced all `request.url_root` |
| `config.py` | `QR_BASE_URL` env var |
| nginx (GOV) | Removed duplicate `proxy_set_header Host` (doubled hostname in QR links) |
### Set 6 — Theme System
| File | Fix |
|---|---|
| `static/css/theme-gov.css` | New file — GOV brand green overrides |
| `app.py` | `inject_company_name()` returns `THEME_NAME` and `CURRENT_YEAR` |
| `templates/base.html` + `base_authenticated.html` | Conditional theme CSS; `{{ COMPANY_NAME }}` for brand text; `{{ CURRENT_YEAR }}` in footer |
### Set 7 — Security Hardening
| File | Fix |
|---|---|
| `routes/statistics.py`, `routes/payroll.py` | SQL injection: f-string filters → parameterized queries |
| `routes/auth.py` | Open redirect: `_is_safe_url()` on `next` param |
| `app.py` | `SECRET_KEY` default guard: `sys.exit(1)` |
| `routes/auth.py` | Session fixation: `session.clear()` before new keys |
| `routes/auth.py` | `@login_required` + `@admin_required` on `/register` |
| `app.py` | CSRF `SecurityManager` wired; `before_request` validator |
| All POST forms (22 templates) | `{{ csrf_token() }}` hidden field |
| All AJAX POST calls | `X-CSRF-Token` header |
| `advanced_security_middleware.py` | `request.json` guarded with content-type check (fixes 415) |
| `advanced_security_middleware.py` | Stable key from `SECRET_KEY` via SHA-256 (fixes per-worker warning) |
| `advanced_security_middleware.py` | `validate_session_security()` removed from `security_check()` (fixes 401 under multi-worker) |
| All route files | Bare `except:` → `except Exception` (11 locations) |
| `routes/qr_codes.py` | Server-side photo size enforcement; HTTP 413 on oversized payload |
### Set 8 — Code Quality
| File | Fix |
|---|---|
| 7 route files | `Model.query.get_or_404()` → `db.session.get()` + `abort(404)` (21 call sites) |
| `config.py` | `PERMANENT_SESSION_LIFETIME` → `timedelta(hours=10)` |
| `routes/attendance.py` | Split into 4 files sharing one blueprint |
| `requirements.txt` | `mysql-connector-python` removed |
| `routes/attendance.py` | LIMIT 1001 + `records_truncated` flag + yellow banner |
### Set 9 — Template Sync (May 2026)
| File | Fix |
|---|---|
| `templates/base.html` (GOV) | Footer hardcoded `2025 QR Code Management System` → `{{ CURRENT_YEAR }} {{ COMPANY_NAME }}` |
| `templates/base.html` (GOV) | Theme CSS block moved to after `style.css`, before Font Awesome (matches LT order) |
| `templates/base_authenticated.html` (GOV) | Indentation sync with LT |
| `templates/projects.html` (GOV) | Confirm dialog on project toggle + loading state on Edit buttons (sync with LT) |
### Set 10 — Per-QR Photo Verification Toggle (May 2026)
| File | Change |
|---|---|
| `models/qrcode.py` | Added `photo_verification_enabled` TINYINT(1) DEFAULT 1 |
| `routes/qr_codes.py` | Create: reads toggle from form, passes to constructor + logs |
| `routes/qr_codes.py` | Edit: reads toggle from form, assigns to record + logs |
| `routes/qr_codes.py` | Checkin: checks per-QR AND global flag (both must be True) |
| `templates/create_qr_code.html` | Photo Verification toggle section added |
| `templates/edit_qr_code.html` | Photo Verification toggle section added (CSS in `extra_head`, JS in `extra_scripts` with `addEventListener`) |
| `tools/migration_photo_verification_toggle.py` | pymysql migration — adds column safely |
---
## 21. Infrastructure & Deployment
- **Deploy user:** `qrcode` on both servers
- **Restart:** `sudo supervisorctl restart qrcode`
- **Template-only changes:** no restart needed
- **Python/model changes:** always run migration first, then deploy files, then restart
- **Gunicorn:** gevent workers — avoid threading-unsafe patterns
### Deploy Order for DB Column Additions
1. Run `python3 tools/migration_<name>.py` (pymysql, safe to re-run)
2. Deploy updated Python files
3. Restart service
---
## 22. Session Changelog
### Sessions 1–8 — Foundational Work
See §20 Sets 1–8 for detailed bug fix history. Covers: initial critical fixes, security hardening (CSRF, SQL injection, session fixation, rate limiting, open redirect), export pipeline enhancements, GOV deployment, theme system, code quality sprint.
### Session (May 2026) — Template Sync + Photo Verification Toggle
- Diffed LT and GOV codebases — confirmed only 3 template files differed (whitespace + minor logic)
- Synced `base.html`, `base_authenticated.html`, `projects.html` — GOV brought to parity with LT
- Added per-QR `photo_verification_enabled` toggle (DB column + model + routes + UI + migration)
- Fixed migration script: replaced `urlparse` with regex parser (handles special chars in DB passwords), replaced SQLAlchemy `conn.execute(str)` with `pymysql` direct (avoids ORM loading model before column exists)
- Fixed `edit_qr_code.html` toggle bug: JS was injected into `{% block title %}` (corrupted by earlier injection) — moved CSS to `extra_head`, JS to `extra_scripts` using `addEventListener`, removed inline `onchange` attribute