Sep 15 - Update the attendance records table to update every 20 min

Claude.md

@@ -342,6 +342,7 @@ import routes.attendance_export  # noqa: F401
 | `/api/attendance/locations` | `attendance.attendance_locations_api` | `attendance.py` |
 | `/api/attendance/stats` | `attendance.attendance_stats_api` | `attendance.py` |
 | `/api/search_employees` | `attendance.search_employees_api` | `attendance.py` |
+| `/api/attendance/live-updates` | `attendance.attendance_live_updates_api` | `attendance.py` — live table polling, see §19 |
 | `/api/get_project_locations` | `attendance.get_project_locations_api` | `attendance.py` |
 | `/api/time-attendance/locations` | `attendance.time_attendance_locations_api` | `attendance.py` |
 | `/attendance/<id>/edit` | `attendance.edit_attendance` | `attendance_edit.py` |
@@ -1034,6 +1035,37 @@ Semantics (identical on both sides, mirroring `_work_type_codes_for()`):
 **Do not reintroduce** `record.employeeId.toLowerCase() === id` in `applyFilters()` — that
 exact-match test is what dropped every SP/PW/PT row the query had already returned.
 
+### Live Updates (Sept 15, 2026)
+
+The records table picks up new check-ins without a reload. It uses **polling, not SSE/WebSockets**:
+a held-open stream per tab would pin a gevent worker connection and still need a DB poll behind
+it (the workers share no pub/sub).
+
+- **Endpoint:** `GET /api/attendance/live-updates?since_id=&date_from=&date_to=&location=&employee=&project=`
+  (`attendance_live_updates_api`, `@login_required`), JSON with `no-store` headers.
+- **Cheap path:** `SELECT COALESCE(MAX(id), 0) FROM attendance_data` (primary key). Not newer than
+  `since_id` → empty answer, no other query. Otherwise ONE query over `ad.id > since_id AND
+  ad.id <= latest` plus the page's filters, `ORDER BY ad.id LIMIT 201` (200 per batch; `has_more`
+  → the page fetches the next batch after 1 s). `verification_photo` (base64) is never selected; the
+  `location_accuracy` information_schema check is cached per process.
+- **Same filters as the page:** `_attendance_filter_conditions()` builds the WHERE clause for both
+  the report route and the endpoint, Project Manager scope included. **Never duplicate that logic.**
+- **Cursor:** the route reads `MAX(id)` **before** the report query and renders it as
+  `data-live-since-id` on `#attendanceReportContainer`. No attribute (PM without access, lookup
+  failed) → live updates stay off.
+- **Payload shape:** `_live_record_payload()` returns exactly what `loadTableData()` reads from the
+  server-rendered `<tbody>` (text, truncation, accuracy parsing, verification badge).
+  **If the row markup in `attendance_report.html` changes, update `_live_record_payload()` too.**
+- **Client** (`attendance_report.js`, LIVE UPDATES section): polls every 20 s, never overlapping;
+  paused while `document.hidden`, one immediate check when the tab is shown; exponential backoff
+  40 s → 5 min after errors; stops for good on redirect / 401 / 403 (session ended). New rows are
+  prepended newest-first, de-duplicated by id, highlighted for 8 s, and `refreshTableKeepingView()`
+  keeps the user's search, sort and page. A page showing the empty state reloads once when a
+  matching record arrives. A green **Live** pill in the table header shows the state.
+- **Not live:** edits, deletions and the summary statistics — they refresh on reload.
+- `createTableRow()` HTML-escapes every value (`escapeHtml`; `escapeJsString` inside `onclick`;
+  `truncateText` runs before escaping) — device and address text come from the public check-in page.
+
 ---
 
 ## 20. Known Bugs Fixed — Do Not Reintroduce
@@ -1205,6 +1237,15 @@ exact-match test is what dropped every SP/PW/PT row the query had already return
 | — | Verified by loading the real hook (before/after) into a Flask app with a controlled clock: Remember Me now survives 11 h, 25 days of daily use and 29 idle days, and expires after 31 idle days; non-Remember-Me still ends at 10 h |
 | — | No forced re-login on deploy: valid sessions keep working; non-Remember-Me sessions from before the deploy (no `login_epoch`) get their 10 hours counted from their first request after it |
 
+### Set 20 — Attendance Report Live Updates (Sept 15, 2026)
+| File | Change |
+|---|---|
+| `routes/attendance.py` | `_attendance_filter_conditions()` extracted from `attendance_report()`; `live_latest_id` cursor; `GET /api/attendance/live-updates` + `_live_record_payload()` (see §19) |
+| `templates/attendance_report.html` | `data-live-since-id` on the container; Live status pill + new-row highlight CSS |
+| `static/js/attendance_report.js` | LIVE UPDATES module; `filterRecords()` split out of `applyFilters()`, `sortFilteredData()` split out of `sortTable()` |
+| `static/js/attendance_report.js` | **Security:** `createTableRow()` put raw device / address / name text into `innerHTML`. Device comes from the check-in User-Agent, so a crafted UA could inject markup into the report — every value is now escaped |
+| — | Verified: filter helper produces identical SQL + params to the previous inline block for 180 input combinations; `_live_record_payload()` matches what `loadTableData()` reads from the Jinja-rendered `<tbody>` for 20 row variants; the real `attendance_report.js` driven with a fake DOM/timers/fetch passes 26 checks (insert, de-dup, page + sort kept, hidden-tab pause, no overlap, backoff + cap, stop on logout, escaping). Not yet exercised against a live MySQL server or a real browser |
+
 ---
 
 ## 21. Infrastructure & Deployment