Sep 15 - Update the attendance records table to update every 20 min

This commit is contained in:
2026-09-15 12:32:35 -04:00
parent 4786f34f4d
commit e5410d5141
4 changed files with 742 additions and 131 deletions
+41
View File
@@ -342,6 +342,7 @@ import routes.attendance_export # noqa: F401
| `/api/attendance/locations` | `attendance.attendance_locations_api` | `attendance.py` |
| `/api/attendance/stats` | `attendance.attendance_stats_api` | `attendance.py` |
| `/api/search_employees` | `attendance.search_employees_api` | `attendance.py` |
| `/api/attendance/live-updates` | `attendance.attendance_live_updates_api` | `attendance.py` — live table polling, see §19 |
| `/api/get_project_locations` | `attendance.get_project_locations_api` | `attendance.py` |
| `/api/time-attendance/locations` | `attendance.time_attendance_locations_api` | `attendance.py` |
| `/attendance/<id>/edit` | `attendance.edit_attendance` | `attendance_edit.py` |
@@ -1034,6 +1035,37 @@ Semantics (identical on both sides, mirroring `_work_type_codes_for()`):
**Do not reintroduce** `record.employeeId.toLowerCase() === id` in `applyFilters()` — that
exact-match test is what dropped every SP/PW/PT row the query had already returned.
### Live Updates (Sept 15, 2026)
The records table picks up new check-ins without a reload. It uses **polling, not SSE/WebSockets**:
a held-open stream per tab would pin a gevent worker connection and still need a DB poll behind
it (the workers share no pub/sub).
- **Endpoint:** `GET /api/attendance/live-updates?since_id=&date_from=&date_to=&location=&employee=&project=`
(`attendance_live_updates_api`, `@login_required`), JSON with `no-store` headers.
- **Cheap path:** `SELECT COALESCE(MAX(id), 0) FROM attendance_data` (primary key). Not newer than
`since_id` → empty answer, no other query. Otherwise ONE query over `ad.id > since_id AND
ad.id <= latest` plus the page's filters, `ORDER BY ad.id LIMIT 201` (200 per batch; `has_more`
→ the page fetches the next batch after 1 s). `verification_photo` (base64) is never selected; the
`location_accuracy` information_schema check is cached per process.
- **Same filters as the page:** `_attendance_filter_conditions()` builds the WHERE clause for both
the report route and the endpoint, Project Manager scope included. **Never duplicate that logic.**
- **Cursor:** the route reads `MAX(id)` **before** the report query and renders it as
`data-live-since-id` on `#attendanceReportContainer`. No attribute (PM without access, lookup
failed) → live updates stay off.
- **Payload shape:** `_live_record_payload()` returns exactly what `loadTableData()` reads from the
server-rendered `<tbody>` (text, truncation, accuracy parsing, verification badge).
**If the row markup in `attendance_report.html` changes, update `_live_record_payload()` too.**
- **Client** (`attendance_report.js`, LIVE UPDATES section): polls every 20 s, never overlapping;
paused while `document.hidden`, one immediate check when the tab is shown; exponential backoff
40 s → 5 min after errors; stops for good on redirect / 401 / 403 (session ended). New rows are
prepended newest-first, de-duplicated by id, highlighted for 8 s, and `refreshTableKeepingView()`
keeps the user's search, sort and page. A page showing the empty state reloads once when a
matching record arrives. A green **Live** pill in the table header shows the state.
- **Not live:** edits, deletions and the summary statistics — they refresh on reload.
- `createTableRow()` HTML-escapes every value (`escapeHtml`; `escapeJsString` inside `onclick`;
`truncateText` runs before escaping) — device and address text come from the public check-in page.
---
## 20. Known Bugs Fixed — Do Not Reintroduce
@@ -1205,6 +1237,15 @@ exact-match test is what dropped every SP/PW/PT row the query had already return
| — | Verified by loading the real hook (before/after) into a Flask app with a controlled clock: Remember Me now survives 11 h, 25 days of daily use and 29 idle days, and expires after 31 idle days; non-Remember-Me still ends at 10 h |
| — | No forced re-login on deploy: valid sessions keep working; non-Remember-Me sessions from before the deploy (no `login_epoch`) get their 10 hours counted from their first request after it |
### Set 20 — Attendance Report Live Updates (Sept 15, 2026)
| File | Change |
|---|---|
| `routes/attendance.py` | `_attendance_filter_conditions()` extracted from `attendance_report()`; `live_latest_id` cursor; `GET /api/attendance/live-updates` + `_live_record_payload()` (see §19) |
| `templates/attendance_report.html` | `data-live-since-id` on the container; Live status pill + new-row highlight CSS |
| `static/js/attendance_report.js` | LIVE UPDATES module; `filterRecords()` split out of `applyFilters()`, `sortFilteredData()` split out of `sortTable()` |
| `static/js/attendance_report.js` | **Security:** `createTableRow()` put raw device / address / name text into `innerHTML`. Device comes from the check-in User-Agent, so a crafted UA could inject markup into the report — every value is now escaped |
| — | Verified: filter helper produces identical SQL + params to the previous inline block for 180 input combinations; `_live_record_payload()` matches what `loadTableData()` reads from the Jinja-rendered `<tbody>` for 20 row variants; the real `attendance_report.js` driven with a fake DOM/timers/fetch passes 26 checks (insert, de-dup, page + sort kept, hidden-tab pause, no overlap, backoff + cap, stop on logout, escaping). Not yet exercised against a live MySQL server or a real browser |
---
## 21. Infrastructure & Deployment
+323 -80
View File
@@ -45,6 +45,99 @@ bp = Blueprint('attendance', __name__)
UPPER_EMPLOYEE_ID_SQL = "UPPER(ad.employee_id)"
def _attendance_filter_conditions(user_role, allowed_project_ids, allowed_location_names,
date_from, date_to, location_filter, employee_ids, project_filter):
"""
Build the WHERE conditions + bound parameters for attendance_data queries.
Shared by the Attendance Report page and its live-update endpoint so both
always apply the same Project Manager scope and the same user filters.
Every value is a bound parameter. Returns (filter_conditions, query_params).
"""
filter_conditions = []
query_params = {}
# ============================================================
# APPLY PROJECT MANAGER FILTERS TO SQL QUERY
# ============================================================
if user_role == 'project_manager':
# Filter by allowed projects
if allowed_project_ids:
project_placeholders = ','.join([f':project_{i}' for i in range(len(allowed_project_ids))])
filter_conditions.append(f"qc.project_id IN ({project_placeholders})")
for i, pid in enumerate(allowed_project_ids):
query_params[f'project_{i}'] = pid
# Filter by allowed locations
if allowed_location_names:
location_placeholders = ','.join([f':location_{i}' for i in range(len(allowed_location_names))])
filter_conditions.append(f"ad.location_name IN ({location_placeholders})")
for i, loc in enumerate(allowed_location_names):
query_params[f'location_{i}'] = loc
# ============================================================
# END: APPLY PROJECT MANAGER FILTERS
# ============================================================
# Apply user-selected filters
if date_from:
filter_conditions.append("ad.check_in_date >= :date_from")
query_params['date_from'] = date_from
if date_to:
filter_conditions.append("ad.check_in_date <= :date_to")
query_params['date_to'] = date_to
if location_filter:
# Exact match — dropdown value IS the exact location_name string
filter_conditions.append("ad.location_name = :location")
query_params['location'] = location_filter
if employee_ids:
# Expand each selected ID into every stored spelling so extra-work
# check-ins (SP / PW / PT) are included alongside regular records.
# Two branches: exact match on the raw column (uses the index), plus a
# REGEXP match that catches any separator style ("1234.PW", "1234-SP").
exact_variants, regex_patterns = expand_employee_id_filter(employee_ids)
# Never emit an empty IN () — fall back to the raw selection if expansion
# somehow produced nothing, so the filter can't degrade into "match all".
if not exact_variants:
exact_variants = list(employee_ids)
exact_placeholders = ', '.join([f':employee_{i}' for i in range(len(exact_variants))])
exact_condition = f"ad.employee_id IN ({exact_placeholders})"
for i, variant in enumerate(exact_variants):
query_params[f'employee_{i}'] = variant
if regex_patterns:
regex_clauses = ' OR '.join([
f"{UPPER_EMPLOYEE_ID_SQL} REGEXP :employee_re_{i}"
for i in range(len(regex_patterns))
])
filter_conditions.append(f"({exact_condition} OR {regex_clauses})")
for i, pattern in enumerate(regex_patterns):
query_params[f'employee_re_{i}'] = pattern
else:
filter_conditions.append(exact_condition)
if project_filter:
# For standard QR records: match by the QR code's project_id directly.
# For dynamic QR records: the dynamic QR itself may not be in any project,
# but the employee-selected location corresponds to a standard QR in that
# project. Match those by checking if attendance_data.location_name
# appears in the locations of QR codes belonging to the selected project.
filter_conditions.append(
"(qc.project_id = :project OR "
"(ad.is_dynamic_qr = 1 AND ad.location_name IN ("
" SELECT DISTINCT qc2.location FROM qr_codes qc2 "
" WHERE qc2.project_id = :project AND qc2.qr_type = 'standard' "
" AND qc2.location IS NOT NULL AND qc2.location != ''"
")))"
)
query_params['project'] = project_filter
return filter_conditions, query_params
@bp.route('/attendance', endpoint='attendance_report')
@login_required
@@ -225,93 +318,22 @@ def attendance_report():
WHERE 1=1
"""
# Prepare filter conditions and parameters
filter_conditions = []
query_params = {}
# ============================================================
# APPLY PROJECT MANAGER FILTERS TO SQL QUERY
# ============================================================
if user_role == 'project_manager':
# Filter by allowed projects
if allowed_project_ids:
project_placeholders = ','.join([f':project_{i}' for i in range(len(allowed_project_ids))])
filter_conditions.append(f"qc.project_id IN ({project_placeholders})")
for i, pid in enumerate(allowed_project_ids):
query_params[f'project_{i}'] = pid
# Filter by allowed locations
if allowed_location_names:
location_placeholders = ','.join([f':location_{i}' for i in range(len(allowed_location_names))])
filter_conditions.append(f"ad.location_name IN ({location_placeholders})")
for i, loc in enumerate(allowed_location_names):
query_params[f'location_{i}'] = loc
# ============================================================
# END: APPLY PROJECT MANAGER FILTERS
# ============================================================
# Apply user-selected filters
if date_from:
filter_conditions.append("ad.check_in_date >= :date_from")
query_params['date_from'] = date_from
if date_to:
filter_conditions.append("ad.check_in_date <= :date_to")
query_params['date_to'] = date_to
if location_filter:
# Exact match — dropdown value IS the exact location_name string
filter_conditions.append("ad.location_name = :location")
query_params['location'] = location_filter
# Prepare filter conditions and parameters — built by the helper shared
# with the live-update endpoint, so the Project Manager scope and the
# filters can never differ between the page and its live rows.
filter_conditions, query_params = _attendance_filter_conditions(
user_role, allowed_project_ids, allowed_location_names,
date_from, date_to, location_filter, employee_ids, project_filter)
if employee_ids:
# Expand each selected ID into every stored spelling so extra-work
# check-ins (SP / PW / PT) are included alongside regular records.
# Two branches: exact match on the raw column (uses the index), plus a
# REGEXP match that catches any separator style ("1234.PW", "1234-SP").
exact_variants, regex_patterns = expand_employee_id_filter(employee_ids)
# Never emit an empty IN () — fall back to the raw selection if expansion
# somehow produced nothing, so the filter can't degrade into "match all".
if not exact_variants:
exact_variants = list(employee_ids)
exact_placeholders = ', '.join([f':employee_{i}' for i in range(len(exact_variants))])
exact_condition = f"ad.employee_id IN ({exact_placeholders})"
for i, variant in enumerate(exact_variants):
query_params[f'employee_{i}'] = variant
if regex_patterns:
regex_clauses = ' OR '.join([
f"{UPPER_EMPLOYEE_ID_SQL} REGEXP :employee_re_{i}"
for i in range(len(regex_patterns))
])
filter_conditions.append(f"({exact_condition} OR {regex_clauses})")
for i, pattern in enumerate(regex_patterns):
query_params[f'employee_re_{i}'] = pattern
else:
filter_conditions.append(exact_condition)
exact_variant_count = len([k for k in query_params
if k.startswith('employee_') and not k.startswith('employee_re_')])
logger_handler.logger.info(
f"Attendance report filtered by employee IDs: {employee_ids} "
f"(matching {len(exact_variants)} ID variants incl. SP/PW/PT) "
f"(matching {exact_variant_count} ID variants incl. SP/PW/PT) "
f"by user {session.get('username', 'unknown')}"
)
if project_filter:
# For standard QR records: match by the QR code's project_id directly.
# For dynamic QR records: the dynamic QR itself may not be in any project,
# but the employee-selected location corresponds to a standard QR in that
# project. Match those by checking if attendance_data.location_name
# appears in the locations of QR codes belonging to the selected project.
filter_conditions.append(
"(qc.project_id = :project OR "
"(ad.is_dynamic_qr = 1 AND ad.location_name IN ("
" SELECT DISTINCT qc2.location FROM qr_codes qc2 "
" WHERE qc2.project_id = :project AND qc2.qr_type = 'standard' "
" AND qc2.location IS NOT NULL AND qc2.location != ''"
")))"
)
query_params['project'] = project_filter
# Combine query with filters
if filter_conditions:
base_query += " AND " + " AND ".join(filter_conditions)
@@ -320,6 +342,16 @@ def attendance_report():
ATTENDANCE_PAGE_LIMIT = 1000
base_query += f" ORDER BY ad.check_in_date DESC, ad.check_in_time DESC LIMIT {ATTENDANCE_PAGE_LIMIT + 1}"
# Starting cursor for the live table: the newest attendance id right now.
# Read BEFORE the report query, so a check-in saved in between is picked
# up by the first poll (the browser de-duplicates by id). Primary-key MAX.
try:
live_latest_id = int(db.session.execute(
text("SELECT COALESCE(MAX(id), 0) FROM attendance_data")).scalar() or 0)
except Exception as live_error:
logger_handler.logger.warning(f"Could not read latest attendance id for live updates: {live_error}")
live_latest_id = None # page still renders; live updates stay off
logger_handler.logger.debug(f"Executing attendance query with filters: {list(query_params.keys())}")
# Execute query
@@ -556,6 +588,7 @@ def attendance_report():
attendance_records=processed_records,
records_truncated=records_truncated,
records_limit=ATTENDANCE_PAGE_LIMIT,
live_latest_id=live_latest_id,
locations=locations,
projects=projects,
stats=stats,
@@ -587,6 +620,216 @@ def attendance_report():
flash('Error loading attendance report. Please check the server logs for details.', 'error')
return redirect(url_for('dashboard.dashboard'))
# ============================================================
# LIVE UPDATES for the Attendance Report table
# ============================================================
# The report page polls attendance_live_updates_api(). It is designed to cost
# almost nothing: every poll is one primary-key MAX(id) lookup, and the filtered
# query only runs when a newer record exists, scanning only ids above since_id.
LIVE_UPDATE_MAX_RECORDS = 200
# Cached once the column is confirmed; information_schema is not queried per poll
_live_has_location_accuracy = None
def _live_no_store_json(payload, status=200):
"""JSON response that browsers and proxies must never cache."""
response = jsonify(payload)
response.status_code = status
response.headers['Cache-Control'] = 'no-store, no-cache, max-age=0, must-revalidate'
response.headers['Pragma'] = 'no-cache'
return response
def _live_text(value):
"""Text as Jinja renders it into the report (None renders as 'None')."""
return 'None' if value is None else str(value)
def _live_format_time(check_in_time):
"""The HH:MM text the report template renders for a time / timedelta / str."""
if not check_in_time:
return 'N/A'
if isinstance(check_in_time, str):
return check_in_time
if hasattr(check_in_time, 'strftime'):
return check_in_time.strftime('%H:%M')
if hasattr(check_in_time, 'total_seconds'):
total_seconds = int(check_in_time.total_seconds())
return f"{(total_seconds // 3600) % 24:02d}:{(total_seconds % 3600) // 60:02d}"
return str(check_in_time)
def _live_record_payload(row, has_location_accuracy):
"""
One attendance row in exactly the shape loadTableData() in
static/js/attendance_report.js reads from the server-rendered table, so a
live row sorts, filters, paginates and renders like the rows loaded with
the page. Mirrors the <tbody> markup of templates/attendance_report.html
keep the two in sync.
"""
(record_id, employee_id, check_in_date, check_in_time, location_name, location_event,
qr_address, checked_in_address, location_accuracy, gps_accuracy, device_info,
created_timestamp, updated_timestamp, employee_name, verification_required,
verification_status, is_dynamic_qr) = row
# Verification badge, as extractVerificationData() reads it
if verification_required and verification_status == 'pending':
verification = (True, 'pending')
elif verification_status in ('approved', 'rejected'):
verification = (True, verification_status)
else:
verification = (False, None)
# Accuracy number, as extractLocationAccuracy() parses the rendered badge text
if verification[0]:
accuracy = round(float(location_accuracy), 3) if location_accuracy is not None else None
elif has_location_accuracy and location_accuracy:
accuracy = round(float(location_accuracy), 3)
elif gps_accuracy:
accuracy = round(float(gps_accuracy), 1) * 0.000621371 # badge shows metres
else:
accuracy = None
if isinstance(check_in_date, str):
date_text = check_in_date
else:
date_text = check_in_date.strftime('%m/%d/%Y') if check_in_date else ''
checked_in_text = (checked_in_address or 'N/A')[:50]
if len(checked_in_address or '') > 50:
checked_in_text += '...'
device_text = device_info or ''
device_text = device_text[:20] + ('...' if len(device_text) > 20 else '')
return {
'id': str(record_id),
'employeeId': _live_text(employee_id).strip(),
'employeeName': (employee_name or 'Unknown Employee').strip(),
'location': ('(No location recorded)' if location_name == 'Dynamic'
else _live_text(location_name)).strip(),
'event': _live_text(location_event).strip(),
'date': date_text.strip(),
'time': _live_format_time(check_in_time).strip(),
'qr_address': ((qr_address[:50] + ('...' if len(qr_address) > 50 else ''))
if qr_address else 'N/A').strip(),
'checked_in_address': checked_in_text.strip(),
'location_accuracy': accuracy,
'accuracy_level': ('unknown' if accuracy is None
else ('accurate' if accuracy < 0.3 else 'inaccurate')),
'device': device_text.strip(),
'isModified': bool(updated_timestamp and created_timestamp
and updated_timestamp > created_timestamp),
'isDynamic': bool(is_dynamic_qr) or location_name == 'Dynamic',
'verification_required': verification[0],
'verification_status': verification[1],
}
@bp.route('/api/attendance/live-updates', endpoint='attendance_live_updates_api')
@login_required
def attendance_live_updates_api():
"""
New attendance records for the Attendance Report's live table.
GET params: since_id (newest attendance id the page already knows) plus the
page's own filters: date_from, date_to, location, employee, project.
Returns {success, latest_id, records, has_more}; the page sends latest_id
back as since_id on its next poll.
"""
global _live_has_location_accuracy
try:
try:
since_id = int(request.args.get('since_id', ''))
except (TypeError, ValueError):
return _live_no_store_json({'success': False, 'message': 'since_id is required.'}, 400)
# Cheap path — taken by almost every poll: nothing newer than since_id
latest_id = int(db.session.execute(
text("SELECT COALESCE(MAX(id), 0) FROM attendance_data")).scalar() or 0)
empty = {'success': True, 'latest_id': max(latest_id, since_id), 'records': [], 'has_more': False}
if latest_id <= since_id:
return _live_no_store_json(empty)
# Same Project Manager scope as the report page
user_role = session.get('role')
allowed_project_ids, allowed_location_names = [], []
if user_role == 'project_manager':
user_id = session.get('user_id')
allowed_project_ids = [p.project_id for p in
UserProjectPermission.query.filter_by(user_id=user_id).all()]
allowed_location_names = [l.location_name for l in
UserLocationPermission.query.filter_by(user_id=user_id).all()]
if not allowed_project_ids and not allowed_location_names:
return _live_no_store_json(empty)
employee_filter = request.args.get('employee', '')
employee_ids = [e.strip() for e in employee_filter.split(',') if e.strip()]
filter_conditions, query_params = _attendance_filter_conditions(
user_role, allowed_project_ids, allowed_location_names,
request.args.get('date_from', ''), request.args.get('date_to', ''),
request.args.get('location', ''), employee_ids, request.args.get('project', ''))
# Only ids the page has not seen yet (a primary-key range scan)
filter_conditions = ["ad.id > :live_since_id", "ad.id <= :live_latest_id"] + filter_conditions
query_params['live_since_id'] = since_id
query_params['live_latest_id'] = latest_id
if _live_has_location_accuracy is None and check_location_accuracy_column_exists():
_live_has_location_accuracy = True
has_location_accuracy = bool(_live_has_location_accuracy)
accuracy_column = "ad.location_accuracy" if has_location_accuracy else "NULL"
# Same columns as the report query minus verification_photo (base64 image)
where_clause = " AND ".join(filter_conditions)
rows = db.session.execute(text(f"""
SELECT
ad.id,
ad.employee_id,
ad.check_in_date,
ad.check_in_time,
ad.location_name,
qc.location_event,
COALESCE(ad.qr_address, qc.location_address) AS qr_address,
ad.address AS checked_in_address,
{accuracy_column} AS location_accuracy,
ad.accuracy AS gps_accuracy,
ad.device_info,
ad.created_timestamp,
ad.updated_timestamp,
CONCAT(e.firstName, ' ', e.lastName) AS employee_name,
ad.verification_required,
ad.verification_status,
COALESCE(ad.is_dynamic_qr, 0) AS is_dynamic_qr
FROM attendance_data ad
LEFT JOIN qr_codes qc ON ad.qr_code_id = qc.id
LEFT JOIN employee e ON CAST(ad.employee_id AS UNSIGNED) = e.id
WHERE {where_clause}
ORDER BY ad.id ASC
LIMIT {LIVE_UPDATE_MAX_RECORDS + 1}
"""), query_params).fetchall()
has_more = len(rows) > LIVE_UPDATE_MAX_RECORDS
rows = rows[:LIVE_UPDATE_MAX_RECORDS]
# Capped batch: continue after the last row sent. Otherwise every id up
# to latest_id has been examined.
cursor = rows[-1][0] if has_more else latest_id
records = []
for row in rows:
try:
records.append(_live_record_payload(row, has_location_accuracy))
except Exception as rec_error:
logger_handler.logger.warning(f"Live updates: skipped attendance record {row[0]}: {rec_error}")
return _live_no_store_json({'success': True, 'latest_id': cursor,
'records': records, 'has_more': has_more})
except Exception as e:
db.session.rollback()
logger_handler.logger.error(f"Error loading attendance live updates: {e}", exc_info=True)
return _live_no_store_json({'success': False, 'message': 'Could not load new records.'}, 500)
@bp.route('/api/time-attendance/locations', endpoint='time_attendance_locations_api')
@login_required
def time_attendance_locations_api():
+318 -50
View File
@@ -24,6 +24,7 @@ document.addEventListener("DOMContentLoaded", function () {
initializeCharts();
setupEventListeners();
initializeDateRangeFilters();
initializeLiveUpdates();
});
function initializeReport() {
@@ -164,6 +165,20 @@ function stripLeadingZeros(value) {
}
function applyFilters() {
filteredData = filterRecords();
currentPage = 1;
updateTable();
updatePagination();
updateFilterStats();
}
/**
* Records matching the on-page search / location / employee filters.
* Shared by applyFilters() and refreshTableKeepingView() (live updates), which
* must keep the user's current page and sort instead of jumping to page 1.
*/
function filterRecords() {
const searchTerm =
document.getElementById("searchInput")?.value.toLowerCase() || "";
const locationFilter = document.getElementById("location")?.value || "";
@@ -184,7 +199,7 @@ function applyFilters() {
.map(parseEmployeeIdWorkType)
: [];
filteredData = attendanceData.filter((record) => {
return attendanceData.filter((record) => {
const matchesSearch =
!searchTerm ||
record.employeeId.toLowerCase().includes(searchTerm) ||
@@ -209,11 +224,6 @@ function applyFilters() {
return matchesSearch && matchesLocation && matchesEmployee;
});
currentPage = 1;
updateTable();
updatePagination();
updateFilterStats();
}
function sortTable(columnIndex) {
@@ -687,6 +697,10 @@ function createTableRow(record, displayIndex) {
if (record.isDynamic) {
row.classList.add('dynamic-qr-record');
}
// Brief highlight for rows delivered by live updates
if (record.isLiveNew) {
row.classList.add('live-new-record');
}
// Debug logging for first few records
if (displayIndex <= 3) {
@@ -704,7 +718,7 @@ function createTableRow(record, displayIndex) {
if (record.verification_required && record.verification_status === 'pending') {
// Show Review Needed badge for pending verification - LINK to review page
locationAccuracyBadge = `<a href="/verification-review/${record.id}"
locationAccuracyBadge = `<a href="/verification-review/${encodeURIComponent(record.id)}"
class="location-accuracy-badge badge-review-needed"
style="cursor: pointer; text-decoration: none;"
title="Click to review verification photo - Distance: ${record.location_accuracy ? record.location_accuracy.toFixed(3) : 'N/A'} miles">
@@ -782,12 +796,8 @@ function createTableRow(record, displayIndex) {
addressDisplayHTML = `
<i class="${addressIcon}" style="color: #059669; margin-right: 4px;"
title="High accuracy - showing QR location"></i>
<span title="${addressTitle}" class="${addressClass}">
${
addressToShow.length > 45
? addressToShow.substring(0, 45) + "..."
: addressToShow
}
<span title="${escapeHtml(addressTitle)}" class="${addressClass}">
${escapeHtml(truncateText(addressToShow, 45))}
</span>
`;
} else {
@@ -799,14 +809,10 @@ function createTableRow(record, displayIndex) {
addressDisplayHTML = `
<i class="fas fa-exclamation-triangle" style="color: #f59e0b; margin-right: 4px;"
title="Lower accuracy - showing actual check-in location"></i>
<span title="${addressTitle} (Accuracy: ${accuracy.toFixed(
<span title="${escapeHtml(addressTitle)} (Accuracy: ${accuracy.toFixed(
3
)} mi)" class="${addressClass}">
${
addressToShow.length > 45
? addressToShow.substring(0, 45) + "..."
: addressToShow
}
${escapeHtml(truncateText(addressToShow, 45))}
</span>
`;
}
@@ -820,12 +826,8 @@ function createTableRow(record, displayIndex) {
addressDisplayHTML = `
<i class="${addressIcon}"></i>
<span title="${addressTitle}" class="${addressClass}">
${
addressToShow.length > 45
? addressToShow.substring(0, 45) + "..."
: addressToShow
}
<span title="${escapeHtml(addressTitle)}" class="${addressClass}">
${escapeHtml(truncateText(addressToShow, 45))}
</span>
`;
}
@@ -834,45 +836,41 @@ function createTableRow(record, displayIndex) {
<td>${displayIndex}</td>
<td>
<div class="employee-info">
<span class="employee-id">${record.employeeId}</span>
<span class="employee-id">${escapeHtml(record.employeeId)}</span>
</div>
</td>
<td>
<div class="employee-name">
<i class="fas fa-user"></i>
<span>${record.employeeName || 'Unknown'}</span>
<span>${escapeHtml(record.employeeName || 'Unknown')}</span>
</div>
</td>
<td>
<div class="location-info">
<i class="fas fa-map-marker-alt"></i>
${record.location}
${escapeHtml(record.location)}
</div>
</td>
<td>
<div class="event-info">
${record.event}
${escapeHtml(record.event)}
</div>
</td>
<td>
<div class="date-info">
${record.date}
${escapeHtml(record.date)}
</div>
</td>
<td>
<div class="time-info">
${record.time}
${escapeHtml(record.time)}
</div>
</td>
<td>
<div class="address-info qr-address">
<i class="fas fa-qrcode" style="color: #6366f1; margin-right: 4px;" title="QR Code Address (Fixed)"></i>
<span title="QR Address: ${record.qr_address}">
${
record.qr_address.length > 50
? record.qr_address.substring(0, 50) + "..."
: record.qr_address
}
<span title="QR Address: ${escapeHtml(record.qr_address)}">
${escapeHtml(truncateText(record.qr_address, 50))}
</span>
</div>
</td>
@@ -889,12 +887,8 @@ function createTableRow(record, displayIndex) {
<td>
<div class="device-info">
<i class="fas fa-mobile-alt"></i>
<span title="${record.device}">
${
record.device.length > 20
? record.device.substring(0, 20) + "..."
: record.device
}
<span title="${escapeHtml(record.device)}">
${escapeHtml(truncateText(record.device, 20))}
</span>
</div>
</td>
@@ -902,7 +896,7 @@ function createTableRow(record, displayIndex) {
<div class="record-actions">
${
record.verification_required && record.verification_status === 'pending'
? `<a href="/verification-review/${record.id}"
? `<a href="/verification-review/${encodeURIComponent(record.id)}"
class="action-btn btn-review"
title="Review Verification Photo">
<i class="fas fa-camera"></i>
@@ -912,12 +906,12 @@ function createTableRow(record, displayIndex) {
${
hasEditPermission
? `
<button onclick="editRecord('${record.id}')"
<button onclick="editRecord('${escapeHtml(escapeJsString(record.id))}')"
class="action-btn btn-edit"
title="Edit Record">
<i class="fas fa-edit"></i>
</button>
<button onclick="deleteRecord('${record.id}', '${record.employeeId}')"
<button onclick="deleteRecord('${escapeHtml(escapeJsString(record.id))}', '${escapeHtml(escapeJsString(record.employeeId))}')"
class="action-btn btn-delete"
title="Delete Record">
<i class="fas fa-trash"></i>
@@ -961,6 +955,18 @@ function sortTable(columnIndex) {
sortDirection = "asc";
}
sortFilteredData();
updateTable();
updateSortIndicators(columnIndex);
}
/**
* Sort filteredData by the current sortColumn / sortDirection without toggling
* the direction reused when live updates re-apply the user's chosen sort.
*/
function sortFilteredData() {
const columnIndex = sortColumn;
const sortKey = getSortKey(columnIndex);
filteredData.sort((a, b) => {
@@ -993,9 +999,6 @@ function sortTable(columnIndex) {
return sortDirection === "asc" ? result : -result;
});
updateTable();
updateSortIndicators(columnIndex);
}
// Enhanced statistics display for location accuracy
@@ -1240,4 +1243,269 @@ document.addEventListener("DOMContentLoaded", function () {
}
console.log("Enhanced export functionality initialized");
});
});
// ============================================================
// HTML SAFETY HELPERS
// ============================================================
// Row values are plain text (read with textContent, or JSON from the server)
// but createTableRow() builds markup with innerHTML. Device and address text
// come from the public check-in page, so every value is escaped first.
function escapeHtml(value) {
return String(value === null || value === undefined ? "" : value)
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/'/g, "&#39;");
}
// For a value placed inside a single-quoted JS string in an onclick attribute:
// escape for JS here, then escapeHtml() for the attribute.
function escapeJsString(value) {
return String(value === null || value === undefined ? "" : value)
.replace(/\\/g, "\\\\")
.replace(/'/g, "\\'")
.replace(/[\r\n]+/g, " ");
}
// Truncate BEFORE escaping, so an entity like &amp; is never cut in half
function truncateText(value, maxLength) {
const text = String(value === null || value === undefined ? "" : value);
return text.length > maxLength ? text.substring(0, maxLength) + "..." : text;
}
// ============================================================
// LIVE UPDATES — new check-ins appear without reloading the page
// ============================================================
// Polls GET /api/attendance/live-updates (routes/attendance.py), which answers
// from a primary-key MAX(id) lookup when nothing is new. To keep the load
// negligible the poll:
// - runs every LIVE_POLL_INTERVAL_MS and never overlaps itself
// - pauses while the tab is hidden, and checks once as soon as it is shown
// - backs off exponentially after errors, up to LIVE_MAX_BACKOFF_MS
// - stops for good when the session has ended (redirect / 401 / 403)
// New rows keep the user's current page, sort and search.
const LIVE_POLL_INTERVAL_MS = 20000;
const LIVE_MAX_BACKOFF_MS = 5 * 60 * 1000;
const LIVE_HIGHLIGHT_MS = 8000;
const LIVE_MAX_RECORDS = 5000; // rows kept in memory on a page left open for days
const LIVE_FILTER_PARAMS = ["date_from", "date_to", "location", "employee", "project"];
const liveUpdates = {
enabled: false,
sinceId: 0,
timer: null,
inFlight: false,
failures: 0,
stopped: false,
newSinceLoad: 0,
statusEl: null,
};
function initializeLiveUpdates() {
const container = document.getElementById("attendanceReportContainer");
const sinceId = container ? parseInt(container.dataset.liveSinceId, 10) : NaN;
if (isNaN(sinceId)) {
return; // rendered without a cursor (no access / id lookup failed): stay static
}
liveUpdates.enabled = true;
liveUpdates.sinceId = sinceId;
createLiveStatusIndicator();
setLiveStatus("live");
document.addEventListener("visibilitychange", function () {
if (!liveUpdates.enabled || liveUpdates.stopped) return;
if (document.hidden) {
clearTimeout(liveUpdates.timer); // no requests from background tabs
} else {
pollLiveUpdates(); // catch up straight away when the tab is shown again
}
});
scheduleLivePoll(LIVE_POLL_INTERVAL_MS);
}
function scheduleLivePoll(delayMs) {
clearTimeout(liveUpdates.timer);
if (liveUpdates.stopped || document.hidden) return;
liveUpdates.timer = setTimeout(pollLiveUpdates, delayMs);
}
function pollLiveUpdates() {
if (!liveUpdates.enabled || liveUpdates.stopped || liveUpdates.inFlight || document.hidden) {
return;
}
clearTimeout(liveUpdates.timer);
liveUpdates.inFlight = true;
// The server applies the same filters the page was loaded with
const pageParams = new URLSearchParams(window.location.search);
const params = new URLSearchParams();
LIVE_FILTER_PARAMS.forEach(function (name) {
const value = pageParams.get(name);
if (value) params.set(name, value);
});
params.set("since_id", String(liveUpdates.sinceId));
let nextDelay = LIVE_POLL_INTERVAL_MS;
fetch("/api/attendance/live-updates?" + params.toString(), {
cache: "no-store",
credentials: "same-origin",
headers: { "X-Requested-With": "XMLHttpRequest", Accept: "application/json" },
})
.then(function (response) {
const contentType = response.headers.get("Content-Type") || "";
if (
response.redirected ||
response.status === 401 ||
response.status === 403 ||
(response.ok && contentType.indexOf("application/json") === -1)
) {
// login_required redirected to the login page: the session is over
stopLiveUpdates();
return null;
}
if (!response.ok) {
throw new Error("HTTP " + response.status);
}
return response.json();
})
.then(function (data) {
if (!data) return;
if (!data.success) {
throw new Error(data.message || "live update failed");
}
liveUpdates.failures = 0;
if (Array.isArray(data.records) && data.records.length > 0) {
insertLiveRecords(data.records);
}
const latestId = parseInt(data.latest_id, 10);
if (!isNaN(latestId) && latestId > liveUpdates.sinceId) {
liveUpdates.sinceId = latestId;
}
if (data.has_more) {
nextDelay = 1000; // a large burst arrived: fetch the next batch promptly
}
if (!liveUpdates.stopped) setLiveStatus("live");
})
.catch(function (error) {
liveUpdates.failures += 1;
nextDelay = Math.min(
LIVE_POLL_INTERVAL_MS * Math.pow(2, liveUpdates.failures),
LIVE_MAX_BACKOFF_MS
);
console.log(
"Live updates: check failed (" + error.message + "), retrying in " +
Math.round(nextDelay / 1000) + "s"
);
setLiveStatus("retry");
})
.then(function () {
liveUpdates.inFlight = false;
if (!liveUpdates.stopped) scheduleLivePoll(nextDelay);
});
}
function stopLiveUpdates() {
liveUpdates.stopped = true;
clearTimeout(liveUpdates.timer);
setLiveStatus("stopped");
}
function insertLiveRecords(records) {
// The page showed "No Attendance Records Found" (no table to add to):
// reload once, keeping the filters, so the table renders normally.
if (!document.getElementById("attendanceTable")) {
stopLiveUpdates();
if (typeof refreshReport === "function") {
refreshReport();
} else {
window.location.reload();
}
return;
}
const knownIds = new Set(attendanceData.map(function (r) { return String(r.id); }));
const fresh = records
.filter(function (r) { return !knownIds.has(String(r.id)); })
.sort(function (a, b) { return Number(b.id) - Number(a.id); }); // newest first
if (fresh.length === 0) return;
fresh.forEach(function (r) { r.isLiveNew = true; });
attendanceData = fresh.concat(attendanceData);
if (attendanceData.length > LIVE_MAX_RECORDS) {
attendanceData.length = LIVE_MAX_RECORDS;
}
attendanceData.forEach(function (r, i) { r.index = i + 1; });
liveUpdates.newSinceLoad += fresh.length;
refreshTableKeepingView();
// Rows re-rendered after this (page change, sort) no longer highlight
setTimeout(function () {
fresh.forEach(function (r) { r.isLiveNew = false; });
}, LIVE_HIGHLIGHT_MS);
}
// Re-render after new rows arrive WITHOUT resetting the user's view:
// same search/filters, same sort, same page (clamped if it no longer exists).
function refreshTableKeepingView() {
filteredData = filterRecords();
if (sortColumn !== -1) {
sortFilteredData();
}
if (entriesPerPage !== "all") {
const totalPages = Math.max(1, Math.ceil(filteredData.length / entriesPerPage));
if (currentPage > totalPages) currentPage = totalPages;
}
updateTable();
updatePagination();
}
function createLiveStatusIndicator() {
const heading = document.querySelector(".attendance-table-section .table-header h3");
if (!heading) return;
const status = document.createElement("span");
status.id = "liveUpdateStatus";
status.className = "live-status";
const dot = document.createElement("span");
dot.className = "live-dot";
const text = document.createElement("span");
text.className = "live-text";
status.appendChild(dot);
status.appendChild(text);
heading.appendChild(status);
liveUpdates.statusEl = status;
}
function setLiveStatus(state) {
const status = liveUpdates.statusEl;
if (!status) return;
const text = status.querySelector(".live-text");
status.classList.remove("live-status--retry", "live-status--stopped");
if (state === "retry") {
status.classList.add("live-status--retry");
text.textContent = "Reconnecting…";
status.title = "Could not check for new records. Retrying automatically.";
} else if (state === "stopped") {
status.classList.add("live-status--stopped");
text.textContent = "Live updates paused";
status.title = "Live updates stopped. Refresh the page to resume.";
} else {
text.textContent = liveUpdates.newSinceLoad > 0
? "Live · " + liveUpdates.newSinceLoad + " new"
: "Live";
status.title =
"New check-ins appear automatically (checked every " +
LIVE_POLL_INTERVAL_MS / 1000 +
" seconds, paused while this tab is hidden). Last checked " +
new Date().toLocaleTimeString() + ".";
}
}
+60 -1
View File
@@ -14,6 +14,65 @@ tr.dynamic-qr-record {
tr.dynamic-qr-record:hover {
background-color: rgba(59, 91, 219, 0.07) !important;
}
/* Live updates: status pill in the table header + highlight for rows that
arrived while the page was open (see LIVE UPDATES in attendance_report.js) */
.live-status {
display: inline-flex;
align-items: center;
gap: 0.4rem;
margin-left: 0.75rem;
padding: 0.15rem 0.6rem;
border-radius: 999px;
font-size: 0.75rem;
font-weight: 600;
vertical-align: middle;
background: #dcfce7;
color: #166534;
cursor: default;
}
.live-status .live-dot {
width: 0.5rem;
height: 0.5rem;
border-radius: 50%;
background: #16a34a;
animation: live-pulse 2s ease-in-out infinite;
}
.live-status.live-status--retry {
background: #fef3c7;
color: #92400e;
}
.live-status.live-status--retry .live-dot {
background: #f59e0b;
}
.live-status.live-status--stopped {
background: #f1f5f9;
color: #475569;
}
.live-status.live-status--stopped .live-dot {
background: #94a3b8;
animation: none;
}
@keyframes live-pulse {
0%, 100% { opacity: 1; }
50% { opacity: 0.35; }
}
tr.live-new-record > td {
animation: live-row-in 6s ease-out;
}
@keyframes live-row-in {
0% { background-color: #bbf7d0; }
100% { background-color: transparent; }
}
@media (prefers-reduced-motion: reduce) {
.live-status .live-dot,
tr.live-new-record > td {
animation: none;
}
tr.live-new-record > td {
background-color: #f0fdf4;
}
}
</style>
<!-- Fullscreen CSS -->
<link rel="stylesheet" href="{{ url_for('static', filename='css/attendance_fullscreen.css') }}">
@@ -30,7 +89,7 @@ tr.dynamic-qr-record:hover {
</button>
<!-- Attendance Report Container with Fullscreen Support -->
<div id="attendanceReportContainer" class="attendance-page" data-user-role="{{ session.role }}">
<div id="attendanceReportContainer" class="attendance-page" data-user-role="{{ session.role }}"{% if live_latest_id is defined and live_latest_id is not none %} data-live-since-id="{{ live_latest_id }}"{% endif %}>
<!-- Header Section -->
<div class="attendance-header">
<div class="header-content">