# TechDesk — IT Helpdesk System: Knowledge Base for Claude This document is the authoritative reference for Claude when working on the TechDesk codebase. It captures the full architecture, design decisions, known pitfalls, and hard-won lessons from the development history of this application. --- ## 1. Project Overview **TechDesk** is an internal IT helpdesk web application deployed at `tickets.ltservicesinc.com`. It is built for LT Services Inc. and serves two audiences: - **Employees** — submit and track IT support tickets - **IT Staff / Admins** — manage, assign, resolve, and report on tickets ### Technology Stack | Layer | Technology | |---|---| | Language | Python 3 | | Web Framework | Flask | | ORM | SQLAlchemy (via Flask-SQLAlchemy) | | Database | MySQL (driver: PyMySQL) | | Migrations | Alembic (via Flask-Migrate) | | Templates | Jinja2 | | Real-time | Flask-SocketIO (eventlet async mode) | | WSGI Server | Gunicorn (single worker, eventlet worker class) | | Reverse Proxy | nginx (port 7000 upstream, SSL via Certbot) | | Auth | Flask-Login | | Forms / CSRF | Flask-WTF | | Rate Limiting | Flask-Limiter | | Email | Flask-Mail | | Background Jobs | APScheduler (BackgroundScheduler, daemon=True) | | Rich Text Editor | TinyMCE 6 | | AI Chatbot | Groq API (`llama-3.3-70b-versatile`) | | HTML Sanitization | bleach | | Static Files | Served directly by nginx from `/home/it-ticket/myapp/app/static` | ### Deployment Path ``` /home/it-ticket/myapp/ ├── app/ │ ├── __init__.py # App factory (create_app) │ ├── models.py # All SQLAlchemy models │ ├── routes/ │ │ ├── admin.py # /admin/* — IT staff and admin routes │ │ ├── api.py # /api/* — JSON endpoints + SocketIO events │ │ ├── auth.py # /auth/* — login, register, password reset │ │ ├── chatbot.py # /chatbot/* — AI chatbot endpoint │ │ └── tickets.py # / and /tickets/* — employee-facing routes │ ├── services/ │ │ ├── email_ingestion_service.py │ │ ├── log_service.py │ │ ├── notification_service.py │ │ ├── sla_service.py │ │ └── validation_service.py │ ├── templates/ │ │ ├── base.html │ │ ├── admin/ # Admin panel templates │ │ ├── auth/ # Login, register, password reset templates │ │ └── tickets/ # Employee-facing ticket templates │ └── static/ │ └── uploads/ # User-uploaded files (default; overridable via UPLOAD_FOLDER) ├── config/ │ └── config.py # DevelopmentConfig / ProductionConfig ├── migrations/ │ └── versions/ # Alembic migration scripts ├── gunicorn.conf.py ├── run.py # Entry point — calls eventlet.monkey_patch() first └── .env # Secrets (not committed) ``` --- ## 2. User Roles & Access Control Three roles are defined in `UserRole`: | Role | Constant | Access | |---|---|---| | Employee | `employee` | Submit tickets, view own tickets, knowledge base, chatbot | | IT Staff | `it_staff` | Everything above + manage all tickets, assignment, bulk actions | | Admin | `admin` | Everything above + user management, system settings, ticket deletion, KB management | ### Decorators (defined in `admin.py`) ```python @admin_required # role == 'admin' only @it_required # role in ('it_staff', 'admin') ``` Both decorators must be applied **after** `@login_required`. --- ## 3. Database Models (`app/models.py`) ### Core Models | Model | Table | Key Notes | |---|---|---| | `User` | `users` | Roles: employee / it_staff / admin. `is_admin` and `is_it_staff` are properties. | | `Ticket` | `tickets` | Central entity. Has `generate_ticket_number()` with `FOR UPDATE` row locking to prevent race conditions. | | `Comment` | `comments` | Bodies stored as sanitized HTML (rendered at write time via `render_comment_body`). | | `Attachment` | `attachments` | Tied to both `ticket_id` and `comment_id`. Physical files live in `UPLOAD_FOLDER`. | | `Notification` | `notifications` | Per-user; read/unread. Cascades delete with ticket. | | `TicketHistory` | `ticket_history` | Field-level audit trail. `old_value`/`new_value` are TEXT (widened in migration 001). | | `ActivityLog` | `activity_logs` | System-wide action log. `user_id` has `ondelete='SET NULL'`. | | `KnowledgeBase` | `knowledge_base` | Articles with TinyMCE body, tags, view count, publish toggle. | | `KBAttachment` | `kb_attachments` | Files attached to KB articles. Shared `UPLOAD_FOLDER` with ticket attachments. | | `SystemSetting` | `system_settings` | Key-value config store. Use `SystemSetting.get(key)` / `SystemSetting.set(key, value)`. | | `TicketLink` | `ticket_links` | Symmetric bidirectional links between tickets (related / duplicate / follow_up). | | `CannedResponse` | `canned_responses` | Pre-written reply templates for IT staff. | | `KBFeedback` | `kb_feedback` | One thumbs up/down vote per user per article. | | `PasswordResetToken` | `password_reset_tokens` | SHA-256 hashed, 1-hour expiry, single-use. | | `TicketTemplate` | `ticket_templates` | Pre-filled ticket scaffolds selectable on the new ticket form. | | `TicketSatisfaction` | `ticket_satisfaction` | One survey per resolved ticket; token-authenticated survey URL. | | `TicketWatcher` | `ticket_watchers` | Users subscribed to updates on a ticket. Unique `(ticket_id, user_id)`. Added migration 005. | | `TimeEntry` | `time_entries` | IT staff time log per ticket. Stores `minutes` + optional `note`. Added migration 005. | ### Cascade Rules `Ticket` has `cascade='all, delete-orphan'` on: - `comments` - `attachments` - `notifications` - `history` - `links_as_source` / `links_as_target` (via `TicketLink`) - `satisfaction` (via `TicketSatisfaction`) - `watchers` (via `TicketWatcher`) - `time_entries` (via `TimeEntry`) **When deleting a ticket, physical files in `UPLOAD_FOLDER` must be removed manually before the DB delete** — SQLAlchemy cascades handle DB rows only. --- ## 4. Blueprint & Route Structure | Blueprint | Prefix | File | Guard | |---|---|---|---| | `auth_bp` | (none) | `routes/auth.py` | Public + `@login_required` per route | | `tickets_bp` | (none) | `routes/tickets.py` | `@login_required` per route | | `admin_bp` | `/admin` | `routes/admin.py` | `@login_required` + `@it_required` or `@admin_required` | | `api_bp` | `/api` | `routes/api.py` | `@login_required` per route | | `chatbot_bp` | `/chatbot` | `routes/chatbot.py` | `@login_required` per route | ### Key Routes Reference **Auth (`auth.py`)** - `GET/POST /login` — rate limited: 10/min, 50/hr - `GET/POST /register` — rate limited: 5/min, 20/hr; guarded by `registration_enabled` SystemSetting - `GET /logout` - `GET/POST /profile` - `GET/POST /forgot-password` — rate limited: 5/hr - `GET/POST /reset-password/` — rate limited: 10/hr - `GET /avatar/` — login required (prevents unauthenticated access) - `GET /logo/` — public (used in login page) **Tickets (`tickets.py`)** - `GET /` and `GET /dashboard` — role-aware: employees see `dashboard_employee.html`, IT staff see `dashboard_it.html` - `GET/POST /tickets/new` - `GET/POST /tickets/behalf` — IT staff only (file ticket on behalf of employee) - `GET /tickets` — employee ticket list (own tickets only) - `GET/POST /tickets/` — ticket detail + comment submission - `POST /tickets//update` — IT staff update (status, priority, assignment, etc.) - `POST /comments//delete` - `GET /attachments/` — file download - `GET /notifications` / `POST /notifications/mark-read` - `GET /kb` / `GET /kb/` — knowledge base - `POST /tickets//link` / `POST /tickets//unlink/` - `POST /tickets//reopen` - `POST /tickets//watch` / `POST /tickets//unwatch` — AJAX; toggle watcher subscription - `POST /tickets//log-time` — IT staff only; AJAX-aware; logs `TimeEntry` - `GET /canned-responses` — JSON endpoint for IT staff comment box - `POST /kb//feedback` - `GET/POST /survey/` — public (no login required) **Admin (`admin.py`)** - `GET /admin/` — IT Operations Overview dashboard - `GET /admin/users` / `POST /admin/users/new` / `POST /admin/users//edit` / `POST /admin/users//delete` - `GET /admin/tickets` — all tickets with search, filter, pagination - `POST /admin/tickets//delete` — **admin only**; deletes ticket + physical files - `POST /admin/tickets/bulk-action` — actions: resolve, close, assign_me, assign_to, unassign, **delete** (delete is admin-only; `assign_to` requires `assign_to_id` form field) - `GET /admin/tickets/export` — CSV export with active filters - `GET /admin/kb` / `GET/POST /admin/kb/new` / `GET/POST /admin/kb//edit` / `POST /admin/kb//delete` / `POST /admin/kb//publish` - `POST /admin/kb/upload-image` — TinyMCE image upload - `GET /admin/kb/files/` — file serve (login required) - `GET/POST /admin/logs` — activity log viewer - `GET/POST /admin/canned-responses/*` - `POST /admin/tickets/bulk-action` - `GET/POST /admin/ticket-templates/*` - `GET /admin/satisfaction` — satisfaction report - `GET/POST /admin/settings` **API (`api.py`)** - `GET /api/notifications` — JSON list - `GET /api/notifications/unread-count` - `POST /api/notifications//read` - `POST /api/notifications/mark-all-read` - `GET /api/tickets//comments` — JSON (used by real-time comment rendering) - `GET /api/stats/tickets` — dashboard KPI stats - `GET /api/users/search` — typeahead for ticket assignment - `GET /api/tickets/search` — typeahead for ticket linking - SocketIO events: `connect`, `disconnect`, `join_ticket`, `leave_ticket` **Chatbot (`chatbot.py`)** - `POST /chatbot/message` — rate limited: 20/min, 100/hr; calls Groq API --- ## 5. Services ### `log_service.py` ```python log_action(user_id, action, entity_type=None, entity_id=None, details=None) log_ticket_history(ticket, field_name, old_value, new_value, changed_by_id) ``` **Critical rules:** - Neither function calls `db.session.commit()`. The caller must commit. - On failure, only the failed log entry is `expunge()`d — **never call `db.session.rollback()`** inside these functions, as that would undo the parent operation. - Always call `log_action` **before** `db.session.commit()`. - `log_ticket_history` records granular field-level changes on tickets. **Standard action strings used across the codebase:** | Action String | Where | |---|---| | `ticket_create` | New ticket submitted | | `ticket_update` | Ticket fields changed | | `comment_create` | Comment added | | `comment_delete` | Comment deleted | | `admin_user_create` | Admin creates a user | | `admin_user_edit` | Admin edits a user | | `admin_user_deactivate` | Admin deactivates a user | | `admin_ticket_delete` | Admin deletes a ticket | | `ticket_bulk_resolve` | Bulk resolve | | `ticket_bulk_close` | Bulk close | | `ticket_bulk_assign_me` | Bulk assign to self | | `ticket_bulk_assign_to` | Bulk assign to specific user | | `ticket_bulk_unassign` | Bulk unassign | | `kb_create` / `kb_edit` / `kb_delete` | KB article CRUD | | `kb_attachment_delete` | KB attachment removed | | `time_log` | IT staff logs time on a ticket | | `ticket_create_chatbot` | Ticket created via AI chatbot | ### `notification_service.py` Functions: `notify_new_ticket`, `notify_status_change`, `notify_comment_added`, `notify_assignment`, `send_satisfaction_survey`, `notify_watchers`, `send_weekly_digest` **Critical rule:** Always call notification functions **after** `db.session.commit()`. Calling them before commit means they act on uncommitted state. - **`notify_watchers(ticket, event_title, event_message, exclude_user_id=None)`** — sends in-app notifications to all `TicketWatcher` rows for a ticket. Call after commit from `update_ticket`. Pass `exclude_user_id=current_user.id` so the actor doesn't notify themselves. - **`send_weekly_digest(app)`** — APScheduler job; runs every Monday at 08:00 UTC via `CronTrigger`. Sends a rich HTML email to all active IT staff / admin users with `email_notif=True`. Summarises new, resolved, still-open, and overdue tickets for the past 7 days. ### `sla_service.py` - `set_due_date(ticket, app)` — called at ticket creation; reads SLA hours from `SystemSetting` - `check_sla_breaches(app)` — runs every 30 minutes via APScheduler; creates notifications for overdue tickets - `clear_sla_notification(ticket_id)` — called when a ticket is resolved or closed ### `validation_service.py` - `validate_password(password, confirm)` — returns error string or None - `validate_file(file, allowed_extensions)` — magic-byte MIME validation (not just extension) - `render_comment_body(raw_text)` — converts Markdown/plain text to sanitized HTML via bleach; called at write time, not display time ### `email_ingestion_service.py` - `check_inbound_email(app)` — APScheduler job; polls IMAP mailbox and converts emails to tickets - Deduplication via `Message-ID` headers stored in `SystemSetting` - Unknown senders fall back to the system admin user account --- ## 6. Configuration & Environment ### `.env` Keys ``` SECRET_KEY= DB_USER= DB_PASSWORD= DB_HOST= DB_PORT=3306 DB_NAME=it_tickets MAIL_SERVER= MAIL_PORT=587 MAIL_USE_TLS=True MAIL_USERNAME= MAIL_PASSWORD= MAIL_DEFAULT_SENDER= IT_DEPT_EMAIL= APP_BASE_URL=https://tickets.ltservicesinc.com UPLOAD_FOLDER= # Absolute path; defaults to app/static/uploads GROQ_API_KEY= GROQ_MODEL=llama-3.3-70b-versatile ADMIN_EMAIL= ADMIN_PASSWORD= RATELIMIT_STORAGE_URI=memory:// ``` ### `SystemSetting` Keys (runtime config, editable via Admin → Settings) | Key | Default | Description | |---|---|---| | `registration_enabled` | `true` | Allow self-registration | | `survey_enabled` | `true` | Send satisfaction survey on resolve | | `app_timezone` | `America/New_York` | Display timezone for all dates | | `sla_critical_hours` | `4` | SLA breach threshold for CRITICAL | | `sla_high_hours` | `8` | SLA breach threshold for HIGH | | `sla_medium_hours` | `48` | SLA breach threshold for MEDIUM | | `sla_low_hours` | `120` | SLA breach threshold for LOW | | `app_name` | `TechDesk` | Sidebar/page title branding | | `app_subtitle` | `IT Helpdesk System` | Sidebar subtitle | | `company_name` | `` | Login/register page branding | | `logo_stored_name` | `` | Uploaded logo filename | | `logo_initials` | `TD` | Fallback initials when no logo | | `primary_color` | `#2563eb` | CSS accent color (hex) | | `email_ingestion_enabled` | `0` | Enable IMAP email-to-ticket | | `email_ingestion_host/port/user/password/folder/move_to/interval` | various | IMAP settings | --- ## 7. Known Pitfalls & Hard Rules These are non-negotiable lessons learned from production incidents and debugging sessions. Violating any of these will introduce bugs. ### 7.1 eventlet / Gunicorn Ordering - `eventlet.monkey_patch()` **must** run before the app is loaded. - It is called in `run.py` at module level before `create_app()`. - `preload_app = False` in `gunicorn.conf.py` is **mandatory**. Setting it to `True` causes the app to load in the master process before `monkey_patch()` runs, leaving real OS RLocks un-greened and triggering: `"1 RLock(s) were not greened"` ### 7.2 Flask-Limiter Storage - Flask-Limiter does **not** support `mysql+pymysql://` as a storage URI. - Always use `memory://` (or `redis://` if Redis becomes available). - The `RATELIMIT_STORAGE_URI` config key controls this. ### 7.3 SQLAlchemy Session & Generators - Generators that access SQLAlchemy ORM objects execute **lazily**. - If a generator is returned from a route and consumed outside the request context (after the session closes), it silently yields nothing. - Always collect ORM results into a list **within** the session context before yielding or returning from a route. This was the cause of empty CSV exports. ### 7.4 `db.create_all()` and Alembic - `db.create_all()` is called in `create_app()` for first-run convenience. - If tables already exist (normal in production), `create_all()` is a no-op. - **Do not use `db.create_all()` as a substitute for migrations.** New columns added in migrations will not be created by `create_all()` on existing tables. - Alembic revision IDs must fit within `VARCHAR(64)` in `alembic_version` (widened in migration 003). Do not use revision IDs longer than 64 characters. - Never break the migration chain: each migration's `Revises:` must point to the previous revision ID exactly. ### 7.5 `db.session.get()` vs `Model.query.get()` - `Model.query.get()` is **deprecated** in SQLAlchemy 2.x. - Always use `db.session.get(Model, pk)` for primary-key lookups. - For 404 handling: `db.session.get(Ticket, ticket_id) or abort(404)` ### 7.6 CSRF on AJAX Requests - All `fetch()` / `XMLHttpRequest` POST requests must include the CSRF token. - Source it from `` in `base.html`. - Header name: `X-CSRFToken` ### 7.7 WebSocket Emit Timing - Never emit a SocketIO event inside a request handler **before** the HTTP→WS transport upgrade completes. - Use `socketio.start_background_task()` to defer emits that should not block the request cycle. - Emitting synchronously before upgrade causes disconnect storms. ### 7.8 `@login_required` on File-Serve Routes - Routes that serve files via `send_from_directory` (e.g. `kb_serve_file`, `serve_avatar`) must have `@login_required`. - **Exception:** Logo serving (`serve_logo`) must be **public** because the login page renders the logo before authentication. - `@login_required` on image-serve routes blocks `` tag requests in some browser/session contexts. Test `` rendering explicitly after adding auth guards to file routes. ### 7.9 Notification Timing - `notify_assignment()`, `notify_status_change()`, and all notification functions must be called **after** `db.session.commit()`. - Calling them before commit means they reference uncommitted state and may send emails/notifications for operations that subsequently fail. ### 7.10 `regexp_replace` on MySQL - MySQL 8.0's `REGEXP_REPLACE(expr, pat, repl)` accepts **exactly 3 arguments**. - The 4th `flags` argument (`'g'` for global) is a PostgreSQL/MariaDB extension. - MySQL applies global replacement by default — the `'g'` flag is unnecessary and causes: `(1583, "Incorrect parameters in the call to native function 'regexp_replace'")` - **Always write:** `func.regexp_replace(Comment.body, r'<[^>]+>', '')` — **never:** `func.regexp_replace(Comment.body, r'<[^>]+>', '', 'g')` ### 7.11 Chatbot Duplicate Ticket Prevention - After a ticket is created via the chatbot, `chatHistory` on the frontend must be purged to prevent re-submission on the next message. - The server strips `"action": "create_ticket"` blocks from history before sending to the Groq API to prevent the model from re-triggering creation. ### 7.12 Ticket Deletion — Physical File Cleanup - SQLAlchemy `cascade='all, delete-orphan'` removes `Attachment` **rows** but does **not** delete files from disk. - Always iterate `ticket.attachments.all()` and `os.remove()` each `attachment.stored_name` from `UPLOAD_FOLDER` before calling `db.session.delete(ticket)`. - Wrap each `os.remove()` in try/except and log warnings for missing files (do not abort the delete if a file is already gone). ### 7.13 Bulk Actions — Delete vs Status Actions - Bulk delete must **early-return** after its own commit so the resolve/close/assign/unassign loop does not run on already-deleted tickets. - Bulk delete is **admin-only** even though the `bulk_ticket_action` route is decorated with `@it_required`. Check `current_user.is_admin` inside the route for the delete branch. --- ## 8. Comment Body Storage - Comments are stored as **sanitized HTML** at write time, not display time. - `render_comment_body(raw_text)` in `validation_service.py` converts Markdown/plain text to HTML and sanitizes via bleach. - When searching comment bodies in SQL, the stored HTML must be stripped before `LIKE` matching: `func.regexp_replace(Comment.body, r'<[^>]+>', '')`. - Migration 003 backfilled all existing plain-text comment bodies to HTML. --- ## 9. Real-time Comments (SocketIO) - Clients join a ticket room on the detail page via `join_ticket` event. - New comments are submitted via AJAX (`fetch` POST) with `X-CSRFToken` header. - The server broadcasts the new comment to the room via SocketIO after commit. - The `get_comments` API endpoint (`/api/tickets//comments`) returns comment HTML for rendering; it calls `_resolve_mime_type` to handle NULL/octet-stream MIME types with extension-based fallback. --- ## 10. File Uploads - All uploads (ticket attachments, comment images, KB files, avatars, logo) land in the same `UPLOAD_FOLDER` directory. - Files are stored with UUID-based `stored_name` values (not the original filename) to prevent path traversal and collisions. - `validate_file()` uses magic-byte inspection (not just extension) for MIME validation. - Allowed extensions: `png, jpg, jpeg, gif, pdf, doc, docx, txt, zip, log` - Max upload size: `MAX_CONTENT_LENGTH` (default 16 MB) - Per-comment attachment limits are enforced both client-side and server-side. --- ## 11. Branding & Theming - App name, subtitle, logo, and primary color are all configurable via `Admin → Settings` and stored in `SystemSetting`. - Branding values are injected into every template via the `inject_globals` context processor in `create_app()`. - The primary color is injected as a CSS variable override so the entire UI recolors without a code change. - Logo is served via `GET /logo/` — this route must remain **public** so the login page can render the logo before authentication. --- ## 12. Migrations Migrations live in `migrations/versions/`. The chain is: ``` (base) └── 001_widen_ticket_history_values — VARCHAR(200) → TEXT for old/new_value └── 002_add_system_settings — creates system_settings table └── 003_render_comments — backfills comment bodies to HTML; widens alembic_version.version_num to VARCHAR(64) └── 004_widen_system_setting_value — system_settings.value VARCHAR(500) → TEXT └── 005_add_watchers_and_time_entries — creates ticket_watchers and time_entries tables ``` **Rules:** - Never modify an existing migration that has been applied to production. - Always set `Revises:` to the previous revision ID exactly. - Revision IDs must be ≤ 64 characters. - Run `flask db upgrade` after deploying new migration files. - `db.create_all()` in `create_app()` will not apply pending migrations — it only creates tables that do not exist at all. --- ## 13. Security Measures | Concern | Implementation | |---|---| | CSRF | Flask-WTF `CSRFProtect`; AJAX uses `X-CSRFToken` header | | Rate limiting | Flask-Limiter on login (10/min, 50/hr), register (5/min, 20/hr), forgot-password (5/hr), chatbot (20/min, 100/hr) | | Password hashing | Werkzeug `generate_password_hash` / `check_password_hash` | | Password reset tokens | SHA-256 hashed, 1-hour TTL, single-use | | File upload validation | Magic-byte MIME inspection + extension allowlist | | HTML sanitization | bleach on all comment bodies and KB article bodies | | XSS in KB | bleach allowlist covers TinyMCE-produced tags; `_sanitize_kb_body()` in `admin.py` | | XSS in JS | `_esc()` helper in `base.html` escapes all server-supplied strings before DOM insertion (notifications, toasts) | | SQL injection | SQLAlchemy ORM parameterized queries throughout | | Real client IP | `ProxyFix` middleware trusts one nginx hop; `log_service._get_real_ip()` reads `X-Forwarded-For` | | Role enforcement | `@admin_required` / `@it_required` decorators; per-route checks where needed | | Avatar/file access | All file-serve routes require `@login_required` (except logo) | | HTTP security headers | `@app.after_request` hook in `create_app()` sets `X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy`, `X-XSS-Protection`, `Permissions-Policy` via `setdefault` (never overrides app-set headers) | | Session cookie security | `ProductionConfig` sets `SESSION_COOKIE_SECURE=True`, `SESSION_COOKIE_HTTPONLY=True`, `SESSION_COOKIE_SAMESITE='Lax'` | --- ## 14. Activity Logging Reference Every create, edit, and delete action must call `log_action()` before the `db.session.commit()` that finalizes the operation. For ticket field changes, also call `log_ticket_history()` for each changed field. Python logger calls (`logger.info(...)`) use structured bracket prefixes for easy `grep`: ``` [TICKET DELETE] [BULK DELETE] [BULK ACTION] [TICKET HISTORY] [ACTIVITY] [ACTIVITY LOG ERROR] [SLA] [EMAIL INGEST] [SEED] [KB IMAGE UPLOAD] [TZ] ``` --- ## 15. Deployment Checklist After any code change: ```bash # 1. Copy changed files to deployment path cp /home/it-ticket/myapp/ # 2. If new migration files added: cd /home/it-ticket/myapp && flask db upgrade # 3. Restart Gunicorn sudo systemctl restart gunicorn # 4. Verify — check logs for errors sudo journalctl -u gunicorn -n 50 tail -f /home/it-ticket/myapp/logs/gunicorn_error.log ``` nginx does **not** need to be restarted for Python code changes (it only proxies; static files are served directly). --- ## 16. Completed Features (Development History) | Feature | Status | |---|---| | Core ticket CRUD (create, view, update, comment) | ✅ | | Role-based access control (employee / IT staff / admin) | ✅ | | Real-time comments via SocketIO | ✅ | | Image upload in comments (clipboard paste + file picker) | ✅ | | User avatar upload with disk cleanup | ✅ | | Admin-controlled user registration toggle | ✅ | | CSV export with active filter support | ✅ | | Markdown rendering with bleach sanitization (stored at write time) | ✅ | | Full branding customization (logo, name, colors) | ✅ | | Knowledge Base with TinyMCE, attachments, feedback | ✅ | | IT Operations Overview dashboard (KPIs, per-staff metrics) | ✅ | | Canned responses for IT staff | ✅ | | Ticket templates for common request types | ✅ | | Ticket linking (related / duplicate / follow-up) | ✅ | | SLA breach detection with APScheduler | ✅ | | Satisfaction surveys on ticket resolution | ✅ | | Email-to-ticket ingestion via IMAP | ✅ | | IT staff notifications (in-app + individual email) on new ticket | ✅ | | Browser tab title shows unread notification count | ✅ | | AI chatbot via Groq (llama-3.3-70b-versatile) | ✅ | | Self-service password reset with token-authenticated email | ✅ | | Admin ticket deletion (single + bulk) | ✅ | | Activity log viewer | ✅ | | Configurable display timezone | ✅ | | HTTP security headers (X-Frame-Options, CSP-adjacent, etc.) | ✅ | | Session cookie security flags (Secure, HttpOnly, SameSite) | ✅ | | XSS protection in JS notification/toast HTML via `_esc()` | ✅ | | Reusable `confirmModal()` system replacing all `confirm()` dialogs | ✅ | | Toast notifications repositioned to top-right (no FAB overlap) | ✅ | | Page navigation progress bar (thin top bar on link clicks) | ✅ | | Dark mode with localStorage persistence and anti-FOCT script | ✅ | | AJAX save on IT Update Panel (no page reload) | ✅ | | Mobile-responsive tables via `table-responsive` wrappers | ✅ | | Employee dashboard: status progress bar + urgency flags per ticket | ✅ | | Linked tickets shown read-only to employees on ticket detail | ✅ | | SLA status badge on ticket detail (On track / Overdue / Completed) | ✅ | | Template preview panel on new ticket form | ✅ | | Bulk assign to specific IT staff member | ✅ | | Weekly IT digest email (APScheduler, Monday 08:00 UTC) | ✅ | | Chatbot KB context injection (top matching articles in system prompt) | ✅ | | Ticket watchers (subscribe/unwatch, in-app notifications on update) | ✅ | | Time tracking (IT staff log minutes per ticket, AJAX form) | ✅ | --- ## 17. Declined / Rejected Features | Feature | Reason | |---|---| | WebRTC / Remote desktop control | Architectural complexity too high; prefer integrating existing tools (e.g. AnyDesk, TeamViewer) over building from scratch | --- ## 22. License Key System (added 2026-05-22) ### Overview TechDesk ships with an offline RSA-signed license key system for commercial distribution as a self-hosted SaaS product. No phone-home is required — the key itself encodes tier and expiry, verified by a baked-in public key. ### Tiers | Tier | Features | |---|---| | Community (no key) | Core ticketing, KB, user management, bulk actions, CSV export | | Business | Everything above + AI Chatbot, SLA tracking, weekly digest, ticket watchers, time tracking, satisfaction surveys | | Enterprise | Everything above + email ingestion (IMAP) | ### Key Format ``` TDESK-. ``` - `payload_b64` = `base64url(UTF-8 JSON bytes)` - `signature_b64` = `base64url(RSA-SHA256 signature of payload bytes)` - JSON payload: `{ customer, email, tier, issued_at, expires_at }` - RSA-2048 with PKCS1v15 padding and SHA-256 ### License Server (`license_server/`) Vendor-hosted internal Flask app. **Never expose to the public internet.** - `generate_keys.py` — run once to produce `private_key.pem` and `public_key.pem` - `app.py` — password-protected UI; `GET /` lists issued keys, `POST /generate` creates a new key - `licenses.db` — SQLite log of all issued keys - Runs on `127.0.0.1:5001`; set `LICENSE_SERVER_PASSWORD` env var **Rule:** `private_key.pem` must NEVER be committed to git or included in any customer build. After generating, copy `public_key.pem` contents into `PUBLIC_KEY_PEM` in `license_service.py`. ### TechDesk License Client (`app/services/license_service.py`) | Function | Description | |---|---| | `validate_license(key_string)` | Verify RSA signature + expiry. Returns status dict. | | `get_status()` | Read cached `app.config['LICENSE_STATUS']`. Requires app context. | | `feature_enabled(feature_name)` | True if current tier grants access. Requires app context. | | `days_until_expiry()` | Days remaining, or None if invalid. | **Status dict keys:** `valid`, `tier`, `customer`, `email`, `issued_at`, `expires_at`, `days_left`, `reason` **Reason codes:** `no_key`, `no_public_key`, `expired`, `invalid_signature`, `invalid_format` **Feature name constants:** `chatbot`, `sla`, `digest`, `watchers`, `time_tracking`, `surveys`, `email_ingestion` ### Integration Points - **`config/config.py`** — `LICENSE_KEY = os.environ.get('LICENSE_KEY', '')` - **`app/__init__.py`** — validates on startup, stores in `app.config['LICENSE_STATUS']`; gates SLA/digest/email-ingestion APScheduler jobs by tier - **`inject_globals` context processor** — injects `license` dict into all templates - **`app/routes/admin.py`** — `GET /admin/license` shows status + feature checklist - **`app/routes/chatbot.py`** — `feature_enabled('chatbot')` gate; returns 403 JSON if unlicensed - **`app/routes/tickets.py`** — `feature_enabled('watchers')` on watch/unwatch; `feature_enabled('time_tracking')` on log-time - **`app/services/sla_service.py`** — `check_sla_breaches` returns early if `feature_enabled('sla')` is False - **`app/services/notification_service.py`** — `send_weekly_digest` returns early if `feature_enabled('digest')` is False - **`app/templates/base.html`** — admin-only warning banner when `license.valid` is False or expiry < 30 days; chatbot FAB hidden for Community tier - **`app/templates/tickets/detail.html`** — watcher card + time tracking card hidden for Community tier ### Behavior Rules - **Soft failure always** — unlicensed app never crashes or hard-blocks; features degrade gracefully - **Warning banner is admin-only** — employees see no license-related UI - **Public key is safe to ship** — it can only verify, not forge signatures - **Startup log** — `[LICENSE]` prefix in app log shows tier, customer, expiry on every start ### Setup Steps for a New Customer Key 1. On your license server: run `python app.py`, log in, fill in customer details, click Generate 2. Copy the full `TDESK-...` key from the UI 3. Send the key to the customer 4. Customer adds `LICENSE_KEY=TDESK-...` to their `.env` and restarts: `sudo systemctl restart gunicorn` 5. Customer visits `Admin → License` to confirm activation --- *Last updated: 2026-05-22* ## 18. Notification Architecture ### `notify_new_ticket` Fan-out (updated 2026-04-17) When a ticket is created — via the web form, the chatbot, the "on behalf" flow, or email ingestion — `notify_new_ticket(ticket)` delivers three tiers of alerts: 1. **Department alias email** — one email to `IT_DEPT_EMAIL` (the shared inbox). 2. **Individual staff emails** — one email per active `it_staff` / `admin` user whose `email_notif == True`. Consistent with `notify_comment_added` and `notify_assignment`. 3. **In-app notifications** — one `Notification` row per active IT staff / admin user, regardless of `email_notif` setting, pushed via WebSocket. 4. **Creator confirmation** — one in-app notification to the ticket submitter. ### Background Job Email Delivery (`send_email`) `send_email()` spawns a background thread. The daemon mode depends on caller context: - **Inside a web request** → `daemon=True` (process won't be held open by the thread after the response is sent). - **Outside a request context** (APScheduler job) → `daemon=False`. Daemon threads are killed when the parent thread exits; since APScheduler job threads finish quickly, a daemon SMTP thread is terminated before delivery completes, causing **silent email loss**. Non-daemon threads survive until SMTP finishes. The same logic applies to `send_satisfaction_survey`'s inline thread spawn. ### SocketIO Emit from Background Jobs `create_notification` emits `new_notification` via SocketIO. The emit strategy branches on `has_request_context()`: - **Inside a web request** → `socketio.start_background_task(_emit)` (non-blocking, avoids racing the polling→WebSocket upgrade handshake). - **Outside a request context** → `socketio.emit()` called directly. Using `start_background_task` from a background job causes Flask-SocketIO to invoke the `on_connect` handler internally, which references `current_user`. With no session attached, Flask-Login returns `None` → `AttributeError`. --- ## 19. Hard Rules (additions 2026-04-17) ### 7.14 `inject_globals` Context Processor and Background Rendering - `render_template_string()` and `render_template()` trigger **all registered context processors**, including `inject_globals` in `app/__init__.py`. - `inject_globals` accesses `current_user.is_authenticated`. Outside a request context (e.g. APScheduler jobs calling `notify_new_ticket`), Flask-Login resolves `current_user` to `None`, causing `AttributeError`. - **Rule:** Always guard `current_user` access in context processors with `has_request_context()` before calling `current_user.is_authenticated`: ```python if has_request_context() and current_user.is_authenticated: ``` - **Corollary:** Any service function that calls `render_template_string()` or `render_template()` from a background context (APScheduler, daemon threads, CLI commands) will trigger this processor. The guard in `inject_globals` is the single correct fix — do not work around it at the call site. ### 7.15 SocketIO `on_connect` / `on_disconnect` Handlers and Sessionless Contexts - The `on_connect` and `on_disconnect` SocketIO event handlers in `api.py` reference `current_user`. When `socketio.emit()` or `start_background_task` is called without a session-backed request context, these handlers may be invoked with `current_user = None`. - **Rule:** All SocketIO event handlers that reference `current_user` must guard with `if current_user and current_user.is_authenticated:` (not just `if current_user.is_authenticated:`). ### 7.16 `SystemSetting.value` Column Width - `SystemSetting.value` was `VARCHAR(500)`. The `email_ingested_message_ids` key accumulates comma-separated Message-IDs; Outlook Message-IDs are ~100 characters each, so 6+ processed messages overflow the column. - **Widened to `TEXT` in migration 004** (`004_widen_system_setting_value`). - **Rule:** Never use `VARCHAR` for any `SystemSetting` value that can accumulate list data or free-form strings of unbounded length. Use `Text`. ### 7.17 Diagnosis Protocol — Root Cause Before Fix When an error is reported: 1. **Read the full traceback**, not just the exception message. Add `exc_info=True` to the catching `logger.error()` call if the traceback is being swallowed. 2. **Trace the complete call chain** from the error site back to the originating caller — do not assume the fix belongs at the first plausible location. 3. **Grep every file** in the call chain for the symbol that is `None` or missing before writing any code. 4. **Write the fix, verify syntax (`ast.parse`), and assert the changed lines are present** before packaging for deployment. 5. **Never apply a patch to a symptom site** (e.g. adding a guard to `api.py`) when the root cause is upstream (e.g. `inject_globals` crashing during `render_template_string`). Symptom-site patches mask errors without resolving them and accumulate technical debt. --- ## 20. Migration History (updated) | Revision ID | Description | |---|---| | `001_widen_ticket_history_values` | Widen `ticket_history.old_value` / `new_value` from `VARCHAR(200)` to `TEXT` | | `002_add_system_settings` | Create `system_settings` table | | `003_render_comments` | Backfill comment bodies to HTML; widen `alembic_version.version_num` to `VARCHAR(64)` | | `004_widen_system_setting_value` | Widen `system_settings.value` from `VARCHAR(500)` to `TEXT` | | `005_add_watchers_and_time_entries` | Create `ticket_watchers` (UniqueConstraint + index) and `time_entries` (index) tables | ## 21. Browser Tab Notification Counter (added 2026-04-17) ### Overview When the user has TechDesk open in a background tab, the browser tab title displays the current unread notification count as a prefix so they can see pending work without switching back. Examples: - `(3) Dashboard — TechDesk` ← 3 unread notifications - `Dashboard — TechDesk` ← no unread notifications ### Implementation (`app/templates/base.html`) **`` tag** — given `id="page-title-text"` so JavaScript can read the server-rendered base title without hardcoding it: ```html <title id="page-title-text">{% block title %}...{% endblock %} — {{ branding.app_name }} ``` **`updateDocumentTitle(unreadCount)`** — new JS function that sets `document.title` to `"(n) "` when `n > 0`, or restores the original title when `n === 0`. **`_baseDocTitle`** — captured once at page load from the `` element so page-specific titles (e.g. "Ticket Detail", "Dashboard") are preserved exactly. **`updateBadge()`** — extended to call `updateDocumentTitle(next)` on every badge change, covering all update paths: - Real-time WebSocket `new_notification` event - Opening the notification panel (`loadNotifications` recalculates count) - Clicking an individual notification to mark it read - "Mark all read" button **IIFE on page load** — reads the server-rendered badge count and calls `updateDocumentTitle()` immediately, so the title is correct even before any WebSocket event fires. ### Key Design Decisions - No new routes, API calls, or Python changes — purely a frontend enhancement. - `_baseDocTitle` is captured from the DOM, not hardcoded, so it works correctly for every page that extends `base.html` without any per-template changes. - The count in the title stays in sync with the bell badge — a single `updateBadge()` call updates both, so they can never diverge.