IT_Ticket_System/Claude.md

# TechDesk — IT Helpdesk System: Knowledge Base for Claude

This document is the authoritative reference for Claude when working on the
TechDesk codebase. It captures the full architecture, design decisions, known
pitfalls, and hard-won lessons from the development history of this application.

---

## 1. Project Overview

**TechDesk** is an internal IT helpdesk web application deployed at
`tickets.ltservicesinc.com`. It is built for LT Services Inc. and serves two
audiences:

- **Employees** — submit and track IT support tickets
- **IT Staff / Admins** — manage, assign, resolve, and report on tickets

### Technology Stack

| Layer | Technology |
|---|---|
| Language | Python 3 |
| Web Framework | Flask |
| ORM | SQLAlchemy (via Flask-SQLAlchemy) |
| Database | MySQL (driver: PyMySQL) |
| Migrations | Alembic (via Flask-Migrate) |
| Templates | Jinja2 |
| Real-time | Flask-SocketIO (eventlet async mode) |
| WSGI Server | Gunicorn (single worker, eventlet worker class) |
| Reverse Proxy | nginx (port 7000 upstream, SSL via Certbot) |
| Auth | Flask-Login |
| Forms / CSRF | Flask-WTF |
| Rate Limiting | Flask-Limiter |
| Email | Flask-Mail |
| Background Jobs | APScheduler (BackgroundScheduler, daemon=True) |
| Rich Text Editor | TinyMCE 6 |
| AI Chatbot | Groq API (`llama-3.3-70b-versatile`) |
| HTML Sanitization | bleach |
| Static Files | Served directly by nginx from `/home/it-ticket/myapp/app/static` |

### Deployment Path

```
/home/it-ticket/myapp/
├── app/
│   ├── __init__.py          # App factory (create_app)
│   ├── models.py            # All SQLAlchemy models
│   ├── routes/
│   │   ├── admin.py         # /admin/* — IT staff and admin routes
│   │   ├── api.py           # /api/* — JSON endpoints + SocketIO events
│   │   ├── auth.py          # /auth/* — login, register, password reset
│   │   ├── chatbot.py       # /chatbot/* — AI chatbot endpoint
│   │   └── tickets.py       # / and /tickets/* — employee-facing routes
│   ├── services/
│   │   ├── email_ingestion_service.py
│   │   ├── log_service.py
│   │   ├── notification_service.py
│   │   ├── sla_service.py
│   │   └── validation_service.py
│   ├── templates/
│   │   ├── base.html
│   │   ├── admin/           # Admin panel templates
│   │   ├── auth/            # Login, register, password reset templates
│   │   └── tickets/         # Employee-facing ticket templates
│   └── static/
│       └── uploads/         # User-uploaded files (default; overridable via UPLOAD_FOLDER)
├── config/
│   └── config.py            # DevelopmentConfig / ProductionConfig
├── migrations/
│   └── versions/            # Alembic migration scripts
├── gunicorn.conf.py
├── run.py                   # Entry point — calls eventlet.monkey_patch() first
├── .env                     # Secrets (not committed)
└── license_server/          # Vendor-only — NOT shipped to customers
    ├── app.py               # Internal Flask UI for generating license keys
    ├── generate_keys.py     # One-time RSA key pair generator
    ├── requirements.txt     # flask + cryptography
    ├── private_key.pem      # RSA private key — NEVER commit or ship
    └── licenses.db          # SQLite log of issued keys
```

---

## 2. User Roles & Access Control

Three roles are defined in `UserRole`:

| Role | Constant | Access |
|---|---|---|
| Employee | `employee` | Submit tickets, view own tickets, knowledge base, chatbot |
| IT Staff | `it_staff` | Everything above + manage all tickets, assignment, bulk actions |
| Admin | `admin` | Everything above + user management, system settings, ticket deletion, KB management |

### Decorators (defined in `admin.py`)

```python
@admin_required   # role == 'admin' only
@it_required      # role in ('it_staff', 'admin')
```

Both decorators must be applied **after** `@login_required`.

---

## 3. Database Models (`app/models.py`)

### Core Models

| Model | Table | Key Notes |
|---|---|---|
| `User` | `users` | Roles: employee / it_staff / admin. `is_admin` and `is_it_staff` are properties. |
| `Ticket` | `tickets` | Central entity. Has `generate_ticket_number()` with `FOR UPDATE` row locking to prevent race conditions. |
| `Comment` | `comments` | Bodies stored as sanitized HTML (rendered at write time via `render_comment_body`). |
| `Attachment` | `attachments` | Tied to both `ticket_id` and `comment_id`. Physical files live in `UPLOAD_FOLDER`. |
| `Notification` | `notifications` | Per-user; read/unread. Cascades delete with ticket. |
| `TicketHistory` | `ticket_history` | Field-level audit trail. `old_value`/`new_value` are TEXT (widened in migration 001). |
| `ActivityLog` | `activity_logs` | System-wide action log. `user_id` has `ondelete='SET NULL'`. |
| `KnowledgeBase` | `knowledge_base` | Articles with TinyMCE body, tags, view count, publish toggle. |
| `KBAttachment` | `kb_attachments` | Files attached to KB articles. Shared `UPLOAD_FOLDER` with ticket attachments. |
| `SystemSetting` | `system_settings` | Key-value config store. Use `SystemSetting.get(key)` / `SystemSetting.set(key, value)`. |
| `TicketLink` | `ticket_links` | Symmetric bidirectional links between tickets (related / duplicate / follow_up). |
| `CannedResponse` | `canned_responses` | Pre-written reply templates for IT staff. |
| `KBFeedback` | `kb_feedback` | One thumbs up/down vote per user per article. |
| `PasswordResetToken` | `password_reset_tokens` | SHA-256 hashed, 1-hour expiry, single-use. |
| `TicketTemplate` | `ticket_templates` | Pre-filled ticket scaffolds selectable on the new ticket form. |
| `TicketSatisfaction` | `ticket_satisfaction` | One survey per resolved ticket; token-authenticated survey URL. |
| `TicketWatcher` | `ticket_watchers` | Users subscribed to updates on a ticket. Unique `(ticket_id, user_id)`. Added migration 005. |
| `TimeEntry` | `time_entries` | IT staff time log per ticket. Stores `minutes` + optional `note`. Added migration 005. |

### Cascade Rules

`Ticket` has `cascade='all, delete-orphan'` on:
- `comments`
- `attachments`
- `notifications`
- `history`
- `links_as_source` / `links_as_target` (via `TicketLink`)
- `satisfaction` (via `TicketSatisfaction`)
- `watchers` (via `TicketWatcher`)
- `time_entries` (via `TimeEntry`)

**When deleting a ticket, physical files in `UPLOAD_FOLDER` must be removed
manually before the DB delete** — SQLAlchemy cascades handle DB rows only.

---

## 4. Blueprint & Route Structure

| Blueprint | Prefix | File | Guard |
|---|---|---|---|
| `auth_bp` | (none) | `routes/auth.py` | Public + `@login_required` per route |
| `tickets_bp` | (none) | `routes/tickets.py` | `@login_required` per route |
| `admin_bp` | `/admin` | `routes/admin.py` | `@login_required` + `@it_required` or `@admin_required` |
| `api_bp` | `/api` | `routes/api.py` | `@login_required` per route |
| `chatbot_bp` | `/chatbot` | `routes/chatbot.py` | `@login_required` per route |

### Key Routes Reference

**Auth (`auth.py`)**
- `GET/POST /login` — rate limited: 10/min, 50/hr
- `GET/POST /register` — rate limited: 5/min, 20/hr; guarded by `registration_enabled` SystemSetting
- `GET /logout`
- `GET/POST /profile`
- `GET/POST /forgot-password` — rate limited: 5/hr
- `GET/POST /reset-password/<token>` — rate limited: 10/hr
- `GET /avatar/<filename>` — login required (prevents unauthenticated access)
- `GET /logo/<filename>` — public (used in login page)

**Tickets (`tickets.py`)**
- `GET /` and `GET /dashboard` — role-aware: employees see `dashboard_employee.html`, IT staff see `dashboard_it.html`
- `GET/POST /tickets/new`
- `GET/POST /tickets/behalf` — IT staff only (file ticket on behalf of employee)
- `GET /tickets` — employee ticket list (own tickets only)
- `GET/POST /tickets/<id>` — ticket detail + comment submission
- `POST /tickets/<id>/update` — IT staff update (status, priority, assignment, etc.)
- `POST /comments/<id>/delete`
- `GET /attachments/<id>` — file download
- `GET /notifications` / `POST /notifications/mark-read`
- `GET /kb` / `GET /kb/<id>` — knowledge base
- `POST /tickets/<id>/link` / `POST /tickets/<id>/unlink/<link_id>`
- `POST /tickets/<id>/reopen`
- `POST /tickets/<id>/watch` / `POST /tickets/<id>/unwatch` — AJAX; toggle watcher subscription
- `POST /tickets/<id>/log-time` — IT staff only; AJAX-aware; logs `TimeEntry`
- `GET /canned-responses` — JSON endpoint for IT staff comment box
- `POST /kb/<id>/feedback`
- `GET/POST /survey/<token>` — public (no login required)

**Admin (`admin.py`)**
- `GET /admin/` — IT Operations Overview dashboard
- `GET /admin/users` / `POST /admin/users/new` / `POST /admin/users/<id>/edit` / `POST /admin/users/<id>/delete`
- `GET /admin/tickets` — all tickets with search, filter, pagination
- `POST /admin/tickets/<id>/delete` — **admin only**; deletes ticket + physical files
- `POST /admin/tickets/bulk-action` — actions: resolve, close, assign_me, assign_to, unassign, **delete** (delete is admin-only; `assign_to` requires `assign_to_id` form field)
- `GET /admin/tickets/export` — CSV export with active filters
- `GET /admin/kb` / `GET/POST /admin/kb/new` / `GET/POST /admin/kb/<id>/edit` / `POST /admin/kb/<id>/delete` / `POST /admin/kb/<id>/publish`
- `POST /admin/kb/upload-image` — TinyMCE image upload
- `GET /admin/kb/files/<stored_name>` — file serve (login required)
- `GET/POST /admin/logs` — activity log viewer
- `GET/POST /admin/canned-responses/*`
- `POST /admin/tickets/bulk-action`
- `GET/POST /admin/ticket-templates/*`
- `GET /admin/satisfaction` — satisfaction report
- `GET/POST /admin/settings`

**API (`api.py`)**
- `GET /api/notifications` — JSON list
- `GET /api/notifications/unread-count`
- `POST /api/notifications/<id>/read`
- `POST /api/notifications/mark-all-read`
- `GET /api/tickets/<id>/comments` — JSON (used by real-time comment rendering)
- `GET /api/stats/tickets` — dashboard KPI stats
- `GET /api/users/search` — typeahead for ticket assignment
- `GET /api/tickets/search` — typeahead for ticket linking
- SocketIO events: `connect`, `disconnect`, `join_ticket`, `leave_ticket`

**Chatbot (`chatbot.py`)**
- `POST /chatbot/message` — rate limited: 20/min, 100/hr; calls Groq API

---

## 5. Services

### `log_service.py`

```python
log_action(user_id, action, entity_type=None, entity_id=None, details=None)
log_ticket_history(ticket, field_name, old_value, new_value, changed_by_id)
```

**Critical rules:**
- Neither function calls `db.session.commit()`. The caller must commit.
- On failure, only the failed log entry is `expunge()`d — **never call `db.session.rollback()`** inside these functions, as that would undo the parent operation.
- Always call `log_action` **before** `db.session.commit()`.
- `log_ticket_history` records granular field-level changes on tickets.

**Standard action strings used across the codebase:**

| Action String | Where |
|---|---|
| `ticket_create` | New ticket submitted |
| `ticket_update` | Ticket fields changed |
| `comment_create` | Comment added |
| `comment_delete` | Comment deleted |
| `admin_user_create` | Admin creates a user |
| `admin_user_edit` | Admin edits a user |
| `admin_user_deactivate` | Admin deactivates a user |
| `admin_ticket_delete` | Admin deletes a ticket |
| `ticket_bulk_resolve` | Bulk resolve |
| `ticket_bulk_close` | Bulk close |
| `ticket_bulk_assign_me` | Bulk assign to self |
| `ticket_bulk_assign_to` | Bulk assign to specific user |
| `ticket_bulk_unassign` | Bulk unassign |
| `kb_create` / `kb_edit` / `kb_delete` | KB article CRUD |
| `kb_attachment_delete` | KB attachment removed |
| `time_log` | IT staff logs time on a ticket |
| `ticket_create_chatbot` | Ticket created via AI chatbot |

### `notification_service.py`

Functions: `notify_new_ticket`, `notify_status_change`, `notify_comment_added`,
`notify_assignment`, `send_satisfaction_survey`, `notify_watchers`,
`send_weekly_digest`

**Critical rule:** Always call notification functions **after** `db.session.commit()`.
Calling them before commit means they act on uncommitted state.

- **`notify_watchers(ticket, event_title, event_message, exclude_user_id=None)`** — sends in-app notifications to all `TicketWatcher` rows for a ticket. Call after commit from `update_ticket`. Pass `exclude_user_id=current_user.id` so the actor doesn't notify themselves.
- **`send_weekly_digest(app)`** — APScheduler job; runs every Monday at 08:00 UTC via `CronTrigger`. Sends a rich HTML email to all active IT staff / admin users with `email_notif=True`. Summarises new, resolved, still-open, and overdue tickets for the past 7 days.

### `sla_service.py`

- `set_due_date(ticket, app)` — called at ticket creation; reads SLA hours from `SystemSetting`
- `check_sla_breaches(app)` — runs every 30 minutes via APScheduler; creates notifications for overdue tickets
- `clear_sla_notification(ticket_id)` — called when a ticket is resolved or closed

### `validation_service.py`

- `validate_password(password, confirm)` — returns error string or None
- `validate_file(file, allowed_extensions)` — magic-byte MIME validation (not just extension)
- `render_comment_body(raw_text)` — converts Markdown/plain text to sanitized HTML via bleach; called at write time, not display time

### `email_ingestion_service.py`

- `check_inbound_email(app)` — APScheduler job; polls IMAP mailbox and converts emails to tickets
- Deduplication via `Message-ID` headers stored in `SystemSetting`
- Unknown senders fall back to the system admin user account

---

## 6. Configuration & Environment

### `.env` Keys

```
SECRET_KEY=
DB_USER=
DB_PASSWORD=
DB_HOST=
DB_PORT=3306
DB_NAME=it_tickets
MAIL_SERVER=
MAIL_PORT=587
MAIL_USE_TLS=True
MAIL_USERNAME=
MAIL_PASSWORD=
MAIL_DEFAULT_SENDER=
IT_DEPT_EMAIL=
APP_BASE_URL=https://tickets.ltservicesinc.com
UPLOAD_FOLDER=          # Absolute path; defaults to app/static/uploads
GROQ_API_KEY=
GROQ_MODEL=llama-3.3-70b-versatile
ADMIN_EMAIL=
ADMIN_PASSWORD=
RATELIMIT_STORAGE_URI=memory://
```

### `SystemSetting` Keys (runtime config, editable via Admin → Settings)

| Key | Default | Description |
|---|---|---|
| `registration_enabled` | `true` | Allow self-registration |
| `survey_enabled` | `true` | Send satisfaction survey on resolve |
| `app_timezone` | `America/New_York` | Display timezone for all dates |
| `sla_critical_hours` | `4` | SLA breach threshold for CRITICAL |
| `sla_high_hours` | `8` | SLA breach threshold for HIGH |
| `sla_medium_hours` | `48` | SLA breach threshold for MEDIUM |
| `sla_low_hours` | `120` | SLA breach threshold for LOW |
| `app_name` | `TechDesk` | Sidebar/page title branding |
| `app_subtitle` | `IT Helpdesk System` | Sidebar subtitle |
| `company_name` | `` | Login/register page branding |
| `logo_stored_name` | `` | Uploaded logo filename |
| `logo_initials` | `TD` | Fallback initials when no logo |
| `primary_color` | `#2563eb` | CSS accent color (hex) |
| `email_ingestion_enabled` | `0` | Enable IMAP email-to-ticket |
| `email_ingestion_host/port/user/password/folder/move_to/interval` | various | IMAP settings |

---

## 7. Known Pitfalls & Hard Rules

These are non-negotiable lessons learned from production incidents and debugging
sessions. Violating any of these will introduce bugs.

### 7.1 eventlet / Gunicorn Ordering

- `eventlet.monkey_patch()` **must** run before the app is loaded.
- It is called in `run.py` at module level before `create_app()`.
- `preload_app = False` in `gunicorn.conf.py` is **mandatory**. Setting it to
  `True` causes the app to load in the master process before `monkey_patch()`
  runs, leaving real OS RLocks un-greened and triggering:
  `"1 RLock(s) were not greened"`

### 7.2 Flask-Limiter Storage

- Flask-Limiter does **not** support `mysql+pymysql://` as a storage URI.
- Always use `memory://` (or `redis://` if Redis becomes available).
- The `RATELIMIT_STORAGE_URI` config key controls this.

### 7.3 SQLAlchemy Session & Generators

- Generators that access SQLAlchemy ORM objects execute **lazily**.
- If a generator is returned from a route and consumed outside the request
  context (after the session closes), it silently yields nothing.
- Always collect ORM results into a list **within** the session context before
  yielding or returning from a route. This was the cause of empty CSV exports.

### 7.4 `db.create_all()` and Alembic

- `db.create_all()` is called in `create_app()` for first-run convenience.
- If tables already exist (normal in production), `create_all()` is a no-op.
- **Do not use `db.create_all()` as a substitute for migrations.** New columns
  added in migrations will not be created by `create_all()` on existing tables.
- Alembic revision IDs must fit within `VARCHAR(64)` in `alembic_version`
  (widened in migration 003). Do not use revision IDs longer than 64 characters.
- Never break the migration chain: each migration's `Revises:` must point to
  the previous revision ID exactly.

### 7.5 `db.session.get()` vs `Model.query.get()`

- `Model.query.get()` is **deprecated** in SQLAlchemy 2.x.
- Always use `db.session.get(Model, pk)` for primary-key lookups.
- For 404 handling: `db.session.get(Ticket, ticket_id) or abort(404)`

### 7.6 CSRF on AJAX Requests

- All `fetch()` / `XMLHttpRequest` POST requests must include the CSRF token.
- Source it from `<meta name="csrf-token" content="{{ csrf_token() }}">` in
  `base.html`.
- Header name: `X-CSRFToken`

### 7.7 WebSocket Emit Timing

- Never emit a SocketIO event inside a request handler **before** the HTTP→WS
  transport upgrade completes.
- Use `socketio.start_background_task()` to defer emits that should not block
  the request cycle.
- Emitting synchronously before upgrade causes disconnect storms.

### 7.8 `@login_required` on File-Serve Routes

- Routes that serve files via `send_from_directory` (e.g. `kb_serve_file`,
  `serve_avatar`) must have `@login_required`.
- **Exception:** Logo serving (`serve_logo`) must be **public** because the
  login page renders the logo before authentication.
- `@login_required` on image-serve routes blocks `<img>` tag requests in some
  browser/session contexts. Test `<img src="...">` rendering explicitly after
  adding auth guards to file routes.

### 7.9 Notification Timing

- `notify_assignment()`, `notify_status_change()`, and all notification
  functions must be called **after** `db.session.commit()`.
- Calling them before commit means they reference uncommitted state and may
  send emails/notifications for operations that subsequently fail.

### 7.10 `regexp_replace` on MySQL

- MySQL 8.0's `REGEXP_REPLACE(expr, pat, repl)` accepts **exactly 3 arguments**.
- The 4th `flags` argument (`'g'` for global) is a PostgreSQL/MariaDB extension.
- MySQL applies global replacement by default — the `'g'` flag is unnecessary
  and causes: `(1583, "Incorrect parameters in the call to native function 'regexp_replace'")`
- **Always write:** `func.regexp_replace(Comment.body, r'<[^>]+>', '')`
  — **never:** `func.regexp_replace(Comment.body, r'<[^>]+>', '', 'g')`

### 7.11 Chatbot Duplicate Ticket Prevention

- After a ticket is created via the chatbot, `chatHistory` on the frontend
  must be purged to prevent re-submission on the next message.
- The server strips `"action": "create_ticket"` blocks from history before
  sending to the Groq API to prevent the model from re-triggering creation.

### 7.12 Ticket Deletion — Physical File Cleanup

- SQLAlchemy `cascade='all, delete-orphan'` removes `Attachment` **rows** but
  does **not** delete files from disk.
- Always iterate `ticket.attachments.all()` and `os.remove()` each
  `attachment.stored_name` from `UPLOAD_FOLDER` before calling `db.session.delete(ticket)`.
- Wrap each `os.remove()` in try/except and log warnings for missing files
  (do not abort the delete if a file is already gone).

### 7.13 Bulk Actions — Delete vs Status Actions

- Bulk delete must **early-return** after its own commit so the
  resolve/close/assign/unassign loop does not run on already-deleted tickets.
- Bulk delete is **admin-only** even though the `bulk_ticket_action` route is
  decorated with `@it_required`. Check `current_user.is_admin` inside the
  route for the delete branch.

---

## 8. Comment Body Storage

- Comments are stored as **sanitized HTML** at write time, not display time.
- `render_comment_body(raw_text)` in `validation_service.py` converts
  Markdown/plain text to HTML and sanitizes via bleach.
- When searching comment bodies in SQL, the stored HTML must be stripped before
  `LIKE` matching: `func.regexp_replace(Comment.body, r'<[^>]+>', '')`.
- Migration 003 backfilled all existing plain-text comment bodies to HTML.

---

## 9. Real-time Comments (SocketIO)

- Clients join a ticket room on the detail page via `join_ticket` event.
- New comments are submitted via AJAX (`fetch` POST) with `X-CSRFToken` header.
- The server broadcasts the new comment to the room via SocketIO after commit.
- The `get_comments` API endpoint (`/api/tickets/<id>/comments`) returns
  comment HTML for rendering; it calls `_resolve_mime_type` to handle
  NULL/octet-stream MIME types with extension-based fallback.

---

## 10. File Uploads

- All uploads (ticket attachments, comment images, KB files, avatars, logo)
  land in the same `UPLOAD_FOLDER` directory.
- Files are stored with UUID-based `stored_name` values (not the original
  filename) to prevent path traversal and collisions.
- `validate_file()` uses magic-byte inspection (not just extension) for MIME
  validation.
- Allowed extensions: `png, jpg, jpeg, gif, pdf, doc, docx, txt, zip, log`
- Max upload size: `MAX_CONTENT_LENGTH` (default 16 MB)
- Per-comment attachment limits are enforced both client-side and server-side.

---

## 11. Branding & Theming

- App name, subtitle, logo, and primary color are all configurable via
  `Admin → Settings` and stored in `SystemSetting`.
- Branding values are injected into every template via the `inject_globals`
  context processor in `create_app()`.
- The primary color is injected as a CSS variable override so the entire UI
  recolors without a code change.
- Logo is served via `GET /logo/<filename>` — this route must remain **public**
  so the login page can render the logo before authentication.

---

## 12. Migrations

Migrations live in `migrations/versions/`. The chain is:

```
(base)
  └── 001_widen_ticket_history_values   — VARCHAR(200) → TEXT for old/new_value
        └── 002_add_system_settings     — creates system_settings table
              └── 003_render_comments   — backfills comment bodies to HTML;
                                          widens alembic_version.version_num to VARCHAR(64)
                    └── 004_widen_system_setting_value — system_settings.value VARCHAR(500) → TEXT
                          └── 005_add_watchers_and_time_entries — creates ticket_watchers
                                                                   and time_entries tables
```

**Rules:**
- Never modify an existing migration that has been applied to production.
- Always set `Revises:` to the previous revision ID exactly.
- Revision IDs must be ≤ 64 characters.
- Run `flask db upgrade` after deploying new migration files.
- `db.create_all()` in `create_app()` will not apply pending migrations —
  it only creates tables that do not exist at all.

---

## 13. Security Measures

| Concern | Implementation |
|---|---|
| CSRF | Flask-WTF `CSRFProtect`; AJAX uses `X-CSRFToken` header |
| Rate limiting | Flask-Limiter on login (10/min, 50/hr), register (5/min, 20/hr), forgot-password (5/hr), chatbot (20/min, 100/hr) |
| Password hashing | Werkzeug `generate_password_hash` / `check_password_hash` |
| Password reset tokens | SHA-256 hashed, 1-hour TTL, single-use |
| File upload validation | Magic-byte MIME inspection + extension allowlist |
| HTML sanitization | bleach on all comment bodies and KB article bodies |
| XSS in KB | bleach allowlist covers TinyMCE-produced tags; `_sanitize_kb_body()` in `admin.py` |
| XSS in JS | `_esc()` helper in `base.html` escapes all server-supplied strings before DOM insertion (notifications, toasts) |
| SQL injection | SQLAlchemy ORM parameterized queries throughout |
| Real client IP | `ProxyFix` middleware trusts one nginx hop; `log_service._get_real_ip()` reads `X-Forwarded-For` |
| Role enforcement | `@admin_required` / `@it_required` decorators; per-route checks where needed |
| Avatar/file access | All file-serve routes require `@login_required` (except logo) |
| HTTP security headers | `@app.after_request` hook in `create_app()` sets `X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy`, `X-XSS-Protection`, `Permissions-Policy` via `setdefault` (never overrides app-set headers) |
| Session cookie security | `ProductionConfig` sets `SESSION_COOKIE_SECURE=True`, `SESSION_COOKIE_HTTPONLY=True`, `SESSION_COOKIE_SAMESITE='Lax'` |

---

## 14. Activity Logging Reference

Every create, edit, and delete action must call `log_action()` before the
`db.session.commit()` that finalizes the operation.

For ticket field changes, also call `log_ticket_history()` for each changed
field.

Python logger calls (`logger.info(...)`) use structured bracket prefixes for
easy `grep`:

```
[TICKET DELETE]
[BULK DELETE]
[BULK ACTION]
[TICKET HISTORY]
[ACTIVITY]
[ACTIVITY LOG ERROR]
[SLA]
[EMAIL INGEST]
[SEED]
[KB IMAGE UPLOAD]
[TZ]
```

---

## 15. Deployment Checklist

After any code change:

```bash
# 1. Copy changed files to deployment path
cp <file> /home/it-ticket/myapp/<path>

# 2. If new migration files added:
cd /home/it-ticket/myapp && flask db upgrade

# 3. Restart Gunicorn
sudo systemctl restart gunicorn

# 4. Verify — check logs for errors
sudo journalctl -u gunicorn -n 50
tail -f /home/it-ticket/myapp/logs/gunicorn_error.log
```

nginx does **not** need to be restarted for Python code changes (it only
proxies; static files are served directly).

---

## 16. Completed Features (Development History)

| Feature | Status |
|---|---|
| Core ticket CRUD (create, view, update, comment) | ✅ |
| Role-based access control (employee / IT staff / admin) | ✅ |
| Real-time comments via SocketIO | ✅ |
| Image upload in comments (clipboard paste + file picker) | ✅ |
| User avatar upload with disk cleanup | ✅ |
| Admin-controlled user registration toggle | ✅ |
| CSV export with active filter support | ✅ |
| Markdown rendering with bleach sanitization (stored at write time) | ✅ |
| Full branding customization (logo, name, colors) | ✅ |
| Knowledge Base with TinyMCE, attachments, feedback | ✅ |
| IT Operations Overview dashboard (KPIs, per-staff metrics) | ✅ |
| Canned responses for IT staff | ✅ |
| Ticket templates for common request types | ✅ |
| Ticket linking (related / duplicate / follow-up) | ✅ |
| SLA breach detection with APScheduler | ✅ |
| Satisfaction surveys on ticket resolution | ✅ |
| Email-to-ticket ingestion via IMAP | ✅ |
| IT staff notifications (in-app + individual email) on new ticket | ✅ |
| Browser tab title shows unread notification count | ✅ |
| AI chatbot via Groq (llama-3.3-70b-versatile) | ✅ |
| Self-service password reset with token-authenticated email | ✅ |
| Admin ticket deletion (single + bulk) | ✅ |
| Activity log viewer | ✅ |
| Configurable display timezone | ✅ |
| HTTP security headers (X-Frame-Options, CSP-adjacent, etc.) | ✅ |
| Session cookie security flags (Secure, HttpOnly, SameSite) | ✅ |
| XSS protection in JS notification/toast HTML via `_esc()` | ✅ |
| Reusable `confirmModal()` system replacing all `confirm()` dialogs | ✅ |
| Toast notifications repositioned to top-right (no FAB overlap) | ✅ |
| Page navigation progress bar (thin top bar on link clicks) | ✅ |
| Dark mode with localStorage persistence and anti-FOCT script | ✅ |
| AJAX save on IT Update Panel (no page reload) | ✅ |
| Mobile-responsive tables via `table-responsive` wrappers | ✅ |
| Employee dashboard: status progress bar + urgency flags per ticket | ✅ |
| Linked tickets shown read-only to employees on ticket detail | ✅ |
| SLA status badge on ticket detail (On track / Overdue / Completed) | ✅ |
| Template preview panel on new ticket form | ✅ |
| Bulk assign to specific IT staff member | ✅ |
| Weekly IT digest email (APScheduler, Monday 08:00 UTC) | ✅ |
| Chatbot KB context injection (top matching articles in system prompt) | ✅ |
| Ticket watchers (subscribe/unwatch, in-app notifications on update) | ✅ |
| Time tracking (IT staff log minutes per ticket, AJAX form) | ✅ |

---

## 17. Declined / Rejected Features

| Feature | Reason |
|---|---|
| WebRTC / Remote desktop control | Architectural complexity too high; prefer integrating existing tools (e.g. AnyDesk, TeamViewer) over building from scratch |

---

## 22. License Key System (added 2026-05-22)

### Overview

TechDesk ships with an offline RSA-signed license key system for commercial
distribution as a self-hosted SaaS product. No phone-home is required — the
key itself encodes tier and expiry, verified by a baked-in public key.

### Tiers

| Tier | Features |
|---|---|
| Community (no key) | Core ticketing, KB, user management, bulk actions, CSV export |
| Business | Everything above + AI Chatbot, SLA tracking, weekly digest, ticket watchers, time tracking, satisfaction surveys |
| Enterprise | Everything above + email ingestion (IMAP) |

### Key Format

```
TDESK-<payload_b64>.<signature_b64>
```

- `payload_b64` = `base64url(UTF-8 JSON bytes)`
- `signature_b64` = `base64url(RSA-SHA256 signature of payload bytes)`
- JSON payload: `{ customer, email, tier, issued_at, expires_at }`
- RSA-2048 with PKCS1v15 padding and SHA-256

### License Server (`license_server/`)

Vendor-hosted internal Flask app. **Never expose to the public internet.**

- `generate_keys.py` — run once to produce `private_key.pem` and `public_key.pem`
- `app.py` — password-protected UI; `GET /` lists issued keys, `POST /generate` creates a new key
- `licenses.db` — SQLite log of all issued keys
- Runs on `127.0.0.1:5001`; set `LICENSE_SERVER_PASSWORD` env var

**Rule:** `private_key.pem` must NEVER be committed to git or included in any customer build.
After generating, copy `public_key.pem` contents into `PUBLIC_KEY_PEM` in `license_service.py`.

### TechDesk License Client (`app/services/license_service.py`)

| Function | Description |
|---|---|
| `validate_license(key_string)` | Verify RSA signature + expiry. Returns status dict. |
| `get_status()` | Read cached `app.config['LICENSE_STATUS']`. Requires app context. |
| `feature_enabled(feature_name)` | True if current tier grants access. Requires app context. |
| `days_until_expiry()` | Days remaining, or None if invalid. |

**Status dict keys:** `valid`, `tier`, `customer`, `email`, `issued_at`, `expires_at`, `days_left`, `reason`

**Reason codes:** `no_key`, `no_public_key`, `expired`, `invalid_signature`, `invalid_format`

**Feature name constants:** `chatbot`, `sla`, `digest`, `watchers`, `time_tracking`, `surveys`, `email_ingestion`

### Integration Points

- **`config/config.py`** — `LICENSE_KEY = os.environ.get('LICENSE_KEY', '')`
- **`app/__init__.py`** — validates on startup, stores in `app.config['LICENSE_STATUS']`; gates SLA/digest/email-ingestion APScheduler jobs by tier
- **`inject_globals` context processor** — injects `license` dict into all templates
- **`app/routes/admin.py`** — `GET /admin/license` shows status + feature checklist
- **`app/routes/chatbot.py`** — `feature_enabled('chatbot')` gate; returns 403 JSON if unlicensed
- **`app/routes/tickets.py`** — `feature_enabled('watchers')` on watch/unwatch; `feature_enabled('time_tracking')` on log-time
- **`app/services/sla_service.py`** — `check_sla_breaches` returns early if `feature_enabled('sla')` is False
- **`app/services/notification_service.py`** — `send_weekly_digest` returns early if `feature_enabled('digest')` is False
- **`app/templates/base.html`** — admin-only warning banner when `license.valid` is False or expiry < 30 days; chatbot FAB hidden for Community tier
- **`app/templates/tickets/detail.html`** — watcher card + time tracking card hidden for Community tier

### Behavior Rules

- **Soft failure always** — unlicensed app never crashes or hard-blocks; features degrade gracefully
- **Warning banner is admin-only** — employees see no license-related UI
- **Public key is safe to ship** — it can only verify, not forge signatures
- **Startup log** — `[LICENSE]` prefix in app log shows tier, customer, expiry on every start

### Setup Steps for a New Customer Key

1. On your license server: run `python app.py`, log in, fill in customer details, click Generate
2. Copy the full `TDESK-...` key from the UI
3. Send the key to the customer
4. Customer adds `LICENSE_KEY=TDESK-...` to their `.env` and restarts: `sudo systemctl restart gunicorn`
5. Customer visits `Admin → License` to confirm activation

---

*Last updated: 2026-05-22*

## 18. Notification Architecture

### `notify_new_ticket` Fan-out (updated 2026-04-17)

When a ticket is created — via the web form, the chatbot, the "on behalf" flow,
or email ingestion — `notify_new_ticket(ticket)` delivers three tiers of alerts:

1. **Department alias email** — one email to `IT_DEPT_EMAIL` (the shared inbox).
2. **Individual staff emails** — one email per active `it_staff` / `admin` user
   whose `email_notif == True`. Consistent with `notify_comment_added` and
   `notify_assignment`.
3. **In-app notifications** — one `Notification` row per active IT staff / admin
   user, regardless of `email_notif` setting, pushed via WebSocket.
4. **Creator confirmation** — one in-app notification to the ticket submitter.

### Background Job Email Delivery (`send_email`)

`send_email()` spawns a background thread. The daemon mode depends on caller
context:

- **Inside a web request** → `daemon=True` (process won't be held open by the
  thread after the response is sent).
- **Outside a request context** (APScheduler job) → `daemon=False`. Daemon
  threads are killed when the parent thread exits; since APScheduler job threads
  finish quickly, a daemon SMTP thread is terminated before delivery completes,
  causing **silent email loss**. Non-daemon threads survive until SMTP finishes.

The same logic applies to `send_satisfaction_survey`'s inline thread spawn.

### SocketIO Emit from Background Jobs

`create_notification` emits `new_notification` via SocketIO. The emit strategy
branches on `has_request_context()`:

- **Inside a web request** → `socketio.start_background_task(_emit)` (non-blocking,
  avoids racing the polling→WebSocket upgrade handshake).
- **Outside a request context** → `socketio.emit()` called directly. Using
  `start_background_task` from a background job causes Flask-SocketIO to invoke
  the `on_connect` handler internally, which references `current_user`. With no
  session attached, Flask-Login returns `None` → `AttributeError`.

---

## 19. Hard Rules (additions 2026-04-17)

### 7.14 `inject_globals` Context Processor and Background Rendering

- `render_template_string()` and `render_template()` trigger **all registered
  context processors**, including `inject_globals` in `app/__init__.py`.
- `inject_globals` accesses `current_user.is_authenticated`. Outside a request
  context (e.g. APScheduler jobs calling `notify_new_ticket`), Flask-Login
  resolves `current_user` to `None`, causing `AttributeError`.
- **Rule:** Always guard `current_user` access in context processors with
  `has_request_context()` before calling `current_user.is_authenticated`:
  ```python
  if has_request_context() and current_user.is_authenticated:
  ```
- **Corollary:** Any service function that calls `render_template_string()` or
  `render_template()` from a background context (APScheduler, daemon threads,
  CLI commands) will trigger this processor. The guard in `inject_globals` is
  the single correct fix — do not work around it at the call site.

### 7.15 SocketIO `on_connect` / `on_disconnect` Handlers and Sessionless Contexts

- The `on_connect` and `on_disconnect` SocketIO event handlers in `api.py`
  reference `current_user`. When `socketio.emit()` or `start_background_task`
  is called without a session-backed request context, these handlers may be
  invoked with `current_user = None`.
- **Rule:** All SocketIO event handlers that reference `current_user` must guard
  with `if current_user and current_user.is_authenticated:` (not just
  `if current_user.is_authenticated:`).

### 7.16 `SystemSetting.value` Column Width

- `SystemSetting.value` was `VARCHAR(500)`. The `email_ingested_message_ids`
  key accumulates comma-separated Message-IDs; Outlook Message-IDs are ~100
  characters each, so 6+ processed messages overflow the column.
- **Widened to `TEXT` in migration 004** (`004_widen_system_setting_value`).
- **Rule:** Never use `VARCHAR` for any `SystemSetting` value that can accumulate
  list data or free-form strings of unbounded length. Use `Text`.

### 7.17 Diagnosis Protocol — Root Cause Before Fix

When an error is reported:
1. **Read the full traceback**, not just the exception message. Add `exc_info=True`
   to the catching `logger.error()` call if the traceback is being swallowed.
2. **Trace the complete call chain** from the error site back to the originating
   caller — do not assume the fix belongs at the first plausible location.
3. **Grep every file** in the call chain for the symbol that is `None` or missing
   before writing any code.
4. **Write the fix, verify syntax (`ast.parse`), and assert the changed lines are
   present** before packaging for deployment.
5. **Never apply a patch to a symptom site** (e.g. adding a guard to `api.py`)
   when the root cause is upstream (e.g. `inject_globals` crashing during
   `render_template_string`). Symptom-site patches mask errors without resolving
   them and accumulate technical debt.

---

## 20. Migration History (updated)

| Revision ID | Description |
|---|---|
| `001_widen_ticket_history_values` | Widen `ticket_history.old_value` / `new_value` from `VARCHAR(200)` to `TEXT` |
| `002_add_system_settings` | Create `system_settings` table |
| `003_render_comments` | Backfill comment bodies to HTML; widen `alembic_version.version_num` to `VARCHAR(64)` |
| `004_widen_system_setting_value` | Widen `system_settings.value` from `VARCHAR(500)` to `TEXT` |
| `005_add_watchers_and_time_entries` | Create `ticket_watchers` (UniqueConstraint + index) and `time_entries` (index) tables |

## 21. Browser Tab Notification Counter (added 2026-04-17)

### Overview

When the user has TechDesk open in a background tab, the browser tab title
displays the current unread notification count as a prefix so they can see
pending work without switching back.

Examples:
- `(3) Dashboard — TechDesk`  ← 3 unread notifications
- `Dashboard — TechDesk`       ← no unread notifications

### Implementation (`app/templates/base.html`)

**`<title>` tag** — given `id="page-title-text"` so JavaScript can read the
server-rendered base title without hardcoding it:
```html
<title id="page-title-text">{% block title %}...{% endblock %} — {{ branding.app_name }}</title>
```

**`updateDocumentTitle(unreadCount)`** — new JS function that sets
`document.title` to `"(n) <base title>"` when `n > 0`, or restores the
original title when `n === 0`.

**`_baseDocTitle`** — captured once at page load from the `<title>` element so
page-specific titles (e.g. "Ticket Detail", "Dashboard") are preserved exactly.

**`updateBadge()`** — extended to call `updateDocumentTitle(next)` on every
badge change, covering all update paths:
- Real-time WebSocket `new_notification` event
- Opening the notification panel (`loadNotifications` recalculates count)
- Clicking an individual notification to mark it read
- "Mark all read" button

**IIFE on page load** — reads the server-rendered badge count and calls
`updateDocumentTitle()` immediately, so the title is correct even before any
WebSocket event fires.

### Key Design Decisions

- No new routes, API calls, or Python changes — purely a frontend enhancement.
- `_baseDocTitle` is captured from the DOM, not hardcoded, so it works correctly
  for every page that extends `base.html` without any per-template changes.
- The count in the title stays in sync with the bell badge — a single
  `updateBadge()` call updates both, so they can never diverge.