nngo/JQC_multi_tenant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Jun 27 Update documents

Browse Source
This commit is contained in:
nngo
2026-06-27 10:28:57 -04:00
parent 9f0f7d2f18
commit 7b764c76af
3 changed files with 466 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+230 -21
View File
@@ -2,7 +2,7 @@
> **Audience:** AI assistants and developers working on this codebase.
> **Purpose:** Authoritative reference for architecture, conventions, gotchas, and decisions.
> **Last reviewed:** June 2026 (Phase 19 complete + mobile API gap-fill Phases A–E + customer UI refinements + Phase 22 comment visibility + Phase 23 support chat/tickets + inspector performance Excel export + inspection list filters + customer issue logging + AI chatbot + dashboard grouped sections + issues/inspections PDF export + date/ID filters + Reports expansion Phases R1–R4 + Phase 24 issue_created notify defaults + Phase 25 inspection GPS + Phase 26 issue vendor fields + Phase 27 facility score alerts)
> **Last reviewed:** June 2026 (Phase 19 complete + mobile API gap-fill Phases A–E + customer UI refinements + Phase 22 comment visibility + Phase 23 support chat/tickets + inspector performance Excel export + inspection list filters + customer issue logging + AI chatbot + dashboard grouped sections + issues/inspections PDF export + date/ID filters + Reports expansion Phases R1–R4 + Phase 24 issue_created notify defaults + Phase 25 inspection GPS + Phase 26 issue vendor fields + Phase 27 facility score alerts + **MT-0 through MT-3 multi-tenant control plane**)
---
@@ -29,6 +29,7 @@
19. [Infrastructure](#19-infrastructure)
20. [Known Constraints & Hard Rules](#20-known-constraints--hard-rules)
21. [Change Philosophy](#21-change-philosophy)
22. [Multi-Tenant Architecture (MT-0 → MT-3)](#22-multi-tenant-architecture-mt-0--mt-3)
---
@@ -82,7 +83,14 @@ The application is actively deployed in production and maintained by a single de
```
lt_janitorial_quality_control/
├── app/
│ ├── __init__.py # Application factory — limiter, csrf, db, mail, login_manager
│ ├── __init__.py # Application factory — limiter, csrf, db (RoutingSession), mail, login_manager, init_tenancy
│ ├── tenancy/ # MT-1 — tenant resolution + DB routing (inert unless MULTI_TENANT_ENABLED=true)
│ │ ├── __init__.py # public exports: RoutingSession, init_tenancy, TenantContext
│ │ ├── context.py # TenantContext frozen dataclass (g.tenant)
│ │ ├── engine_cache.py # per-tenant SQLAlchemy engine cache + invalidate()
│ │ ├── middleware.py # init_tenancy() — before_request Host→tenant resolver
│ │ ├── resolver.py # resolve_tenant(host) → TenantContext | None
│ │ └── routing.py # RoutingSession — routes db.session to g.tenant_engine
│ ├── api/ # Mobile REST API
│ │ ├── __init__.py # api_bp parent blueprint + register_api()
│ │ ├── auth.py # /api/v1/auth/* and /api/v1/devices/*
@@ -133,7 +141,24 @@ lt_janitorial_quality_control/
│ └── utils/
├── migrations/
│ └── versions/
│ └── phase23_support_tickets.py ← HEAD
│ ├── 0003_add_user_active.py ← squashed baseline (chain root, MT-2)
│ └── phase32_device_token_columns ← HEAD
├── migrations_tenant/ # MT-2 — standalone Alembic env for per-tenant upgrades
│ ├── env.py # URL-driven, no Flask; reuses migrations/versions
│ └── script.py.mako
├── control/ # MT-0 — control plane (tenant registry, plans, provisioning)
│ ├── __init__.py
│ ├── base.py # ControlBase, engine/session (CONTROL_DATABASE_URL)
│ ├── cli.py # seed / create-superadmin / list-plans
│ ├── crypto.py # Fernet encrypt/decrypt (CONTROL_FERNET_KEY)
│ ├── models.py # Plan, PlanFeature, Tenant, TenantDomain,
│ │ # Superadmin, ProvisioningJob, TenantAudit
│ ├── provision.py # MT-3 — create_tenant / register_tenant_zero / delete_tenant
│ ├── seed.py # idempotent plan seeder
│ ├── tenant_migrate.py # MT-2 — upgrade_tenant / bootstrap_tenant / chain_head CLI
│ ├── time_utils.py # now_eastern() mirror (no app import)
│ └── migrations/ # control Alembic chain (control{N}_…)
│ └── versions/control0001_init.py ← HEAD
└── ...
```
@@ -157,6 +182,15 @@ lt_janitorial_quality_control/
| `REDIS_URL` | Optional. When set, Flask-Limiter uses Redis for shared rate-limit counters across Gunicorn workers. |
| `GROQ_API_KEY` | Optional. When set, enables the AI chatbot at `/support/chat`. Absent → chat input disabled; customers see a "Submit to Support" fallback only. |
| `GROQ_MODEL` | Optional. Groq model ID. Defaults to `llama-3.3-70b-versatile`. |
| `MULTI_TENANT_ENABLED` | `false` by default. Set `true` to activate Host→tenant routing. Requires all control-plane vars below. |
| `CONTROL_DATABASE_URL` | Control-plane MySQL URI, e.g. `mysql+pymysql://jqc_control:pw@127.0.0.1/jqc_control`. Required when `MULTI_TENANT_ENABLED=true`. |
| `CONTROL_FERNET_KEY` | Fernet key for encrypting tenant DB passwords. Generate once; store in `/etc/jqc/control.env`. |
| `PROVISION_DB_URL` | MySQL account that can `CREATE DATABASE` / `CREATE USER` / `GRANT`. e.g. `mysql+pymysql://jqc_provisioner:pw@127.0.0.1/`. |
| `TENANT_BASE_DOMAIN` | Apex domain for subdomains, e.g. `jqc.app`. Used by provisioner to build `<slug>.jqc.app`. |
| `MULTI_TENANT_EXEMPT_PATHS` | Comma-separated path prefixes that bypass the tenant gate (e.g. `/health`). `/static/` is always exempt. |
| `TENANT_ENGINE_POOL_SIZE` | Per-tenant engine pool size (default 5). |
| `TENANT_ENGINE_MAX_OVERFLOW` | Per-tenant pool max overflow (default 5). |
| `TENANT_ENGINE_POOL_RECYCLE` | Pool recycle in seconds (default 1800). |
### Email SSL Auto-Detection
@@ -632,24 +666,53 @@ limiter = Limiter(
## 17. Alembic Migration Chain
**Current HEAD:** `phase32_device_token_columns` (30 migrations total).
**Chain root:** `0003_add_user_active` — a guarded squashed baseline (MT-2) that recreates the full 25-table schema with INFORMATION_SCHEMA guards. The original baseline migrations (0001/0002/0003) were lost; this file restores the chain root so Alembic can build the revision map. `down_revision = None`.
```
phase1_projects_roles → phase6_features → phase7_mobile_api → phase8_notification_matrix
→ phase9_user_full_name → phase10_customer_password_setup → phase11_director_role
→ phase12_performance_indexes → phase_b_mobile_local_id
→ phase13_issue_facility → phase14_facility_created_at
→ phase15_audit_log_indexes → phase16_notifications_columns
→ phase17_notification_event_type
→ phase18_issue_reported_by
→ phase19_issue_mobile_photos
→ phase20_inspector_assignments
→ phase21_template_active
→ phase21_performance_indexes
→ phase22_comment_visibility
→ phase23_support_tickets
→ phase24_notify_defaults
→ phase25_inspection_gps
→ phase26_issue_vendor
→ phase27_score_alerts ← HEAD
0003_add_user_active (baseline, MT-2)
→ phase1_projects_roles → phase6_features → phase7_mobile_api → phase8_notification_matrix
→ phase9_user_full_name → phase10_customer_password_setup → phase11_director_role
→ phase12_performance_indexes → phase_b_mobile_local_id
→ phase13_issue_facility → phase14_facility_created_at
→ phase15_audit_log_indexes → phase16_notifications_columns
→ phase17_notification_event_type
→ phase18_issue_reported_by
→ phase19_issue_mobile_photos
→ phase20_inspector_assignments
→ phase21_template_active
→ phase21_performance_indexes
→ phase22_comment_visibility
→ phase23_support_tickets
→ phase24_notify_defaults
→ phase25_inspection_gps
→ phase26_issue_vendor
→ phase27_score_alerts
→ ... → phase32_device_token_columns ← HEAD
```
### Fresh DB provisioning (multi-tenant)
**Never use `flask db upgrade` on an empty database.** Fourteen of the phase migrations are not idempotent (no `INFORMATION_SCHEMA` guards) and will fail on a fresh DB that already has the baseline schema. Use the provisioner instead:
```bash
python -m control.tenant_migrate bootstrap --tenant <slug>
# Runs: upgrade to 0003_add_user_active (builds full schema) → stamp head
# Phase migrations are SKIPPED — baseline covers the full schema.
```
Ongoing incremental upgrades (phase33+, which MUST be guarded) use:
```bash
python -m control.tenant_migrate upgrade --tenant all
```
### Standard single-tenant migration deploy
```bash
flask db upgrade # existing LT box — safe, already at head
sudo systemctl restart jqc
```
### phase21_performance_indexes
@@ -945,6 +1008,47 @@ timeout = 30
### Nginx
- `client_max_body_size 50M`
- Passes `X-Forwarded-For`
- **Multi-tenant wildcard block** (alongside the existing single-domain block):
```nginx
server {
listen 80;
server_name *.jqc.app jqc.app;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host; # resolver reads this
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
```
**TLS for subdomains:** wildcard cert (`*.jqc.app`) via DNS-01 challenge (certbot + DNS plugin). HTTP-01 works for individual subdomains only and cannot auto-renew a wildcard. Custom-domain TLS via Caddy on-demand TLS (MT-6).
### Control-plane environment file
**All** control/provisioning env vars must live in one canonical file. Both the systemd unit and CLI sessions must source the same file — a key mismatch between them causes `cryptography.fernet.InvalidToken` at request time.
```bash
# /etc/jqc/control.env (chmod 640, chown root:jqc)
CONTROL_DATABASE_URL=mysql+pymysql://jqc_control:<pw>@127.0.0.1/jqc_control
CONTROL_FERNET_KEY=<generated key — no quotes>
PROVISION_DB_URL=mysql+pymysql://jqc_provisioner:<pw>@127.0.0.1/
TENANT_BASE_DOMAIN=jqc.app
MULTI_TENANT_ENABLED=true
```
```ini
# systemd unit [Service]
EnvironmentFile=/etc/jqc/control.env
```
```bash
# CLI sessions
set -a; . /etc/jqc/control.env; set +a
```
### Recommended Cron Schedule
```bash
@@ -1026,6 +1130,12 @@ timeout = 30
| 68 | **Support ticket customer replies revert status from `answered` → `open`** | When a customer posts a follow-up on an answered ticket, the route sets `ticket.status = 'open'` so admins see it in their open queue. Admin must manually close or re-answer. |
| 69 | **Customer issue create: `assigned_to` field hidden, `facility_id` scoped to `get_customer_scope()`** | `issues.create()` detects `role == 'customer'`, scopes facilities to the customer's assigned set, sets `staff = []` for the assigned_to dropdown, and hides the field in `form.html`. `IssueForm.facility_id.choices` must still include all active facilities so POST validation passes. |
| 70 | **`notify()` does NOT commit — caller must `db.session.commit()` after all `notify()` calls** | `notify()` adds a `Notification` row to the session but leaves the commit to the caller. The support helpers (`_notify_admins_new_ticket`, `_notify_customer_reply`, `_notify_admins_customer_reply`) each call `db.session.commit()` after the `notify()` loop. |
| 71 | **`CONTROL_FERNET_KEY` must be identical between CLI sessions and the Gunicorn service** | Provisioning encrypts tenant creds; the app decrypts them at request time. A key mismatch causes `cryptography.fernet.InvalidToken`. Canonical source: `/etc/jqc/control.env` — sourced by both `EnvironmentFile=` in the systemd unit and `set -a; . /etc/jqc/control.env; set +a` in CLI sessions. |
| 72 | **`MULTI_TENANT_ENABLED` must be flipped only after tenant-zero (LT) is registered** | The resolver gates all traffic by Host. If LT's domains aren't in `tenant_domains` when the flag goes `true`, LT's own traffic gets a 404 "Workspace not found" page. Register LT first via `register-tenant-zero`, confirm with `curl -H "Host: lts.jqc.app" http://127.0.0.1:8000/`, then flip the flag. |
| 73 | **Tenant DB passwords use `_gen_password()` — not `secrets.token_urlsafe()`** | MySQL `validate_password` MEDIUM policy requires lower + upper + digit + special. `token_urlsafe` is alphanumeric-only and fails intermittently. `_gen_password()` guarantees all four character classes. |
| 74 | **`bootstrap_tenant()` for fresh DBs; `upgrade_tenant()` for incremental** | Fourteen phase migrations are unguarded. Running the full chain on an empty DB (from the baseline) causes duplicate-column errors. `bootstrap_tenant()` runs the baseline to HEAD then stamps — phases skipped. `upgrade_tenant()` is for phase33+ incremental upgrades on already-provisioned DBs. |
| 75 | **`delete_tenant()` + `_add_domains()` are retry-safe** | `_add_domains` uses `_upsert_domain()` (delete-then-insert) to handle orphan rows from failed partial runs. `delete_tenant()` also purges by derived domain string, not only by `tenant_id`, catching orphans whose parent tenant row was rolled back. |
| 76 | **`register-tenant-zero` never bootstraps and never drops the DB** | `register_tenant_zero()` only reads the existing head, inserts control rows, and maps domains. `delete_tenant()` on tenant-zero must never use `--drop-db` — the guard checks `db_name == db_name_for(slug)` and refuses non-provisioner-named DBs (LT's DB name is `jqc_lt`, not `jqc_lts`). |
---
@@ -1038,4 +1148,103 @@ timeout = 30
5. **Migration existence checks** — all migrations safe to re-run
6. **Full file contents for 1–3 file changes**; deployment map for larger changesets
7. **Explicit deploy instructions** — migration steps separated from code steps
8. **Root cause analysis** on errors — never apply temporary workarounds
8. **Root cause analysis** on errors — never apply temporary workarounds
---
## 22. Multi-Tenant Architecture (MT-0 → MT-3)
See `MULTI_TENANT_PLAN.md` for the full phased roadmap. This section summarises what is built and operational.
### Model
Shared codebase + **database-per-tenant**. One Flask/Gunicorn process serves all tenants. `before_request` resolves the `Host` header → tenant → per-tenant MySQL database. All existing `db.session` calls route transparently — zero changes to models or routes.
### Control plane (`control/`)
Self-contained package, own `ControlBase` + engine/session, own Alembic chain. No imports from `app/`.
| Module | Purpose |
|---|---|
| `base.py` | `ControlBase`, `control_session()` context manager, engine from `CONTROL_DATABASE_URL` |
| `models.py` | `Plan`, `PlanFeature`, `Tenant`, `TenantDomain`, `Superadmin`, `ProvisioningJob`, `TenantAudit` |
| `crypto.py` | Fernet encrypt/decrypt for `tenants.db_password_enc` |
| `seed.py` | Idempotent plan seeder (Free / Starter / Pro / Enterprise) |
| `cli.py` | `seed`, `create-superadmin`, `list-plans` |
| `tenant_migrate.py` | `bootstrap_tenant()`, `upgrade_tenant()`, `chain_head()`, `current_revision()` + CLI |
| `provision.py` | `create_tenant()`, `register_tenant_zero()`, `delete_tenant()` + CLI |
### Data-plane changes (`app/`)
| File | Change |
|---|---|
| `app/__init__.py` | `db = SQLAlchemy(session_options={'class_': RoutingSession})` + `init_tenancy(app)` |
| `config.py` | `MULTI_TENANT_ENABLED` (default `false`) + pool tunables |
| `app/tenancy/` | New package — see §3 repo layout |
### Plan tiers
| Axis | Free | Starter | Pro | Enterprise |
|---|---|---|---|---|
| Max users | 3 | 15 | 50 | unlimited |
| Max facilities | 2 | 10 | 50 | unlimited |
| Inspections / month | 50 | 500 | 5 000 | unlimited |
| Issues / month | 50 | 500 | 5 000 | unlimited |
| Mobile API | ✗ | ✓ | ✓ | ✓ |
| Scheduled reports | ✗ | ✗ | ✓ | ✓ |
| Branding | ✗ | ✗ | ✓ | ✓ |
| Custom domain | ✗ | ✗ | ✓ | ✓ |
| Subdomain | ✓ | ✓ | ✓ | ✓ |
Quota-exceed behaviour: **soft warn** — allow submit, flag for upgrade, never reject.
### Provisioner MySQL account (required grants)
```sql
GRANT ALL PRIVILEGES ON *.* TO 'jqc_provisioner'@'localhost' WITH GRANT OPTION;
GRANT CREATE USER ON *.* TO 'jqc_provisioner'@'localhost';
FLUSH PRIVILEGES;
```
### CLI quick-reference
```bash
# Control schema + plans + first superadmin (run once)
alembic -c control/migrations/alembic.ini upgrade head
python -m control.cli seed
python -m control.cli create-superadmin --username admin --email you@example.com
# Adopt existing LT database as tenant-zero (run once, no data move)
python -m control.provision register-tenant-zero \
--slug lts --name "LT Services" --plan enterprise \
--db-host 127.0.0.1 --db-name <LT_DB> --db-user <LT_USER> --db-password '<pw>' \
--custom-domain jqc.ltservicesinc.com --base-domain jqc.app
# Provision a new tenant (creates MySQL DB + user + schema + first admin)
python -m control.provision create-tenant \
--slug acme --name "Acme Corp" --plan pro --admin-email ops@acme.com
# Delete / deregister a tenant
python -m control.provision delete-tenant --slug ztest --drop-db --yes # provisioned DB
python -m control.provision delete-tenant --slug lts --yes # adopted DB — no --drop-db
# Migration status + upgrades
python -m control.tenant_migrate heads
python -m control.tenant_migrate current --tenant all
python -m control.tenant_migrate upgrade --tenant all # incremental (phase33+)
python -m control.tenant_migrate bootstrap --tenant acme # fresh DB only
```
### Enable multi-tenancy (cutover sequence)
```bash
# 1. Register LT as tenant-zero (see above)
# 2. Smoke-test routing (flag still off)
curl -sI -H "Host: lts.jqc.app" http://127.0.0.1:8000/ | head -2
curl -sI -H "Host: ztest.jqc.app" http://127.0.0.1:8000/ | head -2
# 3. Wildcard DNS: *.jqc.app A <SERVER_IP>
# 4. Nginx wildcard server block (see §19)
# 5. Wildcard TLS cert (DNS-01, certbot)
# 6. Add to /etc/jqc/control.env: MULTI_TENANT_ENABLED=true
# 7. sudo systemctl daemon-reload && sudo systemctl restart jqc
```