Jul 14 - Using CDN - Phase 2a
This commit is contained in:
@@ -1359,12 +1359,20 @@ timeout = 30
|
||||
- **Deferred to Phase 2 (not yet routed through storage):** photo *deletes* (issues.py ~1013, inspections.py ~1340) and PDF-export image *reads* (`utils/pdf_export.py`). These still hit local disk directly — fine while `local`, must be routed before the `s3` flip. **Bug to fix when routing:** the inspection-delete cleanup builds `os.path.join(UPLOAD_FOLDER, '..', 'static', rel_path)` which normalizes to `app/static/static/...` (double `static`) and never deletes — routing it through `storage.delete(key)` fixes it. The issue-delete path (`root_path + 'static' + rel_path`) is already correct.
|
||||
|
||||
### Phase 2 — R2 backend (build, do NOT flip yet)
|
||||
- [ ] `pip install boto3`; add to `requirements.txt`.
|
||||
- [ ] `.env`: `STORAGE_BACKEND`, `R2_ENDPOINT_URL`, `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_BUCKET`, `R2_PRESIGN_TTL=86400`.
|
||||
- [ ] `S3Backend`: boto3 client (`endpoint_url=R2_ENDPOINT_URL`, `region_name='auto'`). `save` → `put_object` (set `ContentType`); `media_url` → `generate_presigned_url('get_object', ..., ExpiresIn=TTL)`; `read` → `get_object`; `exists` → `head_object`.
|
||||
- [ ] Fallback: if `head_object` 404s, `S3Backend.read`/`media_url` fall back to the local static path.
|
||||
- [ ] Route the deferred callers through storage (required before the flip): photo *deletes* (issues.py, inspections.py) → `storage.delete(key)`; PDF-export image *reads* (`utils/pdf_export.py`, `os.path.join(static_folder, v)`) → `storage.read(key)`. Fix the inspection-delete double-`static` path in the process.
|
||||
- [ ] Leave `STORAGE_BACKEND=local` in production for now.
|
||||
**2a — delivered:**
|
||||
- [x] `boto3>=1.34` added to `requirements.txt` (imported **lazily** in `S3Backend` — a `local` deploy doesn't need it installed).
|
||||
- [x] R2 config keys in `config.py`: `R2_ENDPOINT_URL`, `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_BUCKET`, `R2_PRESIGN_TTL` (default 86400), `R2_MEDIA_FALLBACK` (default false).
|
||||
- [x] `S3Backend` in `storage.py`: private bucket; `save` → `put_object` (ContentType from ext); `media_url` → presigned GET (TTL); `read` → `get_object`; `exists`/`_head` → `head_object`; `delete` → `delete_object`. Client cached in `app.extensions`, `region_name='auto'`, SigV4.
|
||||
- [x] Transition fallback: `read()` falls back to the local copy on a missing object; `media_url()` falls back to a local static URL **only when `R2_MEDIA_FALLBACK=true`** (off by default — avoids a HEAD per image; cutover is gated on full verification anyway).
|
||||
- [x] Routed photo **deletes** (issues.py, inspections.py) → `storage.delete(key)`; **fixed** the inspection-delete double-`static` bug in the process.
|
||||
|
||||
**2b — remaining before cutover:**
|
||||
- [ ] Route PDF-export image **reads** through storage. `pdf_export.py` resolves `os.path.join(static_folder, key)` at several sites (`_form_fields_section`, `generate_issue_pdf`, `_compress_image`). Lowest-risk approach: a `storage.materialize_to_dir(keys)` helper — local backend returns the real static folder (no-op); s3 downloads referenced keys to a temp dir mirroring `key` layout, and the two entry points (`generate_inspection_pdf`, `generate_issue_pdf`) point `static_folder` at it (try/finally cleanup). Leaves all render sites untouched.
|
||||
|
||||
**Operator setup (before flip, not code):**
|
||||
- [ ] Create R2 bucket (e.g. `jqc-media`), US region; create an R2 API token → Access Key + Secret + endpoint.
|
||||
- [ ] `.env`: `R2_ENDPOINT_URL`, `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_BUCKET=jqc-media`, `R2_PRESIGN_TTL=86400`. Keep `STORAGE_BACKEND=local` for now.
|
||||
- [ ] `pip install boto3` into the venv.
|
||||
|
||||
### Phase 3 — Verified migration sync
|
||||
- [ ] `scripts/migrate_photos_to_r2.py` — idempotent, resumable, checksummed.
|
||||
|
||||
Reference in New Issue
Block a user