classifieds/CLAUDE.md

# CLAUDE.md — Classifieds Marketplace

> Master spec and working agreement for an AI coding assistant.
> Read this fully before writing any code. Follow conventions exactly.
> Ask before deviating from schema, architecture, or naming.

---

## 1. Project Overview

National US classifieds marketplace targeting Vietnamese and Hispanic communities,
with full trilingual UI (English / Vietnamese / Spanish) and location-based filtering.

**Mission:** Connect people with needs — buy, sell, request, hire, and find services
across their local community.

**Six listing categories:**
1. For Sale — goods, items
2. Wanted / Requesting — reverse listings ("looking for X")
3. Jobs — employer posts + optional job-seeker posts
4. Services — offering labor/skills
5. Supplies — bulk/wholesale, B2B
6. Community — events, announcements, freebies

Each category has subcategories and a JSON-driven `field_schema`
(admin-editable, no redeploy required).

---

## 2. Tech Stack (fixed — do not substitute)

| Layer | Technology |
|---|---|
| Language | Python 3.10+ |
| Web framework | Flask (app factory + blueprints) |
| Database | **MySQL 8.0** — `utf8mb4` / `utf8mb4_unicode_ci`. NOT MariaDB. |
| ORM + migrations | SQLAlchemy 2.x + Alembic (Flask-Migrate) |
| Cache / queue / rate-limit | Redis |
| WSGI | Gunicorn (unix socket) |
| Reverse proxy / static | Nginx |
| Process manager | systemd |
| OS | Ubuntu 22.04 Server |
| Background jobs | APScheduler or RQ (expiry sweep, email, image processing) |
| Payments | Stripe (Billing + Payment Intents + Tax + Customer Portal) |
| Email | SMTP relay via Brevo (smart relay, existing infra) |
| i18n | Flask-Babel |
| Forms / CSRF | Flask-WTF |
| Password hashing | Argon2 (`argon2-cffi`) |
| Image processing | Pillow (re-encode, strip EXIF, thumbnail) |
| CAPTCHA | Cloudflare Turnstile (bypassed in dev when keys blank) |

**Current `requirements.txt`:**
```
Flask==3.0.3
Flask-SQLAlchemy==3.1.1
Flask-Migrate==4.0.7
Flask-Login==0.6.3
Flask-WTF==1.2.1
Flask-Babel==4.0.0
Flask-Limiter==3.8.0
SQLAlchemy==2.0.34
alembic==1.13.2
argon2-cffi==23.1.0
email-validator==2.2.0
itsdangerous==2.2.0
python-dotenv==1.0.1
redis==5.0.8
requests==2.32.3
PyMySQL==1.1.1
gunicorn==23.0.0
Pillow==10.4.0
```

---

## 3. Architecture

```
Internet
   │ 443 (TLS via Certbot)
Nginx
   ├─ /static/  → app/static/           (30d cache)
   ├─ /media/   → instance/media/       (7d cache, Phase 2+)
   └─ /         → Gunicorn unix socket  → Flask app factory
                                              │
                    ┌─────────────────────────┼──────────────┐
                  MySQL 8.0              Redis            Stripe
                (utf8mb4)        (sessions, cache,    (webhooks,
                                  rate-limit, RQ)      billing)
                                              │
                                         SMTP relay
                                          (Brevo)
```

**ProxyFix** is applied in the app factory (`x_for=1, x_proto=1, x_host=1`) so
`request.is_secure`, secure cookies, and rate-limit IP are all correct behind Nginx.

**systemd units (in `deploy/`):**
- `classifieds.service` — Gunicorn web process
- `classifieds-expire.service` + `classifieds-expire.timer` — hourly expiry sweep

---

## 4. Project File Structure

```
classifieds/
├── wsgi.py                      # WSGI entry point for Gunicorn + flask CLI
├── seed.py                      # Seed plans, categories, zip_geo; --admin flag
├── babel.cfg                    # pybabel extraction config
├── requirements.txt
├── .env.example                 # All env vars documented (no inline comments)
├── .gitignore
├── CLAUDE.md                    # This file
├── README.md                    # Setup, run, deploy instructions
│
├── app/
│   ├── __init__.py              # App factory: create_app()
│   ├── config.py                # BaseConfig / DevConfig / ProdConfig
│   ├── extensions.py            # db, migrate, login_manager, csrf, babel, limiter
│   │
│   ├── models/
│   │   ├── __init__.py          # Exports all models (import order matters)
│   │   ├── enums.py             # Role, UserStatus, TrustTier, TrustEventType,
│   │   │                        #   ListingStatus, Lang
│   │   ├── plan.py              # Plan (tier config JSON)
│   │   ├── user.py              # User (Argon2, RBAC helpers, Flask-Login)
│   │   ├── trust.py             # TrustEvent (append-only trust ledger)
│   │   ├── category.py          # Category (self-ref subcategories, field_schema)
│   │   ├── listing.py           # Listing + ListingImage
│   │   ├── geo.py               # ZipGeo + Metro
│   │   ├── messaging.py         # Conversation + Message
│   │   └── favorite.py          # Favorite
│   │
│   ├── blueprints/
│   │   ├── auth/                # register, login, logout, verify-email, reset
│   │   ├── main/                # index, /healthz
│   │   ├── i18n/                # /lang/<code> locale switcher
│   │   ├── listings/            # browse, detail, create, edit, delete, images
│   │   └── messaging/           # inbox, conversation, start, favorites, /api/unread
│   │
│   ├── services/
│   │   ├── email.py             # send_email (SMTP or dev console)
│   │   ├── turnstile.py         # Cloudflare Turnstile verify
│   │   ├── trust.py             # record_event, trust tier computation
│   │   ├── geo.py               # geocode_zip, haversine_mi, bounding_box
│   │   ├── field_schema.py      # validate_attributes, hot_values
│   │   ├── images.py            # process_upload, delete_image_files
│   │   ├── listings.py          # create/update/browse/search/radius/expiry
│   │   ├── messaging.py         # conversations, send, mark-read, inbox, unread
│   │   ├── favorites.py         # toggle, is_favorited, user_favorites
│   │   └── contact.py           # mask_body, contact_revealed, contact_density
│   │
│   ├── utils/
│   │   ├── __init__.py          # RBAC decorators: role_required, admin_required,
│   │   │                        #   moderator_required
│   │   ├── security.py          # hash_password, verify_password, generate_token,
│   │   │                        #   read_token (itsdangerous)
│   │   └── text.py              # normalize() — accent-insensitive (phở→pho, ñ→n)
│   │
│   ├── templates/
│   │   ├── base.html            # Layout: nav (Browse/Post/My listings/Saved/
│   │   │                        #   Messages+badge/Sign out), lang switcher, flashes
│   │   ├── index.html           # Landing page
│   │   ├── auth/                # login, register, reset_request, reset, _macros
│   │   ├── listings/            # browse, detail, form, mine
│   │   ├── messaging/           # inbox, conversation, start, favorites
│   │   └── errors/              # 403, 404, 500
│   │
│   ├── static/
│   │   └── style.css            # Single CSS file (Phase 1+2+3 accumulated)
│   │
│   └── translations/            # Flask-Babel .po/.mo for vi + es
│
├── migrations/                  # Alembic migration scripts (flask db migrate/upgrade)
│
├── deploy/
│   ├── classifieds.service      # systemd: Gunicorn web
│   ├── classifieds-expire.service # systemd: oneshot expiry sweep
│   ├── classifieds-expire.timer # systemd: hourly timer
│   ├── gunicorn.conf.py         # workers, socket, timeouts
│   └── nginx.conf.sample        # HTTPS redirect + proxy + static + media blocks
│
└── tests/
    └── test_smoke.py            # 34-check integration test (SQLite + in-memory Redis)
```

---

## 5. Route Map (24 routes, Phase 1–3)

| Method | Path | Blueprint | Auth |
|---|---|---|---|
| GET | `/` | main | — |
| GET | `/healthz` | main | — |
| GET/POST | `/auth/register` | auth | — |
| GET/POST | `/auth/login` | auth | — |
| GET | `/auth/logout` | auth | login |
| GET | `/auth/verify/<token>` | auth | — |
| GET/POST | `/auth/reset` | auth | — |
| GET/POST | `/auth/reset/<token>` | auth | — |
| GET | `/lang/<code>` | i18n | — |
| GET | `/listings` | listings | — |
| GET | `/listings/new` | listings | login |
| GET | `/listings/<id>` | listings | — |
| GET/POST | `/listings/<id>/edit` | listings | login+owner |
| POST | `/listings/<id>/delete` | listings | login+owner |
| POST | `/listings/<id>/sold` | listings | login+owner |
| POST | `/listings/<id>/images/<img_id>/delete` | listings | login+owner |
| GET/POST | `/listings/<id>/contact` | messaging | login |
| POST | `/listings/<id>/favorite` | messaging | login |
| GET | `/media/<path>` | listings | — |
| GET | `/my/listings` | listings | login |
| GET | `/my/favorites` | messaging | login |
| GET | `/messages` | messaging | login |
| GET/POST | `/messages/<id>` | messaging | login+participant |
| GET | `/api/unread` | messaging | login |

---

## 6. User Roles & RBAC

| Role | Capabilities |
|---|---|
| **free** | Register, post (limited), message, buy boosts, sees ads |
| **subscriber** | Tier features (Basic/Pro/Business), ad-free |
| **moderator** | Flag queue, hide/remove listings, ban users — no billing |
| **admin** | Full control |

RBAC enforced via decorators in `app/utils/__init__.py`:
- `@role_required('admin')`, `@admin_required`, `@moderator_required`
- Flask-Login `@login_required` for authenticated-only routes
- Ownership checks inline in route handlers (owner or moderator)

---

## 7. Subscription Tiers

Limits stored in `plans.config` JSON (admin-editable without redeploy).
Enforcement reads from plan at runtime via `plan.limit(key, default)`.

| Feature | Free | Basic | Pro | Business |
|---|---|---|---|---|
| Active listings | 3 | 15 | 50 | Unlimited |
| Listing life (days) | 14 | 30 | 60 | 90 |
| Images / listing | 3 | 8 | 15 | 25 |
| Featured slots / mo | 0 | 1 | 5 | 20 |
| Auto-bump | — | — | Weekly | Daily |
| Analytics | — | Basic | Full | Full + export |
| Storefront page | — | — | ✓ | ✓ + custom URL |
| Verified badge | — | — | ✓ | ✓ |
| Ad-free | — | ✓ | ✓ | ✓ |
| Scheduled posting | — | — | ✓ | ✓ |
| Bulk CSV upload | — | — | — | ✓ |
| Priority support | — | — | — | ✓ |

Plan slugs: `free`, `basic`, `pro`, `business`. Seeded by `seed.py`.

---

## 8. Monetization Streams

1. **Subscriptions** — recurring via Stripe Billing (Phase 4)
2. **À la carte boosts** — featured, bump, highlight, urgent — Stripe Payment Intents (Phase 4)
3. **Banner ads** — internal ad server, hidden for subscribers (Phase 5)
4. **Sponsors** — directory + category sponsorship (Phase 5)
5. **Promoted search** — keyword-pinned results (Phase 5+)

---

## 9. Database Schema

**All tables:** MySQL 8.0, `utf8mb4`, UTC datetimes, `BIGINT` PKs with
`BigInteger().with_variant(Integer, "sqlite"), autoincrement=True` for SQLite
test compatibility.

**Money:** integer cents always. Never floats for currency.

### Identity & Billing
```
users          id, email (unique), password_hash, display_name,
               role ENUM(free,subscriber,moderator,admin),
               status ENUM(active,suspended,banned),
               locale(5), tier_id→plans, trust_score INT, trust_tier ENUM,
               verified BOOL, email_verified BOOL, last_login_at,
               created_at, updated_at

plans          id, slug (unique), name, price_monthly_cents, stripe_price_id,
               config JSON, is_active, sort_order, created_at, updated_at

trust_events   id, user_id→users, type ENUM(account_age,listing_survived,
               flag_received,verified_email,payment), delta INT, created_at
```

### Catalog
```
categories     id, slug (unique), name, parent_id→categories (nullable),
               field_schema JSON, icon, sort_order, sponsor_id (nullable,
               FK wired Phase 5), is_active, created_at, updated_at

listings       id, user_id→users, category_id→categories,
               title(140), title_norm(140) [indexed, accent-stripped],
               body TEXT, lang ENUM(en,vi,es),
               price_cents INT nullable,
               zip(12), city(80), state(2), lat FLOAT, lng FLOAT,
               attributes JSON,
               attr_condition(40) [indexed], attr_job_type(40) [indexed],
               attr_salary_min INT [indexed], attr_salary_max INT [indexed],
               status ENUM(active,flagged,sold,expired,removed),
               is_featured BOOL, bump_at DATETIME, flag_count INT,
               view_count INT, expires_at DATETIME [indexed],
               created_at, updated_at
               Indexes: ix_listing_browse(status,expires_at),
                        ix_listing_sort(is_featured,bump_at)

listing_images id, listing_id→listings, path(255), thumb_path(255),
               sort_order, width, height, created_at
```

### Geo
```
zip_geo        zip PK, city, state [indexed], lat, lng, metro [indexed]
               (seed ~42k rows from SimpleMaps/Census ZCTA for prod)

metros         id, slug (unique), name, state, center_lat, center_lng
               (SEO landing page anchor — Phase 7)
```

### Messaging & Social
```
conversations  id, listing_id→listings, buyer_id→users, seller_id→users,
               last_message_at [indexed], created_at
               UNIQUE(listing_id, buyer_id)

messages       id, conversation_id→conversations, sender_id→users,
               body TEXT, read_at DATETIME nullable, created_at

favorites      id, user_id→users, listing_id→listings, created_at
               UNIQUE(user_id, listing_id)
```

### Monetization & Ops (Phase 4–6)
```
subscriptions  id, user_id→users, plan_id→plans, stripe_customer_id,
               stripe_sub_id, status ENUM(active,past_due,canceled,trialing),
               current_period_end, cancel_at_period_end BOOL,
               created_at, updated_at

transactions   id, user_id, type ENUM(subscription,boost,refund),
               amount_cents, currency, stripe_object_id, status,
               meta JSON, created_at

ads            id, advertiser_name, slot ENUM(header,sidebar,inline,footer),
               creative_path, target_url, lang nullable, geo_state nullable,
               starts_at, ends_at, impressions, clicks, is_active, created_at

sponsors       id, name, logo_path, url, tier ENUM(directory,category),
               category_id→categories nullable, starts_at, ends_at, is_active

boosts         id, listing_id→listings, user_id→users,
               type ENUM(featured,bump,highlight,urgent),
               expires_at, transaction_id→transactions, created_at

reports        id, listing_id→listings, reporter_id→users,
               reason ENUM(spam,scam,offensive,duplicate,miscategorized,other),
               note, created_at
               UNIQUE(reporter_id, listing_id)

audit_log      id, actor_id→users, action, target_type, target_id,
               meta JSON, created_at

settings       key PK, value JSON
               (site-wide toggles: registration_open, ads_enabled,
                maintenance_mode, flag_threshold, etc.)

listing_translations  id, listing_id→listings, lang, title, body, cached_at
                      (Phase 2 deferred: auto-translate cache)
```

---

## 10. Key Services (implemented)

### `services/listings.py`
- `create_listing(user, category, ...)` — tier-enforced active-listing cap, ZIP geocode,
  accent-norm `title_norm`, hot-column denormalization, `expires_at` from plan life
- `update_listing(listing, category, ...)` — same validations, in-place update
- `browse_query(...)` — SQLAlchemy query with category/state/price/condition/job_type filters
  and accent-insensitive keyword on `title_norm LIKE`
- `search_with_radius(base_query, lat, lng, radius_mi)` — bounding-box SQL prefilter
  + exact haversine refine, returns `[(listing, distance_mi), ...]`
- `expire_due_listings()` — flips active listings past `expires_at` to expired; called by
  `flask expire-listings` CLI + hourly systemd timer
- `can_create(user)`, `active_count(user)`, `image_cap(user)` — tier enforcement helpers

### `services/geo.py`
- `geocode_zip(zip)` — offline lookup from `zip_geo` table → `(lat, lng, city, state, metro)`
- `haversine_mi(lat1, lng1, lat2, lng2)` — exact distance
- `bounding_box(lat, lng, radius_mi)` → `(min_lat, max_lat, min_lng, max_lng)`

### `services/field_schema.py`
- `validate_attributes(category, raw_dict)` → `(cleaned, errors)` — validates against
  category `field_schema`, coerces types (text/number/select/bool), enforces required
- `hot_values(cleaned)` → dict of `attr_*` column values for denormalization

### `services/images.py`
- `process_upload(file_storage, listing_id, sort_order)` — validates MIME, re-encodes
  to JPEG (strips all EXIF), resizes to 1600px max, generates 400px thumbnail,
  randomized filename → returns unsaved `ListingImage`
- `delete_image_files(image)` — removes full + thumb from disk

### `services/messaging.py`
- `get_or_create_conversation(listing, buyer)` — idempotent; blocks seller self-message
- `send_message(conv, sender, body)` — validates participant, heuristic contact-density
  check (≥3 signals → warning logged), saves `Message`, updates `last_message_at`,
  sends notification email to recipient
- `mark_conversation_read(conv, reader)` — marks all other-party messages read
- `inbox(user, page, per_page)` — paginated conversations (buyer or seller), newest first
- `total_unread(user)` — sum of unread across all conversations; injected into every
  template via context processor for nav badge

### `services/contact.py`
- `mask_body(text, reveal)` — replaces phones/emails/URLs with `[hidden]` when `reveal=False`
- `contact_revealed(user)` — True only when `email_verified AND trust_tier >= trusted`
- `contact_density(text)` — count of contact signals (used for auto-flag heuristic)

### `services/trust.py`
- `record_event(user, event_type, delta)` — appends `TrustEvent`, updates `trust_score`,
  recomputes `trust_tier` (new=0, basic=5, trusted=20, verified=50)

### `services/favorites.py`
- `toggle_favorite(user_id, listing_id)` → bool (now favorited)
- `is_favorited(user_id, listing_id)` → bool
- `user_favorites(user_id, page, per_page)` → pagination

---

## 11. i18n (Trilingual: EN / VI / ES)

Two separate layers — never conflate:

**A. UI chrome** — Flask-Babel. All user-facing strings wrapped in `_()` /
`lazy_gettext()`. Locale resolution: `session['lang']` → `user.locale` →
`Accept-Language` header → default `en`.
```bash
pybabel extract -F babel.cfg -o messages.pot .
pybabel init -i messages.pot -d app/translations -l vi
pybabel init -i messages.pot -d app/translations -l es
# translate .po files ...
pybabel compile -d app/translations
```

**B. User content** — stored as-is, `listings.lang` declared.
Phase 2 deferred: auto-translate button → DeepL/Google → `listing_translations` cache.

**Accent-insensitive search** (`utils/text.py`):
```python
normalize("Phở Bò Đặc Biệt") == "pho bo dac biet"
normalize("Ñandú Jalapeño")   == "nandu jalapeno"
```
Vietnamese `đ/Đ` handled explicitly (NFKD decomposition misses it). Shadow column
`title_norm` stores the result; search queries normalize the input the same way.

---

## 12. Geography & Location

- `zip_geo` table: offline US ZIP → lat/lng/city/state/metro.
- **Seed sample:** 10 rows (Westminster CA, Garden Grove CA, San Jose CA, Houston TX,
  San Antonio TX, Miami FL, Hawthorne CA, Falls Church VA). Replace with full
  SimpleMaps/Census ZCTA (~42k rows) before production.
- On listing save: ZIP → lat/lng stored on listing.
- Browse: state filter (exact), keyword, price, hot-field filters.
- Radius search: bounding-box SQL prefilter → haversine exact refine.
- UI: near-me (browser geolocation) + manual ZIP + radius selector.
- `metros` table wired; SEO landing pages built in Phase 7.

**MySQL spatial upgrade** (optional, post-Phase-2):
```sql
ALTER TABLE listings ADD COLUMN geo POINT
  GENERATED ALWAYS AS (ST_SRID(POINT(lng, lat), 4326)) STORED,
  ADD SPATIAL INDEX spx_listings_geo (geo);
```

---

## 13. Anti-Abuse (post-and-flag model)

Listings go **active immediately** — no pre-approval gate by default.

**Status lifecycle:** `active` → `flagged` / `sold` / `expired` / `removed`

Controls in place (Phase 1–3):
- **Trust tier** — gates contact-info reveal, links in messages visible only to trusted+
- **Rate limits** — Flask-Limiter + Redis: register (10/hr), login (20/hr), post (30/hr),
  message start (20/hr)
- **Contact density heuristic** — ≥3 phone/email/URL signals in a message body → logged warning
- **Turnstile CAPTCHA** on register + post (bypassed in dev when keys blank)
- **Duplicate email** blocked at register

**Phase 6 additions (admin queue):** flag threshold N → auto-flip to `flagged`,
keyword blocklist, `reports` table workflow, `audit_log` for all mod actions.

---

## 14. Security

- Argon2 password hashing (`argon2-cffi`)
- CSRF on all forms (Flask-WTF)
- RBAC decorators on every protected route
- Rate limiting (Flask-Limiter + Redis) on auth + post endpoints
- Image upload: MIME sniff, Pillow re-encode (strips EXIF/metadata), max 8MB,
  max 1600px, randomized filenames
- Cloudflare Turnstile CAPTCHA on register + post
- Contact masking: phone/email/URL hidden for low-trust users
- Signed time-limited tokens for email verify + password reset (itsdangerous)
- ProxyFix for correct `is_secure` + client-IP behind Nginx
- Secrets via `.env` only — never committed
- HTTPS-only in prod; secure + httponly + samesite=Lax cookies
- `SESSION_COOKIE_SECURE=True` in ProdConfig

**Stripe (Phase 4):**
- Webhook signature verification on every event
- Stripe = source of truth; local DB mirrors via webhooks + nightly reconcile

---

## 15. Environment Variables (`.env`)

All comments must be on their own lines — no inline `# comments` after values
(python-dotenv does NOT strip them, causing `int()` parse errors).

```
FLASK_CONFIG=prod
SECRET_KEY=<long random string>
SERVER_NAME=classifieds.example.com

# Database
DB_USER=classifieds
DB_PASSWORD=<password>
DB_HOST=127.0.0.1
DB_PORT=3306
DB_NAME=classifieds
DATABASE_URL=
# leave DATABASE_URL blank; URI is built from DB_* parts

# Redis
REDIS_URL=redis://127.0.0.1:6379/0

# i18n
DEFAULT_LOCALE=en
SUPPORTED_LOCALES=en,vi,es

# Email (Brevo SMTP relay)
MAIL_SERVER=smtp-relay.brevo.com
MAIL_PORT=587
MAIL_USE_TLS=true
MAIL_USERNAME=<brevo login>
MAIL_PASSWORD=<brevo smtp key>
MAIL_FROM=no-reply@example.com
MAIL_FROM_NAME=Classifieds

# Media
MEDIA_ROOT=
# blank = instance/media (Flask instance folder)

# Turnstile (leave blank in dev to bypass)
TURNSTILE_SITE_KEY=
TURNSTILE_SECRET_KEY=

# Token lifetimes
TOKEN_VERIFY_MAX_AGE=86400
TOKEN_RESET_MAX_AGE=3600
```

---

## 16. Setup & Run

### Smoke test (no MySQL/Redis needed)
```bash
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt
python -m tests.test_smoke      # 34 checks, all green
```

### Dev server (absolute SQLite path — important)
Flask-SQLAlchemy resolves `sqlite:///dev.db` against `instance/`, not CWD.
Use an absolute path to avoid ghost-schema issues:
```
DATABASE_URL=sqlite:////tmp/classifieds_dev.db
```
```bash
export FLASK_APP=wsgi:app
flask db init
flask db migrate -m "initial schema"
flask db upgrade
python seed.py --admin admin@example.com 'StrongPass123'
flask run
```

### MySQL 8.0 setup
```sql
CREATE DATABASE classifieds CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'classifieds'@'localhost'  IDENTIFIED BY 'password';
CREATE USER 'classifieds'@'127.0.0.1'  IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON classifieds.* TO 'classifieds'@'localhost';
GRANT ALL PRIVILEGES ON classifieds.* TO 'classifieds'@'127.0.0.1';
FLUSH PRIVILEGES;
```
Both `@'localhost'` and `@'127.0.0.1'` are required — MySQL treats them
as different accounts. PyMySQL connects TCP (127.0.0.1) but MySQL may
resolve to localhost.

If you get `caching_sha2_password` auth errors:
```bash
pip install cryptography
```
Or create user with `IDENTIFIED WITH mysql_native_password BY '...'`.

### Production deploy
```bash
# 1. deploy code to /opt/classifieds (or /home/classifieds/classifieds)
# 2. python3 -m venv venv && pip install -r requirements.txt
# 3. cp .env.example .env && edit .env (no inline comments!)
# 4. flask db upgrade && python seed.py
# 5. cp deploy/classifieds.service /etc/systemd/system/
# 6. cp deploy/classifieds-expire.{service,timer} /etc/systemd/system/
# 7. systemctl enable --now classifieds classifieds-expire.timer
# 8. adapt deploy/nginx.conf.sample → /etc/nginx/sites-available/classifieds
# 9. ln -s /etc/nginx/sites-available/classifieds /etc/nginx/sites-enabled/
# 10. nginx -t && systemctl reload nginx
# 11. certbot --nginx -d classifieds.example.com
```

**Nginx common issues:**
- Config in `sites-available` but NOT symlinked to `sites-enabled` → requests
  fall through to default_server → wrong page served
- `curl -H "Host: classifieds.example.com" http://127.0.0.1/ -I` tests Nginx
  config independently of DNS
- DNS A record must point to the server's public IP before Let's Encrypt works

---

## 17. Coding Conventions

- App factory pattern. No global `app`. Extensions in `extensions.py`.
- Blueprints per domain. **Routes thin; all business logic in `services/`.**
- SQLAlchemy models, Alembic migrations for every schema change.
  Never hand-edit prod schema. Never use `db.create_all()` in production.
- All datetimes stored UTC. Localize only at render.
- Money: integer cents everywhere. Never `float` for currency.
- Enums: DB `ENUM` columns + Python `str, enum.Enum` classes in `models/enums.py`.
- `db.session.get(Model, pk)` — not the legacy `Model.query.get(pk)` (SA 2.0 deprecated).
- Validate `listings.attributes` against category `field_schema` server-side on every
  create/update. Never trust client-submitted attribute values.
- i18n: wrap ALL user-facing UI strings in `_()`. No hardcoded English in templates.
- Env-driven config. No secrets in code or VCS.
- CSRF token on every state-changing form (Flask-WTF handles automatically).
  For inline POST forms without a WTForms object, use `{{ csrf_token() }}`.
- `merge_query(**overrides)` context helper for pagination links (not a custom Jinja filter).

---

## 18. Build Roadmap & To-Do List

### ✅ Phase 1 — Foundation (Done)
- [x] App factory, Dev/Prod config, env-driven settings
- [x] Extensions: SQLAlchemy, Migrate, LoginManager, CSRF, Babel, Limiter
- [x] ProxyFix for Nginx X-Forwarded headers
- [x] `users`, `plans`, `trust_events` schema + Alembic migrations
- [x] Argon2 password hashing, itsdangerous signed tokens
- [x] Auth flows: register, email verify (+5 trust), login, logout, password reset
- [x] Rate limiting on auth endpoints (Flask-Limiter + Redis)
- [x] RBAC decorators: `role_required`, `admin_required`, `moderator_required`
- [x] Trilingual i18n scaffold (EN/VI/ES), locale switcher `/lang/<code>`
- [x] Accent-insensitive normalizer `utils/text.py` (phở→pho, ñ→n)
- [x] Cloudflare Turnstile CAPTCHA hook (dev bypass when keys blank)
- [x] `seed.py`: 4 plans with JSON limit configs + `--admin` flag
- [x] systemd + Gunicorn + Nginx deploy files
- [x] Smoke test: 14 checks

---

### ✅ Phase 2 — Listings Core (Done)
- [x] `categories` model (self-referential subcategories, `field_schema` JSON)
- [x] `listings` + `listing_images` models (title_norm, lat/lng, hot columns, status, expiry)
- [x] `zip_geo` + `metros` models
- [x] 6 categories + subcategories seeded with field schemas
- [x] 10 sample ZIP rows seeded (Westminster, Houston, San Jose, Miami, Falls Church…)
- [x] Field schema validation service (text/number/select/bool, required, coerce)
- [x] Hot-column denormalization (attr_condition, attr_job_type, attr_salary_*)
- [x] Listing CRUD routes: create, edit, delete, mark-sold
- [x] Tier enforcement: active-listing cap, image cap, listing life from plan config
- [x] Image pipeline: MIME validate, Pillow re-encode to JPEG, strip EXIF, thumbnail, random filenames
- [x] Browse + filter: category, state, price range, condition, keyword
- [x] Accent-insensitive keyword search via `title_norm LIKE`
- [x] Radius search: bounding-box SQL prefilter + exact haversine refine
- [x] View counter on listing detail
- [x] My-listings page with active/cap display
- [x] `flask expire-listings` CLI + systemd timer (hourly sweep)
- [x] Dev media serving route `/media/<path>`
- [x] Smoke test: +6 checks (geocode, tier limit, attr validation, search, radius, image, expiry)

---

### ✅ Phase 3 — Messaging + Favorites (Done)
- [x] `conversations` model (unique per listing+buyer, seller auto-set)
- [x] `messages` model (append-only, read_at tracking)
- [x] `favorites` model (unique user+listing)
- [x] Messaging service: get-or-create conversation, send, mark-read, inbox, total-unread
- [x] Self-message block (seller cannot contact own listing)
- [x] Contact density heuristic (≥3 phone/email/URL signals → warning log)
- [x] Contact masking service: mask phone/email/URL for low-trust users
- [x] Trust-gated contact reveal (email_verified AND trust_tier ≥ trusted)
- [x] Notification email to recipient on every new message
- [x] Favorites service: toggle, is_favorited, paginated user favorites
- [x] Inbox route with unread badge per conversation
- [x] Conversation thread route (GET masked, POST send reply)
- [x] Start-conversation from listing detail
- [x] Toggle-favorite endpoint (AJAX-capable + form fallback)
- [x] My-favorites page (saved listings grid)
- [x] `/api/unread` JSON endpoint
- [x] Unread count injected into every page via context processor (nav badge)
- [x] Nav: Messages + Saved links added
- [x] Listing detail: Contact seller + ♥ Save listing buttons for non-owners
- [x] Smoke test: +15 checks (conversation, messages, unread, masking, favorites, routes)

---

### 🔲 Phase 4 — Monetization / Stripe (Next)

**Models**
- [ ] `subscriptions` table (stripe_customer_id, stripe_sub_id, status, period_end, cancel_at_period_end)
- [ ] `transactions` table (type: subscription/boost/refund, amount_cents, stripe_object_id)
- [ ] `boosts` table (listing_id, type: featured/bump/highlight/urgent, expires_at)
- [ ] Alembic migration for all three tables

**Stripe setup**
- [ ] `stripe` pip package added to requirements.txt
- [ ] Stripe keys in `.env` (`STRIPE_SECRET_KEY`, `STRIPE_PUBLISHABLE_KEY`, `STRIPE_WEBHOOK_SECRET`)
- [ ] Create Products + Prices in Stripe dashboard for Basic/Pro/Business plans
- [ ] Map `stripe_price_id` in `plans` table (via seed or admin)
- [ ] Create boost Products in Stripe (featured, bump, highlight, urgent) + prices

**Subscriptions**
- [ ] `payments` blueprint: `/billing/subscribe/<plan_slug>` → Stripe Checkout Session
- [ ] Redirect to Stripe-hosted checkout, success/cancel return URLs
- [ ] Stripe Customer Portal route `/billing/portal` (self-serve upgrade/cancel/update card)
- [ ] Webhook endpoint `/billing/webhook` — verify signature on every event
- [ ] Handle `checkout.session.completed` → create/update `subscriptions` row, set `users.role=subscriber`, assign `tier_id`
- [ ] Handle `customer.subscription.updated` → sync status, period_end, plan change
- [ ] Handle `customer.subscription.deleted` → downgrade to free plan, clear tier
- [ ] Handle `invoice.payment_failed` → flip subscription status to `past_due`, email user
- [ ] Nightly reconcile job: compare local subscription status vs Stripe API (catch missed webhooks)
- [ ] Stripe Tax enabled on checkout (auto US sales tax)

**À la carte boosts (free + paid users)**
- [ ] Boost purchase route `/listings/<id>/boost` → Stripe Payment Intent (one-off)
- [ ] Boost type selector UI (featured / bump / highlight / urgent + price display)
- [ ] On payment success: write `boosts` row with `expires_at`, flip `listing.is_featured` if featured boost
- [ ] Boost expiry sweep added to `expire-listings` worker (clear expired boosts)
- [ ] Browse query: order featured (boosted) listings first

**Tier enforcement upgrades**
- [ ] Enforce `featured_per_month` limit from plan config (count active featured boosts)
- [ ] Enforce `scheduled_posting` — gate date-picker on plan check
- [ ] Enforce `ad_free` — suppress ads for subscriber role
- [ ] Upgrade prompt shown when free user hits any limit

**UI**
- [ ] Pricing page `/pricing` showing plan comparison table
- [ ] Account billing page `/my/billing` (current plan, next renewal, manage button → portal)
- [ ] Boost buttons on listing detail (owner-only) and my-listings page
- [ ] Payment success / cancel flash pages

**Tests**
- [ ] Smoke test: Stripe webhook handler with test payload (mock signature)
- [ ] Smoke test: boost creation writes correct `boosts` row
- [ ] Smoke test: subscription downgrade clears tier

---

### 🔲 Phase 5 — Ads & Sponsors

**Models**
- [ ] `ads` table (slot, creative_path, target_url, lang, geo_state, schedule, impressions, clicks)
- [ ] `sponsors` table (tier: directory/category, category_id, schedule)
- [ ] Alembic migration

**Ad server**
- [ ] Ad service: `get_ad_for_slot(slot, lang, state)` — picks active ad matching targeting, returns one
- [ ] Impression tracking: increment `ads.impressions` on serve (async-safe, batched to Redis then flush)
- [ ] Click tracking: redirect endpoint `/ads/<id>/click` → increment `ads.clicks` → redirect to `target_url`
- [ ] Ad slots in templates: header banner, sidebar (browse page), inline (every 6th listing card), footer
- [ ] Ads suppressed for `role=subscriber` (check `plan.limit('ad_free')`)
- [ ] Ad creative upload (admin): validate image, Pillow re-encode, store in `/media/ads/`

**Sponsors**
- [ ] Sponsor directory page `/sponsors`
- [ ] Category sponsorship: "Jobs powered by X" banner on category browse
- [ ] Sponsor logo in nav or footer (tier: directory)
- [ ] Sponsor admin CRUD

**Promoted search**
- [ ] `promoted_keywords` table (keyword, listing_id, expires_at, priority)
- [ ] Browse query: prepend promoted listings matching keyword before organic results
- [ ] Admin UI to assign promoted keyword slots

**Tests**
- [ ] Smoke: ad served for correct slot/lang/state targeting
- [ ] Smoke: click redirect increments counter
- [ ] Smoke: subscriber sees no ads

---

### 🔲 Phase 6 — Admin Backend

**Dashboard**
- [ ] `/admin` dashboard: KPI cards (active listings, new users 7d/30d, MRR, ad revenue, flag-queue depth)
- [ ] Mini charts: signups/day, listings/day (last 30d), revenue trend

**User management**
- [ ] `/admin/users` — searchable/filterable table (role, status, tier, trust)
- [ ] User detail page: profile, listing history, subscription, trust events
- [ ] Actions: ban/suspend/activate, tier override, trust score adjust, impersonate (logs to audit_log)
- [ ] Bulk actions: ban selected, send email to selected

**Listing moderation**
- [ ] `/admin/listings` — flag queue sorted by `flag_count × recency`
- [ ] Quick actions per listing: approve (clear flags), hide (flagged), remove, view
- [ ] Bulk approve / bulk remove
- [ ] Auto-flag threshold: N distinct-user flags → auto-flip to `flagged` (setting in `settings` table)
- [ ] Keyword blocklist editor (stored in `settings`, checked on listing submit)
- [ ] Duplicate body detection (hash `body` on submit, reject/flag if seen within 24h)

**Reports queue**
- [ ] `/admin/reports` — flagged content with reporter reasons
- [ ] Mark resolved / escalate actions
- [ ] `reports` table: unique per reporter+listing, reason enum

**Category management**
- [ ] `/admin/categories` — CRUD, reorder (drag or sort_order field)
- [ ] Field schema editor per category (JSON form builder: add/remove fields, set type/required/options)
- [ ] Preview of field schema as it would appear on listing form

**Plan / pricing management**
- [ ] `/admin/plans` — edit plan config JSON (limits), name, price, Stripe price ID, toggle active
- [ ] No-redeploy: limits read at runtime from DB

**Ads & sponsors management**
- [ ] `/admin/ads` — upload creative, set slot/targeting/schedule, view impression/click stats
- [ ] `/admin/sponsors` — CRUD sponsor entries, assign category
- [ ] Ad performance report (impressions, clicks, CTR per ad)

**Transactions & billing**
- [ ] `/admin/transactions` — full log with filter by type/status/date
- [ ] Refund action (calls Stripe Refund API, writes refund transaction row)
- [ ] Failed payments list with retry action

**Settings**
- [ ] `/admin/settings` — toggle UI for all `settings` table keys:
  - `registration_open` (bool)
  - `ads_enabled` (bool)
  - `maintenance_mode` (bool)
  - `flag_threshold` (int, default 5)
  - `new_user_trust_gate_days` (int)
  - `contact_density_threshold` (int)

**Audit log**
- [ ] All admin write actions write to `audit_log` (actor, action, target, meta JSON)
- [ ] `/admin/audit` — searchable audit trail

**Analytics**
- [ ] Traffic: page views/day, top pages, search terms used
- [ ] Conversions: registrations, listings posted, messages sent, boosts purchased
- [ ] Top categories by listing count and by view count
- [ ] Top metros by listing count (Phase 7 SEO planning)

**Tests**
- [ ] Smoke: admin dashboard 200, non-admin gets 403
- [ ] Smoke: flag threshold auto-flips listing status
- [ ] Smoke: impersonate logs to audit_log

---

### 🔲 Phase 7 — Polish

**SEO & discoverability**
- [ ] Metro landing pages `/classifieds/<metro-slug>` (e.g. `/classifieds/orange-county`)
- [ ] State landing pages `/classifieds/state/<state>`
- [ ] Category landing pages `/classifieds/category/<slug>`
- [ ] Dynamic `<title>` and `<meta description>` on all pages
- [ ] JSON-LD structured data on listing detail (Product schema)
- [ ] XML sitemap (`/sitemap.xml`) — listings + categories + metros, auto-updated
- [ ] `robots.txt`
- [ ] Open Graph tags (listing title, price, cover image) for social sharing

**Email notifications**
- [ ] Welcome email on registration (with verify link already sent — add branding)
- [ ] Listing expiry warning email (3 days before `expires_at`)
- [ ] Listing expired email (with renew CTA)
- [ ] Message received email (already inline — move to RQ worker for async)
- [ ] Payment failed email
- [ ] Weekly digest email (new listings in saved categories/metro) — opt-in

**Reviews**
- [ ] `reviews` table (listing_id, author_id, rating TINYINT, body, created_at)
- [ ] Leave review on completed transaction (mark-sold triggers prompt)
- [ ] Seller aggregate rating on profile + listing detail
- [ ] Moderation: flag/remove abusive reviews

**Translation (deferred from Phase 2)**
- [ ] Per-listing "Translate" button → DeepL/Google Translate API
- [ ] Cache result in `listing_translations` (pay once per listing+lang)
- [ ] Language filter on browse ("Show VI listings only")

**Performance**
- [ ] Switch sessions to Redis (`SESSION_TYPE=redis`)
- [ ] Query-level caching for hot browse queries (Redis, 60s TTL)
- [ ] Lazy-load listing images (native `loading="lazy"`)
- [ ] Serve WebP thumbnails (Pillow WebP encode alongside JPEG)
- [ ] Add MySQL `FULLTEXT(title, body)` index + switch keyword search to `MATCH ... AGAINST`
- [ ] Pagination `LIMIT/OFFSET` → keyset pagination for large datasets

**UX & mobile**
- [ ] Responsive nav (hamburger menu on mobile)
- [ ] Listing detail image gallery with lightbox
- [ ] Infinite scroll OR "Load more" on browse (AJAX pagination)
- [ ] Toast notifications (non-blocking flash messages)
- [ ] "Back to results" link on listing detail (preserve filter state)
- [ ] Listing preview before publish

**i18n completion**
- [ ] Extract all `_()` strings to `.pot` file
- [ ] Translate all strings to Vietnamese (`vi`)
- [ ] Translate all strings to Spanish (`es`)
- [ ] Compile `.mo` files, test all three locales end-to-end

**Tests**
- [ ] Smoke: sitemap returns valid XML with listing URLs
- [ ] Smoke: OG tags present on listing detail
- [ ] Smoke: translation cached correctly in `listing_translations`

---

### 🔲 Phase 8 — JSON API (Optional, for iOS app)

**Design**
- [ ] RESTful JSON API under `/api/v1/`
- [ ] JWT authentication (separate from session cookies)
- [ ] API versioning strategy documented

**Endpoints**
- [ ] `POST /api/v1/auth/register`
- [ ] `POST /api/v1/auth/login` → returns JWT
- [ ] `GET  /api/v1/listings` (browse + all filters, returns paginated JSON)
- [ ] `GET  /api/v1/listings/<id>`
- [ ] `POST /api/v1/listings` (create, multipart for images)
- [ ] `PUT  /api/v1/listings/<id>` (edit)
- [ ] `DELETE /api/v1/listings/<id>`
- [ ] `GET  /api/v1/messages` (inbox)
- [ ] `GET  /api/v1/messages/<conv_id>`
- [ ] `POST /api/v1/messages/<conv_id>` (send)
- [ ] `POST /api/v1/listings/<id>/favorite`
- [ ] `GET  /api/v1/categories`
- [ ] `GET  /api/v1/zip/<zip>` (geocode lookup)
- [ ] `GET  /api/v1/me` (profile + tier info)

**Infrastructure**
- [ ] JWT library added (`PyJWT` or `flask-jwt-extended`)
- [ ] Rate limiting on all API endpoints (stricter than web)
- [ ] API error responses: consistent `{"error": "...", "code": "..."}` JSON shape
- [ ] CORS headers for iOS app origin
- [ ] API smoke tests (separate from web smoke test)

---

### 🔲 Deferred Items (from earlier phases)

- [ ] **Auto-translate** — per-listing translate button → DeepL/Google → `listing_translations` cache (moved to Phase 7)
- [ ] **Full ZIP dataset** — replace 10-row sample with SimpleMaps US ZIP or Census ZCTA (~42k rows)
- [ ] **MySQL spatial upgrade** — `POINT` generated column + `SPATIAL INDEX` + `ST_Distance_Sphere` (optional perf upgrade)
- [ ] **MySQL FULLTEXT upgrade** — `FULLTEXT(title,body)` index + `MATCH ... AGAINST` (Phase 7)
- [ ] **MySQL generated hot columns** — `GENERATED ALWAYS AS (JSON_EXTRACT(...))` replacing app-maintained `attr_*` columns (optional)
- [ ] **RQ worker** — replace inline email sends + APScheduler with proper RQ queue + worker process
- [ ] **Redis sessions** — switch from filesystem sessions to Redis-backed (`SESSION_TYPE=redis`)
- [ ] **Bulk CSV listing upload** — Business tier feature (parse CSV, validate, batch-create listings)
- [ ] **Scheduled posting** — Pro/Business: set `publish_at` datetime, worker flips to active
- [ ] **Auto-bump/renew** — Pro (weekly) / Business (daily): worker resets `bump_at`, extends `expires_at`
- [ ] **Storefront/profile page** — Pro/Business: public seller page `/seller/<username>` with all active listings
- [ ] **Custom URL** — Business: vanity slug for storefront (e.g. `/shop/alices-cleaning`)
- [ ] **Job-seeker posts** — reverse job listings ("I'm looking for work") under Jobs category

---

## 19. Known Issues / Decisions Locked

| Item | Decision |
|---|---|
| DB portability | SQLite for tests (with `BigInteger().with_variant(Integer,"sqlite")` + `autoincrement=True`), MySQL 8.0 for prod. MariaDB NOT supported. |
| Spatial search | lat/lng + bounding box + haversine for now. MySQL `POINT` + `SPATIAL INDEX` upgrade SQL documented in README. |
| FULLTEXT search | `title_norm LIKE` for now. MySQL `FULLTEXT(title,body)` + `MATCH ... AGAINST` upgrade documented in README. |
| Hot filter columns | App-maintained `attr_*` indexed columns. MySQL `GENERATED ALWAYS` upgrade documented in README. |
| ZIP dataset | 10-row sample seeded. Replace with SimpleMaps US ZIP or Census ZCTA (~42k rows) before prod. |
| Workers | No RQ/Celery yet. Expiry sweep via systemd timer. Message notifications inline. Phase 4+ may add RQ for async email. |
| Session storage | Flask default (filesystem/cookie). Phase 4: switch to Redis sessions for scale. |
| inline `.env` comments | MUST NOT have `# comments` on the same line as values — python-dotenv does not strip them, causing `int()` ValueError. Comments go on their own lines. |

---

_End of spec. Phase 1–3 complete, 34 smoke-test checks green. Phases 4–8 fully detailed as to-do lists above. Next: Phase 4 (Monetization / Stripe)._