# CLAUDE.md — Classifieds Marketplace > Master spec and working agreement for an AI coding assistant. > Read this fully before writing any code. Follow conventions exactly. > Ask before deviating from schema, architecture, or naming. --- ## 1. Project Overview National US classifieds marketplace targeting Vietnamese and Hispanic communities, with full trilingual UI (English / Vietnamese / Spanish) and location-based filtering. **Mission:** Connect people with needs — buy, sell, request, hire, and find services across their local community. **Six listing categories:** 1. For Sale — goods, items 2. Wanted / Requesting — reverse listings ("looking for X") 3. Jobs — employer posts + optional job-seeker posts 4. Services — offering labor/skills 5. Supplies — bulk/wholesale, B2B 6. Community — events, announcements, freebies Each category has subcategories and a JSON-driven `field_schema` (admin-editable, no redeploy required). --- ## 2. Tech Stack (fixed — do not substitute) | Layer | Technology | |---|---| | Language | Python 3.10+ | | Web framework | Flask (app factory + blueprints) | | Database | **MySQL 8.0** — `utf8mb4` / `utf8mb4_unicode_ci`. NOT MariaDB. | | ORM + migrations | SQLAlchemy 2.x + Alembic (Flask-Migrate) | | Cache / queue / rate-limit | Redis | | WSGI | Gunicorn (unix socket) | | Reverse proxy / static | Nginx | | Process manager | systemd | | OS | Ubuntu 22.04 Server | | Background jobs | APScheduler or RQ (expiry sweep, email, image processing) | | Payments | Stripe (Billing + Payment Intents + Tax + Customer Portal) | | Email | SMTP relay via Brevo (smart relay, existing infra) | | i18n | Flask-Babel | | Forms / CSRF | Flask-WTF | | Password hashing | Argon2 (`argon2-cffi`) | | Image processing | Pillow (re-encode, strip EXIF, thumbnail) | | CAPTCHA | Cloudflare Turnstile (bypassed in dev when keys blank) | **Current `requirements.txt`:** ``` Flask==3.0.3 Flask-SQLAlchemy==3.1.1 Flask-Migrate==4.0.7 Flask-Login==0.6.3 Flask-WTF==1.2.1 Flask-Babel==4.0.0 Flask-Limiter==3.8.0 SQLAlchemy==2.0.34 alembic==1.13.2 argon2-cffi==23.1.0 email-validator==2.2.0 itsdangerous==2.2.0 python-dotenv==1.0.1 redis==5.0.8 requests==2.32.3 PyMySQL==1.1.1 gunicorn==23.0.0 Pillow==10.4.0 ``` --- ## 3. Architecture ``` Internet │ 443 (TLS via Certbot) Nginx ├─ /static/ → app/static/ (30d cache) ├─ /media/ → instance/media/ (7d cache, Phase 2+) └─ / → Gunicorn unix socket → Flask app factory │ ┌─────────────────────────┼──────────────┐ MySQL 8.0 Redis Stripe (utf8mb4) (sessions, cache, (webhooks, rate-limit, RQ) billing) │ SMTP relay (Brevo) ``` **ProxyFix** is applied in the app factory (`x_for=1, x_proto=1, x_host=1`) so `request.is_secure`, secure cookies, and rate-limit IP are all correct behind Nginx. **systemd units (in `deploy/`):** - `classifieds.service` — Gunicorn web process - `classifieds-expire.service` + `classifieds-expire.timer` — hourly expiry sweep --- ## 4. Project File Structure ``` classifieds/ ├── wsgi.py # WSGI entry point for Gunicorn + flask CLI ├── seed.py # Seed plans, categories, zip_geo; --admin flag ├── babel.cfg # pybabel extraction config ├── requirements.txt ├── .env.example # All env vars documented (no inline comments) ├── .gitignore ├── CLAUDE.md # This file ├── README.md # Setup, run, deploy instructions │ ├── app/ │ ├── __init__.py # App factory: create_app() │ ├── config.py # BaseConfig / DevConfig / ProdConfig │ ├── extensions.py # db, migrate, login_manager, csrf, babel, limiter │ │ │ ├── models/ │ │ ├── __init__.py # Exports all models (import order matters) │ │ ├── enums.py # Role, UserStatus, TrustTier, TrustEventType, │ │ │ # ListingStatus, Lang │ │ ├── plan.py # Plan (tier config JSON) │ │ ├── user.py # User (Argon2, RBAC helpers, Flask-Login) │ │ ├── trust.py # TrustEvent (append-only trust ledger) │ │ ├── category.py # Category (self-ref subcategories, field_schema) │ │ ├── listing.py # Listing + ListingImage │ │ ├── geo.py # ZipGeo + Metro │ │ ├── messaging.py # Conversation + Message │ │ └── favorite.py # Favorite │ │ │ ├── blueprints/ │ │ ├── auth/ # register, login, logout, verify-email, reset │ │ ├── main/ # index, /healthz │ │ ├── i18n/ # /lang/ locale switcher │ │ ├── listings/ # browse, detail, create, edit, delete, images │ │ └── messaging/ # inbox, conversation, start, favorites, /api/unread │ │ │ ├── services/ │ │ ├── email.py # send_email (SMTP or dev console) │ │ ├── turnstile.py # Cloudflare Turnstile verify │ │ ├── trust.py # record_event, trust tier computation │ │ ├── geo.py # geocode_zip, haversine_mi, bounding_box │ │ ├── field_schema.py # validate_attributes, hot_values │ │ ├── images.py # process_upload, delete_image_files │ │ ├── listings.py # create/update/browse/search/radius/expiry │ │ ├── messaging.py # conversations, send, mark-read, inbox, unread │ │ ├── favorites.py # toggle, is_favorited, user_favorites │ │ └── contact.py # mask_body, contact_revealed, contact_density │ │ │ ├── utils/ │ │ ├── __init__.py # RBAC decorators: role_required, admin_required, │ │ │ # moderator_required │ │ ├── security.py # hash_password, verify_password, generate_token, │ │ │ # read_token (itsdangerous) │ │ └── text.py # normalize() — accent-insensitive (phở→pho, ñ→n) │ │ │ ├── templates/ │ │ ├── base.html # Layout: nav (Browse/Post/My listings/Saved/ │ │ │ # Messages+badge/Sign out), lang switcher, flashes │ │ ├── index.html # Landing page │ │ ├── auth/ # login, register, reset_request, reset, _macros │ │ ├── listings/ # browse, detail, form, mine │ │ ├── messaging/ # inbox, conversation, start, favorites │ │ └── errors/ # 403, 404, 500 │ │ │ ├── static/ │ │ └── style.css # Single CSS file (Phase 1+2+3 accumulated) │ │ │ └── translations/ # Flask-Babel .po/.mo for vi + es │ ├── migrations/ # Alembic migration scripts (flask db migrate/upgrade) │ ├── deploy/ │ ├── classifieds.service # systemd: Gunicorn web │ ├── classifieds-expire.service # systemd: oneshot expiry sweep │ ├── classifieds-expire.timer # systemd: hourly timer │ ├── gunicorn.conf.py # workers, socket, timeouts │ └── nginx.conf.sample # HTTPS redirect + proxy + static + media blocks │ └── tests/ └── test_smoke.py # 34-check integration test (SQLite + in-memory Redis) ``` --- ## 5. Route Map (24 routes, Phase 1–3) | Method | Path | Blueprint | Auth | |---|---|---|---| | GET | `/` | main | — | | GET | `/healthz` | main | — | | GET/POST | `/auth/register` | auth | — | | GET/POST | `/auth/login` | auth | — | | GET | `/auth/logout` | auth | login | | GET | `/auth/verify/` | auth | — | | GET/POST | `/auth/reset` | auth | — | | GET/POST | `/auth/reset/` | auth | — | | GET | `/lang/` | i18n | — | | GET | `/listings` | listings | — | | GET | `/listings/new` | listings | login | | GET | `/listings/` | listings | — | | GET/POST | `/listings//edit` | listings | login+owner | | POST | `/listings//delete` | listings | login+owner | | POST | `/listings//sold` | listings | login+owner | | POST | `/listings//images//delete` | listings | login+owner | | GET/POST | `/listings//contact` | messaging | login | | POST | `/listings//favorite` | messaging | login | | GET | `/media/` | listings | — | | GET | `/my/listings` | listings | login | | GET | `/my/favorites` | messaging | login | | GET | `/messages` | messaging | login | | GET/POST | `/messages/` | messaging | login+participant | | GET | `/api/unread` | messaging | login | --- ## 6. User Roles & RBAC | Role | Capabilities | |---|---| | **free** | Register, post (limited), message, buy boosts, sees ads | | **subscriber** | Tier features (Basic/Pro/Business), ad-free | | **moderator** | Flag queue, hide/remove listings, ban users — no billing | | **admin** | Full control | RBAC enforced via decorators in `app/utils/__init__.py`: - `@role_required('admin')`, `@admin_required`, `@moderator_required` - Flask-Login `@login_required` for authenticated-only routes - Ownership checks inline in route handlers (owner or moderator) --- ## 7. Subscription Tiers Limits stored in `plans.config` JSON (admin-editable without redeploy). Enforcement reads from plan at runtime via `plan.limit(key, default)`. | Feature | Free | Basic | Pro | Business | |---|---|---|---|---| | Active listings | 3 | 15 | 50 | Unlimited | | Listing life (days) | 14 | 30 | 60 | 90 | | Images / listing | 3 | 8 | 15 | 25 | | Featured slots / mo | 0 | 1 | 5 | 20 | | Auto-bump | — | — | Weekly | Daily | | Analytics | — | Basic | Full | Full + export | | Storefront page | — | — | ✓ | ✓ + custom URL | | Verified badge | — | — | ✓ | ✓ | | Ad-free | — | ✓ | ✓ | ✓ | | Scheduled posting | — | — | ✓ | ✓ | | Bulk CSV upload | — | — | — | ✓ | | Priority support | — | — | — | ✓ | Plan slugs: `free`, `basic`, `pro`, `business`. Seeded by `seed.py`. --- ## 8. Monetization Streams 1. **Subscriptions** — recurring via Stripe Billing (Phase 4) 2. **À la carte boosts** — featured, bump, highlight, urgent — Stripe Payment Intents (Phase 4) 3. **Banner ads** — internal ad server, hidden for subscribers (Phase 5) 4. **Sponsors** — directory + category sponsorship (Phase 5) 5. **Promoted search** — keyword-pinned results (Phase 5+) --- ## 9. Database Schema **All tables:** MySQL 8.0, `utf8mb4`, UTC datetimes, `BIGINT` PKs with `BigInteger().with_variant(Integer, "sqlite"), autoincrement=True` for SQLite test compatibility. **Money:** integer cents always. Never floats for currency. ### Identity & Billing ``` users id, email (unique), password_hash, display_name, role ENUM(free,subscriber,moderator,admin), status ENUM(active,suspended,banned), locale(5), tier_id→plans, trust_score INT, trust_tier ENUM, verified BOOL, email_verified BOOL, last_login_at, created_at, updated_at plans id, slug (unique), name, price_monthly_cents, stripe_price_id, config JSON, is_active, sort_order, created_at, updated_at trust_events id, user_id→users, type ENUM(account_age,listing_survived, flag_received,verified_email,payment), delta INT, created_at ``` ### Catalog ``` categories id, slug (unique), name, parent_id→categories (nullable), field_schema JSON, icon, sort_order, sponsor_id (nullable, FK wired Phase 5), is_active, created_at, updated_at listings id, user_id→users, category_id→categories, title(140), title_norm(140) [indexed, accent-stripped], body TEXT, lang ENUM(en,vi,es), price_cents INT nullable, zip(12), city(80), state(2), lat FLOAT, lng FLOAT, attributes JSON, attr_condition(40) [indexed], attr_job_type(40) [indexed], attr_salary_min INT [indexed], attr_salary_max INT [indexed], status ENUM(active,flagged,sold,expired,removed), is_featured BOOL, bump_at DATETIME, flag_count INT, view_count INT, expires_at DATETIME [indexed], created_at, updated_at Indexes: ix_listing_browse(status,expires_at), ix_listing_sort(is_featured,bump_at) listing_images id, listing_id→listings, path(255), thumb_path(255), sort_order, width, height, created_at ``` ### Geo ``` zip_geo zip PK, city, state [indexed], lat, lng, metro [indexed] (seed ~42k rows from SimpleMaps/Census ZCTA for prod) metros id, slug (unique), name, state, center_lat, center_lng (SEO landing page anchor — Phase 7) ``` ### Messaging & Social ``` conversations id, listing_id→listings, buyer_id→users, seller_id→users, last_message_at [indexed], created_at UNIQUE(listing_id, buyer_id) messages id, conversation_id→conversations, sender_id→users, body TEXT, read_at DATETIME nullable, created_at favorites id, user_id→users, listing_id→listings, created_at UNIQUE(user_id, listing_id) ``` ### Monetization & Ops (Phase 4–6) ``` subscriptions id, user_id→users, plan_id→plans, stripe_customer_id, stripe_sub_id, status ENUM(active,past_due,canceled,trialing), current_period_end, cancel_at_period_end BOOL, created_at, updated_at transactions id, user_id, type ENUM(subscription,boost,refund), amount_cents, currency, stripe_object_id, status, meta JSON, created_at ads id, advertiser_name, slot ENUM(header,sidebar,inline,footer), creative_path, target_url, lang nullable, geo_state nullable, starts_at, ends_at, impressions, clicks, is_active, created_at sponsors id, name, logo_path, url, tier ENUM(directory,category), category_id→categories nullable, starts_at, ends_at, is_active boosts id, listing_id→listings, user_id→users, type ENUM(featured,bump,highlight,urgent), expires_at, transaction_id→transactions, created_at reports id, listing_id→listings, reporter_id→users, reason ENUM(spam,scam,offensive,duplicate,miscategorized,other), note, created_at UNIQUE(reporter_id, listing_id) audit_log id, actor_id→users, action, target_type, target_id, meta JSON, created_at settings key PK, value JSON (site-wide toggles: registration_open, ads_enabled, maintenance_mode, flag_threshold, etc.) listing_translations id, listing_id→listings, lang, title, body, cached_at (Phase 2 deferred: auto-translate cache) ``` --- ## 10. Key Services (implemented) ### `services/listings.py` - `create_listing(user, category, ...)` — tier-enforced active-listing cap, ZIP geocode, accent-norm `title_norm`, hot-column denormalization, `expires_at` from plan life - `update_listing(listing, category, ...)` — same validations, in-place update - `browse_query(...)` — SQLAlchemy query with category/state/price/condition/job_type filters and accent-insensitive keyword on `title_norm LIKE` - `search_with_radius(base_query, lat, lng, radius_mi)` — bounding-box SQL prefilter + exact haversine refine, returns `[(listing, distance_mi), ...]` - `expire_due_listings()` — flips active listings past `expires_at` to expired; called by `flask expire-listings` CLI + hourly systemd timer - `can_create(user)`, `active_count(user)`, `image_cap(user)` — tier enforcement helpers ### `services/geo.py` - `geocode_zip(zip)` — offline lookup from `zip_geo` table → `(lat, lng, city, state, metro)` - `haversine_mi(lat1, lng1, lat2, lng2)` — exact distance - `bounding_box(lat, lng, radius_mi)` → `(min_lat, max_lat, min_lng, max_lng)` ### `services/field_schema.py` - `validate_attributes(category, raw_dict)` → `(cleaned, errors)` — validates against category `field_schema`, coerces types (text/number/select/bool), enforces required - `hot_values(cleaned)` → dict of `attr_*` column values for denormalization ### `services/images.py` - `process_upload(file_storage, listing_id, sort_order)` — validates MIME, re-encodes to JPEG (strips all EXIF), resizes to 1600px max, generates 400px thumbnail, randomized filename → returns unsaved `ListingImage` - `delete_image_files(image)` — removes full + thumb from disk ### `services/messaging.py` - `get_or_create_conversation(listing, buyer)` — idempotent; blocks seller self-message - `send_message(conv, sender, body)` — validates participant, heuristic contact-density check (≥3 signals → warning logged), saves `Message`, updates `last_message_at`, sends notification email to recipient - `mark_conversation_read(conv, reader)` — marks all other-party messages read - `inbox(user, page, per_page)` — paginated conversations (buyer or seller), newest first - `total_unread(user)` — sum of unread across all conversations; injected into every template via context processor for nav badge ### `services/contact.py` - `mask_body(text, reveal)` — replaces phones/emails/URLs with `[hidden]` when `reveal=False` - `contact_revealed(user)` — True only when `email_verified AND trust_tier >= trusted` - `contact_density(text)` — count of contact signals (used for auto-flag heuristic) ### `services/trust.py` - `record_event(user, event_type, delta)` — appends `TrustEvent`, updates `trust_score`, recomputes `trust_tier` (new=0, basic=5, trusted=20, verified=50) ### `services/favorites.py` - `toggle_favorite(user_id, listing_id)` → bool (now favorited) - `is_favorited(user_id, listing_id)` → bool - `user_favorites(user_id, page, per_page)` → pagination --- ## 11. i18n (Trilingual: EN / VI / ES) Two separate layers — never conflate: **A. UI chrome** — Flask-Babel. All user-facing strings wrapped in `_()` / `lazy_gettext()`. Locale resolution: `session['lang']` → `user.locale` → `Accept-Language` header → default `en`. ```bash pybabel extract -F babel.cfg -o messages.pot . pybabel init -i messages.pot -d app/translations -l vi pybabel init -i messages.pot -d app/translations -l es # translate .po files ... pybabel compile -d app/translations ``` **B. User content** — stored as-is, `listings.lang` declared. Phase 2 deferred: auto-translate button → DeepL/Google → `listing_translations` cache. **Accent-insensitive search** (`utils/text.py`): ```python normalize("Phở Bò Đặc Biệt") == "pho bo dac biet" normalize("Ñandú Jalapeño") == "nandu jalapeno" ``` Vietnamese `đ/Đ` handled explicitly (NFKD decomposition misses it). Shadow column `title_norm` stores the result; search queries normalize the input the same way. --- ## 12. Geography & Location - `zip_geo` table: offline US ZIP → lat/lng/city/state/metro. - **Seed sample:** 10 rows (Westminster CA, Garden Grove CA, San Jose CA, Houston TX, San Antonio TX, Miami FL, Hawthorne CA, Falls Church VA). Replace with full SimpleMaps/Census ZCTA (~42k rows) before production. - On listing save: ZIP → lat/lng stored on listing. - Browse: state filter (exact), keyword, price, hot-field filters. - Radius search: bounding-box SQL prefilter → haversine exact refine. - UI: near-me (browser geolocation) + manual ZIP + radius selector. - `metros` table wired; SEO landing pages built in Phase 7. **MySQL spatial upgrade** (optional, post-Phase-2): ```sql ALTER TABLE listings ADD COLUMN geo POINT GENERATED ALWAYS AS (ST_SRID(POINT(lng, lat), 4326)) STORED, ADD SPATIAL INDEX spx_listings_geo (geo); ``` --- ## 13. Anti-Abuse (post-and-flag model) Listings go **active immediately** — no pre-approval gate by default. **Status lifecycle:** `active` → `flagged` / `sold` / `expired` / `removed` Controls in place (Phase 1–3): - **Trust tier** — gates contact-info reveal, links in messages visible only to trusted+ - **Rate limits** — Flask-Limiter + Redis: register (10/hr), login (20/hr), post (30/hr), message start (20/hr) - **Contact density heuristic** — ≥3 phone/email/URL signals in a message body → logged warning - **Turnstile CAPTCHA** on register + post (bypassed in dev when keys blank) - **Duplicate email** blocked at register **Phase 6 additions (admin queue):** flag threshold N → auto-flip to `flagged`, keyword blocklist, `reports` table workflow, `audit_log` for all mod actions. --- ## 14. Security - Argon2 password hashing (`argon2-cffi`) - CSRF on all forms (Flask-WTF) - RBAC decorators on every protected route - Rate limiting (Flask-Limiter + Redis) on auth + post endpoints - Image upload: MIME sniff, Pillow re-encode (strips EXIF/metadata), max 8MB, max 1600px, randomized filenames - Cloudflare Turnstile CAPTCHA on register + post - Contact masking: phone/email/URL hidden for low-trust users - Signed time-limited tokens for email verify + password reset (itsdangerous) - ProxyFix for correct `is_secure` + client-IP behind Nginx - Secrets via `.env` only — never committed - HTTPS-only in prod; secure + httponly + samesite=Lax cookies - `SESSION_COOKIE_SECURE=True` in ProdConfig **Stripe (Phase 4):** - Webhook signature verification on every event - Stripe = source of truth; local DB mirrors via webhooks + nightly reconcile --- ## 15. Environment Variables (`.env`) All comments must be on their own lines — no inline `# comments` after values (python-dotenv does NOT strip them, causing `int()` parse errors). ``` FLASK_CONFIG=prod SECRET_KEY= SERVER_NAME=classifieds.example.com # Database DB_USER=classifieds DB_PASSWORD= DB_HOST=127.0.0.1 DB_PORT=3306 DB_NAME=classifieds DATABASE_URL= # leave DATABASE_URL blank; URI is built from DB_* parts # Redis REDIS_URL=redis://127.0.0.1:6379/0 # i18n DEFAULT_LOCALE=en SUPPORTED_LOCALES=en,vi,es # Email (Brevo SMTP relay) MAIL_SERVER=smtp-relay.brevo.com MAIL_PORT=587 MAIL_USE_TLS=true MAIL_USERNAME= MAIL_PASSWORD= MAIL_FROM=no-reply@example.com MAIL_FROM_NAME=Classifieds # Media MEDIA_ROOT= # blank = instance/media (Flask instance folder) # Turnstile (leave blank in dev to bypass) TURNSTILE_SITE_KEY= TURNSTILE_SECRET_KEY= # Token lifetimes TOKEN_VERIFY_MAX_AGE=86400 TOKEN_RESET_MAX_AGE=3600 ``` --- ## 16. Setup & Run ### Smoke test (no MySQL/Redis needed) ```bash python3 -m venv venv && source venv/bin/activate pip install -r requirements.txt python -m tests.test_smoke # 34 checks, all green ``` ### Dev server (absolute SQLite path — important) Flask-SQLAlchemy resolves `sqlite:///dev.db` against `instance/`, not CWD. Use an absolute path to avoid ghost-schema issues: ``` DATABASE_URL=sqlite:////tmp/classifieds_dev.db ``` ```bash export FLASK_APP=wsgi:app flask db init flask db migrate -m "initial schema" flask db upgrade python seed.py --admin admin@example.com 'StrongPass123' flask run ``` ### MySQL 8.0 setup ```sql CREATE DATABASE classifieds CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; CREATE USER 'classifieds'@'localhost' IDENTIFIED BY 'password'; CREATE USER 'classifieds'@'127.0.0.1' IDENTIFIED BY 'password'; GRANT ALL PRIVILEGES ON classifieds.* TO 'classifieds'@'localhost'; GRANT ALL PRIVILEGES ON classifieds.* TO 'classifieds'@'127.0.0.1'; FLUSH PRIVILEGES; ``` Both `@'localhost'` and `@'127.0.0.1'` are required — MySQL treats them as different accounts. PyMySQL connects TCP (127.0.0.1) but MySQL may resolve to localhost. If you get `caching_sha2_password` auth errors: ```bash pip install cryptography ``` Or create user with `IDENTIFIED WITH mysql_native_password BY '...'`. ### Production deploy ```bash # 1. deploy code to /opt/classifieds (or /home/classifieds/classifieds) # 2. python3 -m venv venv && pip install -r requirements.txt # 3. cp .env.example .env && edit .env (no inline comments!) # 4. flask db upgrade && python seed.py # 5. cp deploy/classifieds.service /etc/systemd/system/ # 6. cp deploy/classifieds-expire.{service,timer} /etc/systemd/system/ # 7. systemctl enable --now classifieds classifieds-expire.timer # 8. adapt deploy/nginx.conf.sample → /etc/nginx/sites-available/classifieds # 9. ln -s /etc/nginx/sites-available/classifieds /etc/nginx/sites-enabled/ # 10. nginx -t && systemctl reload nginx # 11. certbot --nginx -d classifieds.example.com ``` **Nginx common issues:** - Config in `sites-available` but NOT symlinked to `sites-enabled` → requests fall through to default_server → wrong page served - `curl -H "Host: classifieds.example.com" http://127.0.0.1/ -I` tests Nginx config independently of DNS - DNS A record must point to the server's public IP before Let's Encrypt works --- ## 17. Coding Conventions - App factory pattern. No global `app`. Extensions in `extensions.py`. - Blueprints per domain. **Routes thin; all business logic in `services/`.** - SQLAlchemy models, Alembic migrations for every schema change. Never hand-edit prod schema. Never use `db.create_all()` in production. - All datetimes stored UTC. Localize only at render. - Money: integer cents everywhere. Never `float` for currency. - Enums: DB `ENUM` columns + Python `str, enum.Enum` classes in `models/enums.py`. - `db.session.get(Model, pk)` — not the legacy `Model.query.get(pk)` (SA 2.0 deprecated). - Validate `listings.attributes` against category `field_schema` server-side on every create/update. Never trust client-submitted attribute values. - i18n: wrap ALL user-facing UI strings in `_()`. No hardcoded English in templates. - Env-driven config. No secrets in code or VCS. - CSRF token on every state-changing form (Flask-WTF handles automatically). For inline POST forms without a WTForms object, use `{{ csrf_token() }}`. - `merge_query(**overrides)` context helper for pagination links (not a custom Jinja filter). --- ## 18. Build Roadmap & To-Do List ### ✅ Phase 1 — Foundation (Done) - [x] App factory, Dev/Prod config, env-driven settings - [x] Extensions: SQLAlchemy, Migrate, LoginManager, CSRF, Babel, Limiter - [x] ProxyFix for Nginx X-Forwarded headers - [x] `users`, `plans`, `trust_events` schema + Alembic migrations - [x] Argon2 password hashing, itsdangerous signed tokens - [x] Auth flows: register, email verify (+5 trust), login, logout, password reset - [x] Rate limiting on auth endpoints (Flask-Limiter + Redis) - [x] RBAC decorators: `role_required`, `admin_required`, `moderator_required` - [x] Trilingual i18n scaffold (EN/VI/ES), locale switcher `/lang/` - [x] Accent-insensitive normalizer `utils/text.py` (phở→pho, ñ→n) - [x] Cloudflare Turnstile CAPTCHA hook (dev bypass when keys blank) - [x] `seed.py`: 4 plans with JSON limit configs + `--admin` flag - [x] systemd + Gunicorn + Nginx deploy files - [x] Smoke test: 14 checks --- ### ✅ Phase 2 — Listings Core (Done) - [x] `categories` model (self-referential subcategories, `field_schema` JSON) - [x] `listings` + `listing_images` models (title_norm, lat/lng, hot columns, status, expiry) - [x] `zip_geo` + `metros` models - [x] 6 categories + subcategories seeded with field schemas - [x] 10 sample ZIP rows seeded (Westminster, Houston, San Jose, Miami, Falls Church…) - [x] Field schema validation service (text/number/select/bool, required, coerce) - [x] Hot-column denormalization (attr_condition, attr_job_type, attr_salary_*) - [x] Listing CRUD routes: create, edit, delete, mark-sold - [x] Tier enforcement: active-listing cap, image cap, listing life from plan config - [x] Image pipeline: MIME validate, Pillow re-encode to JPEG, strip EXIF, thumbnail, random filenames - [x] Browse + filter: category, state, price range, condition, keyword - [x] Accent-insensitive keyword search via `title_norm LIKE` - [x] Radius search: bounding-box SQL prefilter + exact haversine refine - [x] View counter on listing detail - [x] My-listings page with active/cap display - [x] `flask expire-listings` CLI + systemd timer (hourly sweep) - [x] Dev media serving route `/media/` - [x] Smoke test: +6 checks (geocode, tier limit, attr validation, search, radius, image, expiry) --- ### ✅ Phase 3 — Messaging + Favorites (Done) - [x] `conversations` model (unique per listing+buyer, seller auto-set) - [x] `messages` model (append-only, read_at tracking) - [x] `favorites` model (unique user+listing) - [x] Messaging service: get-or-create conversation, send, mark-read, inbox, total-unread - [x] Self-message block (seller cannot contact own listing) - [x] Contact density heuristic (≥3 phone/email/URL signals → warning log) - [x] Contact masking service: mask phone/email/URL for low-trust users - [x] Trust-gated contact reveal (email_verified AND trust_tier ≥ trusted) - [x] Notification email to recipient on every new message - [x] Favorites service: toggle, is_favorited, paginated user favorites - [x] Inbox route with unread badge per conversation - [x] Conversation thread route (GET masked, POST send reply) - [x] Start-conversation from listing detail - [x] Toggle-favorite endpoint (AJAX-capable + form fallback) - [x] My-favorites page (saved listings grid) - [x] `/api/unread` JSON endpoint - [x] Unread count injected into every page via context processor (nav badge) - [x] Nav: Messages + Saved links added - [x] Listing detail: Contact seller + ♥ Save listing buttons for non-owners - [x] Smoke test: +15 checks (conversation, messages, unread, masking, favorites, routes) --- ### 🔲 Phase 4 — Monetization / Stripe (Next) **Models** - [ ] `subscriptions` table (stripe_customer_id, stripe_sub_id, status, period_end, cancel_at_period_end) - [ ] `transactions` table (type: subscription/boost/refund, amount_cents, stripe_object_id) - [ ] `boosts` table (listing_id, type: featured/bump/highlight/urgent, expires_at) - [ ] Alembic migration for all three tables **Stripe setup** - [ ] `stripe` pip package added to requirements.txt - [ ] Stripe keys in `.env` (`STRIPE_SECRET_KEY`, `STRIPE_PUBLISHABLE_KEY`, `STRIPE_WEBHOOK_SECRET`) - [ ] Create Products + Prices in Stripe dashboard for Basic/Pro/Business plans - [ ] Map `stripe_price_id` in `plans` table (via seed or admin) - [ ] Create boost Products in Stripe (featured, bump, highlight, urgent) + prices **Subscriptions** - [ ] `payments` blueprint: `/billing/subscribe/` → Stripe Checkout Session - [ ] Redirect to Stripe-hosted checkout, success/cancel return URLs - [ ] Stripe Customer Portal route `/billing/portal` (self-serve upgrade/cancel/update card) - [ ] Webhook endpoint `/billing/webhook` — verify signature on every event - [ ] Handle `checkout.session.completed` → create/update `subscriptions` row, set `users.role=subscriber`, assign `tier_id` - [ ] Handle `customer.subscription.updated` → sync status, period_end, plan change - [ ] Handle `customer.subscription.deleted` → downgrade to free plan, clear tier - [ ] Handle `invoice.payment_failed` → flip subscription status to `past_due`, email user - [ ] Nightly reconcile job: compare local subscription status vs Stripe API (catch missed webhooks) - [ ] Stripe Tax enabled on checkout (auto US sales tax) **À la carte boosts (free + paid users)** - [ ] Boost purchase route `/listings//boost` → Stripe Payment Intent (one-off) - [ ] Boost type selector UI (featured / bump / highlight / urgent + price display) - [ ] On payment success: write `boosts` row with `expires_at`, flip `listing.is_featured` if featured boost - [ ] Boost expiry sweep added to `expire-listings` worker (clear expired boosts) - [ ] Browse query: order featured (boosted) listings first **Tier enforcement upgrades** - [ ] Enforce `featured_per_month` limit from plan config (count active featured boosts) - [ ] Enforce `scheduled_posting` — gate date-picker on plan check - [ ] Enforce `ad_free` — suppress ads for subscriber role - [ ] Upgrade prompt shown when free user hits any limit **UI** - [ ] Pricing page `/pricing` showing plan comparison table - [ ] Account billing page `/my/billing` (current plan, next renewal, manage button → portal) - [ ] Boost buttons on listing detail (owner-only) and my-listings page - [ ] Payment success / cancel flash pages **Tests** - [ ] Smoke test: Stripe webhook handler with test payload (mock signature) - [ ] Smoke test: boost creation writes correct `boosts` row - [ ] Smoke test: subscription downgrade clears tier --- ### 🔲 Phase 5 — Ads & Sponsors **Models** - [ ] `ads` table (slot, creative_path, target_url, lang, geo_state, schedule, impressions, clicks) - [ ] `sponsors` table (tier: directory/category, category_id, schedule) - [ ] Alembic migration **Ad server** - [ ] Ad service: `get_ad_for_slot(slot, lang, state)` — picks active ad matching targeting, returns one - [ ] Impression tracking: increment `ads.impressions` on serve (async-safe, batched to Redis then flush) - [ ] Click tracking: redirect endpoint `/ads//click` → increment `ads.clicks` → redirect to `target_url` - [ ] Ad slots in templates: header banner, sidebar (browse page), inline (every 6th listing card), footer - [ ] Ads suppressed for `role=subscriber` (check `plan.limit('ad_free')`) - [ ] Ad creative upload (admin): validate image, Pillow re-encode, store in `/media/ads/` **Sponsors** - [ ] Sponsor directory page `/sponsors` - [ ] Category sponsorship: "Jobs powered by X" banner on category browse - [ ] Sponsor logo in nav or footer (tier: directory) - [ ] Sponsor admin CRUD **Promoted search** - [ ] `promoted_keywords` table (keyword, listing_id, expires_at, priority) - [ ] Browse query: prepend promoted listings matching keyword before organic results - [ ] Admin UI to assign promoted keyword slots **Tests** - [ ] Smoke: ad served for correct slot/lang/state targeting - [ ] Smoke: click redirect increments counter - [ ] Smoke: subscriber sees no ads --- ### 🔲 Phase 6 — Admin Backend **Dashboard** - [ ] `/admin` dashboard: KPI cards (active listings, new users 7d/30d, MRR, ad revenue, flag-queue depth) - [ ] Mini charts: signups/day, listings/day (last 30d), revenue trend **User management** - [ ] `/admin/users` — searchable/filterable table (role, status, tier, trust) - [ ] User detail page: profile, listing history, subscription, trust events - [ ] Actions: ban/suspend/activate, tier override, trust score adjust, impersonate (logs to audit_log) - [ ] Bulk actions: ban selected, send email to selected **Listing moderation** - [ ] `/admin/listings` — flag queue sorted by `flag_count × recency` - [ ] Quick actions per listing: approve (clear flags), hide (flagged), remove, view - [ ] Bulk approve / bulk remove - [ ] Auto-flag threshold: N distinct-user flags → auto-flip to `flagged` (setting in `settings` table) - [ ] Keyword blocklist editor (stored in `settings`, checked on listing submit) - [ ] Duplicate body detection (hash `body` on submit, reject/flag if seen within 24h) **Reports queue** - [ ] `/admin/reports` — flagged content with reporter reasons - [ ] Mark resolved / escalate actions - [ ] `reports` table: unique per reporter+listing, reason enum **Category management** - [ ] `/admin/categories` — CRUD, reorder (drag or sort_order field) - [ ] Field schema editor per category (JSON form builder: add/remove fields, set type/required/options) - [ ] Preview of field schema as it would appear on listing form **Plan / pricing management** - [ ] `/admin/plans` — edit plan config JSON (limits), name, price, Stripe price ID, toggle active - [ ] No-redeploy: limits read at runtime from DB **Ads & sponsors management** - [ ] `/admin/ads` — upload creative, set slot/targeting/schedule, view impression/click stats - [ ] `/admin/sponsors` — CRUD sponsor entries, assign category - [ ] Ad performance report (impressions, clicks, CTR per ad) **Transactions & billing** - [ ] `/admin/transactions` — full log with filter by type/status/date - [ ] Refund action (calls Stripe Refund API, writes refund transaction row) - [ ] Failed payments list with retry action **Settings** - [ ] `/admin/settings` — toggle UI for all `settings` table keys: - `registration_open` (bool) - `ads_enabled` (bool) - `maintenance_mode` (bool) - `flag_threshold` (int, default 5) - `new_user_trust_gate_days` (int) - `contact_density_threshold` (int) **Audit log** - [ ] All admin write actions write to `audit_log` (actor, action, target, meta JSON) - [ ] `/admin/audit` — searchable audit trail **Analytics** - [ ] Traffic: page views/day, top pages, search terms used - [ ] Conversions: registrations, listings posted, messages sent, boosts purchased - [ ] Top categories by listing count and by view count - [ ] Top metros by listing count (Phase 7 SEO planning) **Tests** - [ ] Smoke: admin dashboard 200, non-admin gets 403 - [ ] Smoke: flag threshold auto-flips listing status - [ ] Smoke: impersonate logs to audit_log --- ### 🔲 Phase 7 — Polish **SEO & discoverability** - [ ] Metro landing pages `/classifieds/` (e.g. `/classifieds/orange-county`) - [ ] State landing pages `/classifieds/state/` - [ ] Category landing pages `/classifieds/category/` - [ ] Dynamic `` and `<meta description>` on all pages - [ ] JSON-LD structured data on listing detail (Product schema) - [ ] XML sitemap (`/sitemap.xml`) — listings + categories + metros, auto-updated - [ ] `robots.txt` - [ ] Open Graph tags (listing title, price, cover image) for social sharing **Email notifications** - [ ] Welcome email on registration (with verify link already sent — add branding) - [ ] Listing expiry warning email (3 days before `expires_at`) - [ ] Listing expired email (with renew CTA) - [ ] Message received email (already inline — move to RQ worker for async) - [ ] Payment failed email - [ ] Weekly digest email (new listings in saved categories/metro) — opt-in **Reviews** - [ ] `reviews` table (listing_id, author_id, rating TINYINT, body, created_at) - [ ] Leave review on completed transaction (mark-sold triggers prompt) - [ ] Seller aggregate rating on profile + listing detail - [ ] Moderation: flag/remove abusive reviews **Translation (deferred from Phase 2)** - [ ] Per-listing "Translate" button → DeepL/Google Translate API - [ ] Cache result in `listing_translations` (pay once per listing+lang) - [ ] Language filter on browse ("Show VI listings only") **Performance** - [ ] Switch sessions to Redis (`SESSION_TYPE=redis`) - [ ] Query-level caching for hot browse queries (Redis, 60s TTL) - [ ] Lazy-load listing images (native `loading="lazy"`) - [ ] Serve WebP thumbnails (Pillow WebP encode alongside JPEG) - [ ] Add MySQL `FULLTEXT(title, body)` index + switch keyword search to `MATCH ... AGAINST` - [ ] Pagination `LIMIT/OFFSET` → keyset pagination for large datasets **UX & mobile** - [ ] Responsive nav (hamburger menu on mobile) - [ ] Listing detail image gallery with lightbox - [ ] Infinite scroll OR "Load more" on browse (AJAX pagination) - [ ] Toast notifications (non-blocking flash messages) - [ ] "Back to results" link on listing detail (preserve filter state) - [ ] Listing preview before publish **i18n completion** - [ ] Extract all `_()` strings to `.pot` file - [ ] Translate all strings to Vietnamese (`vi`) - [ ] Translate all strings to Spanish (`es`) - [ ] Compile `.mo` files, test all three locales end-to-end **Tests** - [ ] Smoke: sitemap returns valid XML with listing URLs - [ ] Smoke: OG tags present on listing detail - [ ] Smoke: translation cached correctly in `listing_translations` --- ### 🔲 Phase 8 — JSON API (Optional, for iOS app) **Design** - [ ] RESTful JSON API under `/api/v1/` - [ ] JWT authentication (separate from session cookies) - [ ] API versioning strategy documented **Endpoints** - [ ] `POST /api/v1/auth/register` - [ ] `POST /api/v1/auth/login` → returns JWT - [ ] `GET /api/v1/listings` (browse + all filters, returns paginated JSON) - [ ] `GET /api/v1/listings/<id>` - [ ] `POST /api/v1/listings` (create, multipart for images) - [ ] `PUT /api/v1/listings/<id>` (edit) - [ ] `DELETE /api/v1/listings/<id>` - [ ] `GET /api/v1/messages` (inbox) - [ ] `GET /api/v1/messages/<conv_id>` - [ ] `POST /api/v1/messages/<conv_id>` (send) - [ ] `POST /api/v1/listings/<id>/favorite` - [ ] `GET /api/v1/categories` - [ ] `GET /api/v1/zip/<zip>` (geocode lookup) - [ ] `GET /api/v1/me` (profile + tier info) **Infrastructure** - [ ] JWT library added (`PyJWT` or `flask-jwt-extended`) - [ ] Rate limiting on all API endpoints (stricter than web) - [ ] API error responses: consistent `{"error": "...", "code": "..."}` JSON shape - [ ] CORS headers for iOS app origin - [ ] API smoke tests (separate from web smoke test) --- ### 🔲 Deferred Items (from earlier phases) - [ ] **Auto-translate** — per-listing translate button → DeepL/Google → `listing_translations` cache (moved to Phase 7) - [ ] **Full ZIP dataset** — replace 10-row sample with SimpleMaps US ZIP or Census ZCTA (~42k rows) - [ ] **MySQL spatial upgrade** — `POINT` generated column + `SPATIAL INDEX` + `ST_Distance_Sphere` (optional perf upgrade) - [ ] **MySQL FULLTEXT upgrade** — `FULLTEXT(title,body)` index + `MATCH ... AGAINST` (Phase 7) - [ ] **MySQL generated hot columns** — `GENERATED ALWAYS AS (JSON_EXTRACT(...))` replacing app-maintained `attr_*` columns (optional) - [ ] **RQ worker** — replace inline email sends + APScheduler with proper RQ queue + worker process - [ ] **Redis sessions** — switch from filesystem sessions to Redis-backed (`SESSION_TYPE=redis`) - [ ] **Bulk CSV listing upload** — Business tier feature (parse CSV, validate, batch-create listings) - [ ] **Scheduled posting** — Pro/Business: set `publish_at` datetime, worker flips to active - [ ] **Auto-bump/renew** — Pro (weekly) / Business (daily): worker resets `bump_at`, extends `expires_at` - [ ] **Storefront/profile page** — Pro/Business: public seller page `/seller/<username>` with all active listings - [ ] **Custom URL** — Business: vanity slug for storefront (e.g. `/shop/alices-cleaning`) - [ ] **Job-seeker posts** — reverse job listings ("I'm looking for work") under Jobs category --- ## 19. Known Issues / Decisions Locked | Item | Decision | |---|---| | DB portability | SQLite for tests (with `BigInteger().with_variant(Integer,"sqlite")` + `autoincrement=True`), MySQL 8.0 for prod. MariaDB NOT supported. | | Spatial search | lat/lng + bounding box + haversine for now. MySQL `POINT` + `SPATIAL INDEX` upgrade SQL documented in README. | | FULLTEXT search | `title_norm LIKE` for now. MySQL `FULLTEXT(title,body)` + `MATCH ... AGAINST` upgrade documented in README. | | Hot filter columns | App-maintained `attr_*` indexed columns. MySQL `GENERATED ALWAYS` upgrade documented in README. | | ZIP dataset | 10-row sample seeded. Replace with SimpleMaps US ZIP or Census ZCTA (~42k rows) before prod. | | Workers | No RQ/Celery yet. Expiry sweep via systemd timer. Message notifications inline. Phase 4+ may add RQ for async email. | | Session storage | Flask default (filesystem/cookie). Phase 4: switch to Redis sessions for scale. | | inline `.env` comments | MUST NOT have `# comments` on the same line as values — python-dotenv does not strip them, causing `int()` ValueError. Comments go on their own lines. | --- _End of spec. Phase 1–3 complete, 34 smoke-test checks green. Phases 4–8 fully detailed as to-do lists above. Next: Phase 4 (Monetization / Stripe)._