WebChecker--Web-app-/DEPLOY.md

Website Checker — Production Deployment Guide

Ubuntu Server · Nginx · Gunicorn · MySQL · systemd

---

Table of Contents

1. Prerequisites
2. Server Initial Setup
3. MySQL Database
4. Application Deployment
5. Gunicorn systemd Service
6. Nginx Configuration
7. SSL/TLS with Certbot
8. Firewall Rules
9. Post-Deployment Verification
10. Ongoing Maintenance
11. Troubleshooting

---

1. Prerequisites

Item	Requirement
OS	Ubuntu 22.04 LTS or 24.04 LTS
RAM	≥ 1 GB (2 GB recommended)
Disk	≥ 10 GB
Access	Root or sudo user
Domain	A DNS A-record pointing to your server's IP
Python	3.10+ (pre-installed on Ubuntu 22.04+)

---

2. Server Initial Setup

2.1 Update the system

sudo apt update && sudo apt upgrade -y

2.2 Install system dependencies

sudo apt install -y \
    python3 python3-pip python3-venv \
    mysql-server \
    nginx \
    certbot python3-certbot-nginx \
    git curl ufw

2.3 Create a dedicated application user

sudo useradd --system --shell /bin/bash --create-home webchecker

---

3. MySQL Database

3.1 Secure the MySQL installation

sudo mysql_secure_installation

Follow the prompts: set a root password, remove anonymous users, disallow remote root login, remove test database.

3.2 Create the database and application user

sudo mysql -u root -p

Inside the MySQL shell:

CREATE DATABASE webchecker CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

CREATE USER 'webchecker_user'@'127.0.0.1' IDENTIFIED BY 'YourStrongPasswordHere';

GRANT ALL PRIVILEGES ON webchecker.* TO 'webchecker_user'@'127.0.0.1';

FLUSH PRIVILEGES;
EXIT;

Security note: Use 127.0.0.1 (not localhost) so the connector uses TCP rather than the UNIX socket, which matches the DB_HOST=127.0.0.1 setting in .env.

---

4. Application Deployment

4.1 Transfer the application files

Option A — copy from your workstation:

scp -r ./webchecker_web/ youruser@your-server-ip:/tmp/webchecker_web

Option B — clone from a private Git repository:

sudo -u webchecker git clone https://github.com/your-org/webchecker.git /opt/webchecker

Then move files into place (Option A):

sudo mv /tmp/webchecker_web /opt/webchecker
sudo chown -R webchecker:webchecker /opt/webchecker

4.2 Create the Python virtual environment

sudo -u webchecker python3 -m venv /opt/webchecker/venv

4.3 Install Python dependencies

sudo -u webchecker /opt/webchecker/venv/bin/pip install --upgrade pip
sudo -u webchecker /opt/webchecker/venv/bin/pip install -r /opt/webchecker/requirements.txt

4.4 Configure environment variables

sudo cp /opt/webchecker/.env.example /opt/webchecker/.env
sudo nano /opt/webchecker/.env

Fill in every value — especially:

SECRET_KEY=<output of: python3 -c "import secrets; print(secrets.token_hex(32))">
DB_HOST=127.0.0.1
DB_PORT=3306
DB_NAME=webchecker
DB_USER=webchecker_user
DB_PASSWORD=YourStrongPasswordHere
FLASK_ENV=production

# AI analysis (can also be set via Admin → Settings after first login)
GROQ_API_KEY=
GROQ_MODEL=llama-3.3-70b-versatile

# SMTP email for password reset and bid reminders (can also be set via Admin → Settings)
SMTP_HOST=
SMTP_PORT=587
SMTP_USER=
SMTP_PASSWORD=
SMTP_FROM=noreply@yourdomain.com

Secure the file so only the application user can read it:

sudo chown webchecker:webchecker /opt/webchecker/.env
sudo chmod 600 /opt/webchecker/.env

4.5 Initialise the database schema

The application auto-creates all tables on first startup via initialize_database(). Run it once manually to verify:

sudo -u webchecker bash -c '
  cd /opt/webchecker
  source venv/bin/activate
  python - <<EOF
from app import create_app
app = create_app()
with app.app_context():
    from config import initialize_database
    initialize_database()
    print("Database initialised successfully.")
EOF
'

4.6 Create the Gunicorn socket directory

sudo mkdir -p /run/webchecker
sudo chown webchecker:webchecker /run/webchecker

---

5. Gunicorn systemd Service

5.1 Create the service unit file

sudo nano /etc/systemd/system/webchecker.service

Paste the following:

[Unit]
Description=Website Checker — Gunicorn Application Server
After=network.target mysql.service
Requires=mysql.service

[Service]
Type=notify
User=webchecker
Group=webchecker
WorkingDirectory=/opt/webchecker
EnvironmentFile=/opt/webchecker/.env
ExecStart=/opt/webchecker/venv/bin/gunicorn \
    --bind unix:/run/webchecker/webchecker.sock \
    --workers 4 \
    --worker-class sync \
    --timeout 120 \
    --access-logfile /var/log/webchecker/access.log \
    --error-logfile  /var/log/webchecker/error.log \
    --log-level info \
    wsgi:application
ExecReload=/bin/kill -s HUP $MAINPID
KillMode=mixed
TimeoutStopSec=10
PrivateTmp=true
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

5.2 Create the log directory

sudo mkdir -p /var/log/webchecker
sudo chown webchecker:webchecker /var/log/webchecker

5.3 Enable and start the service

sudo systemctl daemon-reload
sudo systemctl enable webchecker
sudo systemctl start webchecker
sudo systemctl status webchecker

You should see Active: active (running). The socket file /run/webchecker/webchecker.sock will be created automatically.

---

6. Nginx Configuration

6.1 Create the Nginx site configuration

sudo nano /etc/nginx/sites-available/webchecker

Paste (replace your-domain.com with your actual domain):

upstream webchecker_app {
    server unix:/run/webchecker/webchecker.sock fail_timeout=0;
}

server {
    listen 80;
    server_name your-domain.com www.your-domain.com;

    # Security headers
    add_header X-Frame-Options           "SAMEORIGIN"   always;
    add_header X-Content-Type-Options    "nosniff"      always;
    add_header X-XSS-Protection          "1; mode=block" always;
    add_header Referrer-Policy           "strict-origin-when-cross-origin" always;

    # Client upload size (raise if users upload large files for AI analysis)
    client_max_body_size 20M;

    # Static files served directly by Nginx (fast path)
    location /static/ {
        alias /opt/webchecker/static/;
        expires 30d;
        access_log off;
        add_header Cache-Control "public, immutable";
    }

    # Proxy everything else to Gunicorn
    location / {
        proxy_pass http://webchecker_app;

        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_connect_timeout 60s;
        proxy_read_timeout    120s;
        proxy_send_timeout    120s;

        proxy_buffering    on;
        proxy_buffer_size  16k;
        proxy_buffers      4 32k;
    }

    # Logging
    access_log /var/log/nginx/webchecker_access.log;
    error_log  /var/log/nginx/webchecker_error.log;
}

6.2 Enable the site

sudo ln -s /etc/nginx/sites-available/webchecker /etc/nginx/sites-enabled/
sudo nginx -t          # test config — must report "syntax is ok"
sudo systemctl reload nginx

---

7. SSL/TLS with Certbot

sudo certbot --nginx -d your-domain.com -d www.your-domain.com

Certbot will automatically:

• Obtain a Let's Encrypt certificate
• Modify the Nginx config to add HTTPS (port 443)
• Insert a redirect from HTTP → HTTPS
• Schedule automatic renewal

Verify auto-renewal works:

sudo certbot renew --dry-run

After Certbot finishes, add an HSTS header inside the server block that Certbot created for port 443:

add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;

Then reload Nginx:

sudo nginx -t && sudo systemctl reload nginx

---

8. Firewall Rules

# Allow SSH (adjust port if you changed it)
sudo ufw allow 22/tcp

# Allow HTTP and HTTPS
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Enable the firewall
sudo ufw enable

# Verify
sudo ufw status verbose

MySQL port 3306 is intentionally not opened — the application connects via 127.0.0.1 (localhost), so no external exposure is required.

---

9. Post-Deployment Verification

9.1 Check the Gunicorn service

sudo systemctl status webchecker
sudo journalctl -u webchecker -n 50 --no-pager

9.2 Check Nginx

sudo systemctl status nginx
sudo tail -f /var/log/nginx/webchecker_error.log

9.3 Smoke-test the application

curl -I https://your-domain.com/auth/login
# Expect: HTTP/2 200

Open a browser and navigate to https://your-domain.com. You should see the login page.

9.4 First login

Log in with the default admin credentials set during initialize_database(). Change the password immediately via Admin → Users (edit your user) or via My Profile → Change Password.

9.5 Configure email and AI settings

Navigate to Admin → Settings and fill in:

• Email — SMTP host, port, security mode, username, password, from address, and recipient list. Click 📬 Test to verify without leaving the page.
• Groq AI — paste your API key and select a model. Click 🔌 Test to confirm the key works.

These values are stored in the database and take precedence over .env fallbacks.

---

10. Ongoing Maintenance

Deploy a new version

# 1. Copy/pull new files to /opt/webchecker
# 2. Install any new dependencies
sudo -u webchecker /opt/webchecker/venv/bin/pip install -r /opt/webchecker/requirements.txt

# 3. Reload Gunicorn (zero-downtime — workers are replaced one by one)
sudo systemctl reload webchecker

# 4. If Nginx config changed:
sudo nginx -t && sudo systemctl reload nginx

View application logs

# Gunicorn access log
sudo tail -f /var/log/webchecker/access.log

# Gunicorn error log
sudo tail -f /var/log/webchecker/error.log

# systemd journal (includes startup errors)
sudo journalctl -u webchecker -f

Backup the database

mysqldump -u webchecker_user -p webchecker | gzip > ~/webchecker_$(date +%Y%m%d).sql.gz

Automate with a daily cron:

sudo crontab -e
# Add:
0 2 * * * mysqldump -u webchecker_user -pYourPassword webchecker | gzip > /var/backups/webchecker_$(date +\%Y\%m\%d).sql.gz

Rotate logs

Create /etc/logrotate.d/webchecker:

/var/log/webchecker/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    sharedscripts
    postrotate
        systemctl reload webchecker > /dev/null 2>&1 || true
    endscript
}

---

11. Troubleshooting

Symptom	Likely Cause	Fix
502 Bad Gateway	Gunicorn not running or socket missing	sudo systemctl restart webchecker; check journalctl -u webchecker
connect() to unix:/run/webchecker/webchecker.sock failed (13: Permission denied)	Nginx user can't read socket	Add nginx to webchecker group: sudo usermod -aG webchecker www-data, then sudo systemctl restart nginx
OperationalError: Access denied for user	Wrong DB credentials in .env	Verify DB_USER/DB_PASSWORD match what was set in MySQL
ImportError or ModuleNotFoundError	Dependency not installed in venv	Run pip install -r requirements.txt with the venv active
cryptography.fernet.InvalidToken	CRYPTO_SECRET changed after credentials were stored	Restore the original key from backup; never rotate without re-encrypting stored data
Sessions expiring immediately	SECRET_KEY changed between restarts	Keep SECRET_KEY stable in .env; never regenerate it on a live system
413 Request Entity Too Large	File upload exceeds client_max_body_size	Increase client_max_body_size in the Nginx config
Static files returning 404	Wrong alias path in Nginx	Confirm /opt/webchecker/static/ exists and the alias directive ends with /
Password reset emails not delivered	SMTP not configured or wrong credentials	Go to Admin → Settings, check SMTP fields, click 📬 Test — the error message shows the SMTP failure reason
Groq AI analysis fails with 401	Invalid or missing API key	Go to Admin → Settings → Groq, paste a valid key, click 🔌 Test to confirm before saving
BuildError for a new route after deploy	Gunicorn still running old bytecode	Run sudo systemctl reload webchecker — templates go live immediately but Python changes require a reload

---

Generated for Website Checker Web — Flask/MySQL/Nginx/Gunicorn/Ubuntu deployment.