WebChecker--Web-app-/DEPLOY.md

# Website Checker — Production Deployment Guide
### Ubuntu Server · Nginx · Gunicorn · MySQL · systemd

---

## Table of Contents

1. [Prerequisites](#1-prerequisites)
2. [Server Initial Setup](#2-server-initial-setup)
3. [MySQL Database](#3-mysql-database)
4. [Application Deployment](#4-application-deployment)
5. [Gunicorn systemd Service](#5-gunicorn-systemd-service)
6. [Nginx Configuration](#6-nginx-configuration)
7. [SSL/TLS with Certbot](#7-ssltls-with-certbot)
8. [Firewall Rules](#8-firewall-rules)
9. [Post-Deployment Verification](#9-post-deployment-verification)
10. [Ongoing Maintenance](#10-ongoing-maintenance)
11. [Troubleshooting](#11-troubleshooting)

---

## 1. Prerequisites

| Item | Requirement |
|------|-------------|
| OS   | Ubuntu 22.04 LTS or 24.04 LTS |
| RAM  | ≥ 1 GB (2 GB recommended) |
| Disk | ≥ 10 GB |
| Access | Root or sudo user |
| Domain | A DNS A-record pointing to your server's IP |
| Python | 3.10+ (pre-installed on Ubuntu 22.04+) |

---

## 2. Server Initial Setup

### 2.1 Update the system

```bash
sudo apt update && sudo apt upgrade -y
```

### 2.2 Install system dependencies

```bash
sudo apt install -y \
    python3 python3-pip python3-venv \
    mysql-server \
    nginx \
    certbot python3-certbot-nginx \
    git curl ufw
```

### 2.3 Create a dedicated application user

```bash
sudo useradd --system --shell /bin/bash --create-home webchecker
```

---

## 3. MySQL Database

### 3.1 Secure the MySQL installation

```bash
sudo mysql_secure_installation
```

Follow the prompts: set a root password, remove anonymous users, disallow remote root login, remove test database.

### 3.2 Create the database and application user

```bash
sudo mysql -u root -p
```

Inside the MySQL shell:

```sql
CREATE DATABASE webchecker CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

CREATE USER 'webchecker_user'@'127.0.0.1' IDENTIFIED BY 'YourStrongPasswordHere';

GRANT ALL PRIVILEGES ON webchecker.* TO 'webchecker_user'@'127.0.0.1';

FLUSH PRIVILEGES;
EXIT;
```

> **Security note:** Use `127.0.0.1` (not `localhost`) so the connector uses TCP rather than the UNIX socket, which matches the `DB_HOST=127.0.0.1` setting in `.env`.

---

## 4. Application Deployment

### 4.1 Transfer the application files

Option A — copy from your workstation:
```bash
scp -r ./webchecker_web/ youruser@your-server-ip:/tmp/webchecker_web
```

Option B — clone from a private Git repository:
```bash
sudo -u webchecker git clone https://github.com/your-org/webchecker.git /opt/webchecker
```

Then move files into place (Option A):
```bash
sudo mv /tmp/webchecker_web /opt/webchecker
sudo chown -R webchecker:webchecker /opt/webchecker
```

### 4.2 Create the Python virtual environment

```bash
sudo -u webchecker python3 -m venv /opt/webchecker/venv
```

### 4.3 Install Python dependencies

```bash
sudo -u webchecker /opt/webchecker/venv/bin/pip install --upgrade pip
sudo -u webchecker /opt/webchecker/venv/bin/pip install -r /opt/webchecker/requirements.txt
```

### 4.4 Configure environment variables

```bash
sudo cp /opt/webchecker/.env.example /opt/webchecker/.env
sudo nano /opt/webchecker/.env
```

Fill in every value — especially:

```dotenv
SECRET_KEY=<output of: python3 -c "import secrets; print(secrets.token_hex(32))">
DB_HOST=127.0.0.1
DB_PORT=3306
DB_NAME=webchecker
DB_USER=webchecker_user
DB_PASSWORD=YourStrongPasswordHere
CRYPTO_SECRET=<output of: python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())">
FLASK_ENV=production
```

Secure the file so only the application user can read it:

```bash
sudo chown webchecker:webchecker /opt/webchecker/.env
sudo chmod 600 /opt/webchecker/.env
```

### 4.5 Initialise the database schema

The application auto-creates all tables on first startup via `initialize_database()`. Run it once manually to verify:

```bash
sudo -u webchecker bash -c '
  cd /opt/webchecker
  source venv/bin/activate
  python - <<EOF
from app import create_app
app = create_app()
with app.app_context():
    from config import initialize_database
    initialize_database()
    print("Database initialised successfully.")
EOF
'
```

### 4.6 Create the Gunicorn socket directory

```bash
sudo mkdir -p /run/webchecker
sudo chown webchecker:webchecker /run/webchecker
```

---

## 5. Gunicorn systemd Service

### 5.1 Create the service unit file

```bash
sudo nano /etc/systemd/system/webchecker.service
```

Paste the following:

```ini
[Unit]
Description=Website Checker — Gunicorn Application Server
After=network.target mysql.service
Requires=mysql.service

[Service]
Type=notify
User=webchecker
Group=webchecker
WorkingDirectory=/opt/webchecker
EnvironmentFile=/opt/webchecker/.env
ExecStart=/opt/webchecker/venv/bin/gunicorn \
    --bind unix:/run/webchecker/webchecker.sock \
    --workers 4 \
    --worker-class sync \
    --timeout 120 \
    --access-logfile /var/log/webchecker/access.log \
    --error-logfile  /var/log/webchecker/error.log \
    --log-level info \
    wsgi:application
ExecReload=/bin/kill -s HUP $MAINPID
KillMode=mixed
TimeoutStopSec=10
PrivateTmp=true
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
```

### 5.2 Create the log directory

```bash
sudo mkdir -p /var/log/webchecker
sudo chown webchecker:webchecker /var/log/webchecker
```

### 5.3 Enable and start the service

```bash
sudo systemctl daemon-reload
sudo systemctl enable webchecker
sudo systemctl start webchecker
sudo systemctl status webchecker
```

You should see `Active: active (running)`. The socket file `/run/webchecker/webchecker.sock` will be created automatically.

---

## 6. Nginx Configuration

### 6.1 Create the Nginx site configuration

```bash
sudo nano /etc/nginx/sites-available/webchecker
```

Paste (replace `your-domain.com` with your actual domain):

```nginx
upstream webchecker_app {
    server unix:/run/webchecker/webchecker.sock fail_timeout=0;
}

server {
    listen 80;
    server_name your-domain.com www.your-domain.com;

    # Security headers
    add_header X-Frame-Options           "SAMEORIGIN"   always;
    add_header X-Content-Type-Options    "nosniff"      always;
    add_header X-XSS-Protection          "1; mode=block" always;
    add_header Referrer-Policy           "strict-origin-when-cross-origin" always;

    # Client upload size (raise if users upload large files for AI analysis)
    client_max_body_size 20M;

    # Static files served directly by Nginx (fast path)
    location /static/ {
        alias /opt/webchecker/static/;
        expires 30d;
        access_log off;
        add_header Cache-Control "public, immutable";
    }

    # Proxy everything else to Gunicorn
    location / {
        proxy_pass http://webchecker_app;

        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_connect_timeout 60s;
        proxy_read_timeout    120s;
        proxy_send_timeout    120s;

        proxy_buffering    on;
        proxy_buffer_size  16k;
        proxy_buffers      4 32k;
    }

    # Logging
    access_log /var/log/nginx/webchecker_access.log;
    error_log  /var/log/nginx/webchecker_error.log;
}
```

### 6.2 Enable the site

```bash
sudo ln -s /etc/nginx/sites-available/webchecker /etc/nginx/sites-enabled/
sudo nginx -t          # test config — must report "syntax is ok"
sudo systemctl reload nginx
```

---

## 7. SSL/TLS with Certbot

```bash
sudo certbot --nginx -d your-domain.com -d www.your-domain.com
```

Certbot will automatically:
- Obtain a Let's Encrypt certificate
- Modify the Nginx config to add HTTPS (port 443)
- Insert a redirect from HTTP → HTTPS
- Schedule automatic renewal

**Verify auto-renewal works:**

```bash
sudo certbot renew --dry-run
```

After Certbot finishes, add an HSTS header inside the `server` block that Certbot created for port 443:

```nginx
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
```

Then reload Nginx:

```bash
sudo nginx -t && sudo systemctl reload nginx
```

---

## 8. Firewall Rules

```bash
# Allow SSH (adjust port if you changed it)
sudo ufw allow 22/tcp

# Allow HTTP and HTTPS
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Enable the firewall
sudo ufw enable

# Verify
sudo ufw status verbose
```

> MySQL port 3306 is intentionally **not** opened — the application connects via `127.0.0.1` (localhost), so no external exposure is required.

---

## 9. Post-Deployment Verification

### 9.1 Check the Gunicorn service

```bash
sudo systemctl status webchecker
sudo journalctl -u webchecker -n 50 --no-pager
```

### 9.2 Check Nginx

```bash
sudo systemctl status nginx
sudo tail -f /var/log/nginx/webchecker_error.log
```

### 9.3 Smoke-test the application

```bash
curl -I https://your-domain.com/auth/login
# Expect: HTTP/2 200
```

Open a browser and navigate to `https://your-domain.com`. You should see the login page.

### 9.4 First login

Log in with the default admin credentials set during `initialize_database()`. **Change the password immediately** via Admin → Change Password.

---

## 10. Ongoing Maintenance

### Deploy a new version

```bash
# 1. Copy/pull new files to /opt/webchecker
# 2. Install any new dependencies
sudo -u webchecker /opt/webchecker/venv/bin/pip install -r /opt/webchecker/requirements.txt

# 3. Reload Gunicorn (zero-downtime — workers are replaced one by one)
sudo systemctl reload webchecker

# 4. If Nginx config changed:
sudo nginx -t && sudo systemctl reload nginx
```

### View application logs

```bash
# Gunicorn access log
sudo tail -f /var/log/webchecker/access.log

# Gunicorn error log
sudo tail -f /var/log/webchecker/error.log

# systemd journal (includes startup errors)
sudo journalctl -u webchecker -f
```

### Backup the database

```bash
mysqldump -u webchecker_user -p webchecker | gzip > ~/webchecker_$(date +%Y%m%d).sql.gz
```

Automate with a daily cron:

```bash
sudo crontab -e
# Add:
0 2 * * * mysqldump -u webchecker_user -pYourPassword webchecker | gzip > /var/backups/webchecker_$(date +\%Y\%m\%d).sql.gz
```

### Rotate logs

Create `/etc/logrotate.d/webchecker`:

```
/var/log/webchecker/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    sharedscripts
    postrotate
        systemctl reload webchecker > /dev/null 2>&1 || true
    endscript
}
```

---

## 11. Troubleshooting

| Symptom | Likely Cause | Fix |
|---------|--------------|-----|
| `502 Bad Gateway` | Gunicorn not running or socket missing | `sudo systemctl restart webchecker`; check `journalctl -u webchecker` |
| `connect() to unix:/run/webchecker/webchecker.sock failed (13: Permission denied)` | Nginx user can't read socket | Add `nginx` to `webchecker` group: `sudo usermod -aG webchecker www-data`, then `sudo systemctl restart nginx` |
| `OperationalError: Access denied for user` | Wrong DB credentials in `.env` | Verify `DB_USER`/`DB_PASSWORD` match what was set in MySQL |
| `ImportError` or `ModuleNotFoundError` | Dependency not installed in venv | Run `pip install -r requirements.txt` with the venv active |
| `cryptography.fernet.InvalidToken` | `CRYPTO_SECRET` changed after credentials were stored | Restore the original key from backup; never rotate without re-encrypting stored data |
| Sessions expiring immediately | `SECRET_KEY` changed between restarts | Keep `SECRET_KEY` stable in `.env`; never regenerate it on a live system |
| `413 Request Entity Too Large` | File upload exceeds `client_max_body_size` | Increase `client_max_body_size` in the Nginx config |
| Static files returning 404 | Wrong `alias` path in Nginx | Confirm `/opt/webchecker/static/` exists and the `alias` directive ends with `/` |

---

*Generated for Website Checker Web — Flask/MySQL/Nginx/Gunicorn/Ubuntu deployment.*