04/30 Web Checker web app ver. 1.0
This commit is contained in:
@@ -0,0 +1,478 @@
|
||||
# Website Checker — Production Deployment Guide
|
||||
### Ubuntu Server · Nginx · Gunicorn · MySQL · systemd
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Prerequisites](#1-prerequisites)
|
||||
2. [Server Initial Setup](#2-server-initial-setup)
|
||||
3. [MySQL Database](#3-mysql-database)
|
||||
4. [Application Deployment](#4-application-deployment)
|
||||
5. [Gunicorn systemd Service](#5-gunicorn-systemd-service)
|
||||
6. [Nginx Configuration](#6-nginx-configuration)
|
||||
7. [SSL/TLS with Certbot](#7-ssltls-with-certbot)
|
||||
8. [Firewall Rules](#8-firewall-rules)
|
||||
9. [Post-Deployment Verification](#9-post-deployment-verification)
|
||||
10. [Ongoing Maintenance](#10-ongoing-maintenance)
|
||||
11. [Troubleshooting](#11-troubleshooting)
|
||||
|
||||
---
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
| Item | Requirement |
|
||||
|------|-------------|
|
||||
| OS | Ubuntu 22.04 LTS or 24.04 LTS |
|
||||
| RAM | ≥ 1 GB (2 GB recommended) |
|
||||
| Disk | ≥ 10 GB |
|
||||
| Access | Root or sudo user |
|
||||
| Domain | A DNS A-record pointing to your server's IP |
|
||||
| Python | 3.10+ (pre-installed on Ubuntu 22.04+) |
|
||||
|
||||
---
|
||||
|
||||
## 2. Server Initial Setup
|
||||
|
||||
### 2.1 Update the system
|
||||
|
||||
```bash
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
```
|
||||
|
||||
### 2.2 Install system dependencies
|
||||
|
||||
```bash
|
||||
sudo apt install -y \
|
||||
python3 python3-pip python3-venv \
|
||||
mysql-server \
|
||||
nginx \
|
||||
certbot python3-certbot-nginx \
|
||||
git curl ufw
|
||||
```
|
||||
|
||||
### 2.3 Create a dedicated application user
|
||||
|
||||
```bash
|
||||
sudo useradd --system --shell /bin/bash --create-home webchecker
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. MySQL Database
|
||||
|
||||
### 3.1 Secure the MySQL installation
|
||||
|
||||
```bash
|
||||
sudo mysql_secure_installation
|
||||
```
|
||||
|
||||
Follow the prompts: set a root password, remove anonymous users, disallow remote root login, remove test database.
|
||||
|
||||
### 3.2 Create the database and application user
|
||||
|
||||
```bash
|
||||
sudo mysql -u root -p
|
||||
```
|
||||
|
||||
Inside the MySQL shell:
|
||||
|
||||
```sql
|
||||
CREATE DATABASE webchecker CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
||||
|
||||
CREATE USER 'webchecker_user'@'127.0.0.1' IDENTIFIED BY 'YourStrongPasswordHere';
|
||||
|
||||
GRANT ALL PRIVILEGES ON webchecker.* TO 'webchecker_user'@'127.0.0.1';
|
||||
|
||||
FLUSH PRIVILEGES;
|
||||
EXIT;
|
||||
```
|
||||
|
||||
> **Security note:** Use `127.0.0.1` (not `localhost`) so the connector uses TCP rather than the UNIX socket, which matches the `DB_HOST=127.0.0.1` setting in `.env`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Application Deployment
|
||||
|
||||
### 4.1 Transfer the application files
|
||||
|
||||
Option A — copy from your workstation:
|
||||
```bash
|
||||
scp -r ./webchecker_web/ youruser@your-server-ip:/tmp/webchecker_web
|
||||
```
|
||||
|
||||
Option B — clone from a private Git repository:
|
||||
```bash
|
||||
sudo -u webchecker git clone https://github.com/your-org/webchecker.git /opt/webchecker
|
||||
```
|
||||
|
||||
Then move files into place (Option A):
|
||||
```bash
|
||||
sudo mv /tmp/webchecker_web /opt/webchecker
|
||||
sudo chown -R webchecker:webchecker /opt/webchecker
|
||||
```
|
||||
|
||||
### 4.2 Create the Python virtual environment
|
||||
|
||||
```bash
|
||||
sudo -u webchecker python3 -m venv /opt/webchecker/venv
|
||||
```
|
||||
|
||||
### 4.3 Install Python dependencies
|
||||
|
||||
```bash
|
||||
sudo -u webchecker /opt/webchecker/venv/bin/pip install --upgrade pip
|
||||
sudo -u webchecker /opt/webchecker/venv/bin/pip install -r /opt/webchecker/requirements.txt
|
||||
```
|
||||
|
||||
### 4.4 Configure environment variables
|
||||
|
||||
```bash
|
||||
sudo cp /opt/webchecker/.env.example /opt/webchecker/.env
|
||||
sudo nano /opt/webchecker/.env
|
||||
```
|
||||
|
||||
Fill in every value — especially:
|
||||
|
||||
```dotenv
|
||||
SECRET_KEY=<output of: python3 -c "import secrets; print(secrets.token_hex(32))">
|
||||
DB_HOST=127.0.0.1
|
||||
DB_PORT=3306
|
||||
DB_NAME=webchecker
|
||||
DB_USER=webchecker_user
|
||||
DB_PASSWORD=YourStrongPasswordHere
|
||||
CRYPTO_SECRET=<output of: python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())">
|
||||
FLASK_ENV=production
|
||||
```
|
||||
|
||||
Secure the file so only the application user can read it:
|
||||
|
||||
```bash
|
||||
sudo chown webchecker:webchecker /opt/webchecker/.env
|
||||
sudo chmod 600 /opt/webchecker/.env
|
||||
```
|
||||
|
||||
### 4.5 Initialise the database schema
|
||||
|
||||
The application auto-creates all tables on first startup via `initialize_database()`. Run it once manually to verify:
|
||||
|
||||
```bash
|
||||
sudo -u webchecker bash -c '
|
||||
cd /opt/webchecker
|
||||
source venv/bin/activate
|
||||
python - <<EOF
|
||||
from app import create_app
|
||||
app = create_app()
|
||||
with app.app_context():
|
||||
from config import initialize_database
|
||||
initialize_database()
|
||||
print("Database initialised successfully.")
|
||||
EOF
|
||||
'
|
||||
```
|
||||
|
||||
### 4.6 Create the Gunicorn socket directory
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /run/webchecker
|
||||
sudo chown webchecker:webchecker /run/webchecker
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Gunicorn systemd Service
|
||||
|
||||
### 5.1 Create the service unit file
|
||||
|
||||
```bash
|
||||
sudo nano /etc/systemd/system/webchecker.service
|
||||
```
|
||||
|
||||
Paste the following:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Website Checker — Gunicorn Application Server
|
||||
After=network.target mysql.service
|
||||
Requires=mysql.service
|
||||
|
||||
[Service]
|
||||
Type=notify
|
||||
User=webchecker
|
||||
Group=webchecker
|
||||
WorkingDirectory=/opt/webchecker
|
||||
EnvironmentFile=/opt/webchecker/.env
|
||||
ExecStart=/opt/webchecker/venv/bin/gunicorn \
|
||||
--bind unix:/run/webchecker/webchecker.sock \
|
||||
--workers 4 \
|
||||
--worker-class sync \
|
||||
--timeout 120 \
|
||||
--access-logfile /var/log/webchecker/access.log \
|
||||
--error-logfile /var/log/webchecker/error.log \
|
||||
--log-level info \
|
||||
wsgi:application
|
||||
ExecReload=/bin/kill -s HUP $MAINPID
|
||||
KillMode=mixed
|
||||
TimeoutStopSec=10
|
||||
PrivateTmp=true
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
### 5.2 Create the log directory
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /var/log/webchecker
|
||||
sudo chown webchecker:webchecker /var/log/webchecker
|
||||
```
|
||||
|
||||
### 5.3 Enable and start the service
|
||||
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable webchecker
|
||||
sudo systemctl start webchecker
|
||||
sudo systemctl status webchecker
|
||||
```
|
||||
|
||||
You should see `Active: active (running)`. The socket file `/run/webchecker/webchecker.sock` will be created automatically.
|
||||
|
||||
---
|
||||
|
||||
## 6. Nginx Configuration
|
||||
|
||||
### 6.1 Create the Nginx site configuration
|
||||
|
||||
```bash
|
||||
sudo nano /etc/nginx/sites-available/webchecker
|
||||
```
|
||||
|
||||
Paste (replace `your-domain.com` with your actual domain):
|
||||
|
||||
```nginx
|
||||
upstream webchecker_app {
|
||||
server unix:/run/webchecker/webchecker.sock fail_timeout=0;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name your-domain.com www.your-domain.com;
|
||||
|
||||
# Security headers
|
||||
add_header X-Frame-Options "SAMEORIGIN" always;
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
add_header X-XSS-Protection "1; mode=block" always;
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
|
||||
|
||||
# Client upload size (raise if users upload large files for AI analysis)
|
||||
client_max_body_size 20M;
|
||||
|
||||
# Static files served directly by Nginx (fast path)
|
||||
location /static/ {
|
||||
alias /opt/webchecker/static/;
|
||||
expires 30d;
|
||||
access_log off;
|
||||
add_header Cache-Control "public, immutable";
|
||||
}
|
||||
|
||||
# Proxy everything else to Gunicorn
|
||||
location / {
|
||||
proxy_pass http://webchecker_app;
|
||||
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
proxy_connect_timeout 60s;
|
||||
proxy_read_timeout 120s;
|
||||
proxy_send_timeout 120s;
|
||||
|
||||
proxy_buffering on;
|
||||
proxy_buffer_size 16k;
|
||||
proxy_buffers 4 32k;
|
||||
}
|
||||
|
||||
# Logging
|
||||
access_log /var/log/nginx/webchecker_access.log;
|
||||
error_log /var/log/nginx/webchecker_error.log;
|
||||
}
|
||||
```
|
||||
|
||||
### 6.2 Enable the site
|
||||
|
||||
```bash
|
||||
sudo ln -s /etc/nginx/sites-available/webchecker /etc/nginx/sites-enabled/
|
||||
sudo nginx -t # test config — must report "syntax is ok"
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. SSL/TLS with Certbot
|
||||
|
||||
```bash
|
||||
sudo certbot --nginx -d your-domain.com -d www.your-domain.com
|
||||
```
|
||||
|
||||
Certbot will automatically:
|
||||
- Obtain a Let's Encrypt certificate
|
||||
- Modify the Nginx config to add HTTPS (port 443)
|
||||
- Insert a redirect from HTTP → HTTPS
|
||||
- Schedule automatic renewal
|
||||
|
||||
**Verify auto-renewal works:**
|
||||
|
||||
```bash
|
||||
sudo certbot renew --dry-run
|
||||
```
|
||||
|
||||
After Certbot finishes, add an HSTS header inside the `server` block that Certbot created for port 443:
|
||||
|
||||
```nginx
|
||||
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
|
||||
```
|
||||
|
||||
Then reload Nginx:
|
||||
|
||||
```bash
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Firewall Rules
|
||||
|
||||
```bash
|
||||
# Allow SSH (adjust port if you changed it)
|
||||
sudo ufw allow 22/tcp
|
||||
|
||||
# Allow HTTP and HTTPS
|
||||
sudo ufw allow 80/tcp
|
||||
sudo ufw allow 443/tcp
|
||||
|
||||
# Enable the firewall
|
||||
sudo ufw enable
|
||||
|
||||
# Verify
|
||||
sudo ufw status verbose
|
||||
```
|
||||
|
||||
> MySQL port 3306 is intentionally **not** opened — the application connects via `127.0.0.1` (localhost), so no external exposure is required.
|
||||
|
||||
---
|
||||
|
||||
## 9. Post-Deployment Verification
|
||||
|
||||
### 9.1 Check the Gunicorn service
|
||||
|
||||
```bash
|
||||
sudo systemctl status webchecker
|
||||
sudo journalctl -u webchecker -n 50 --no-pager
|
||||
```
|
||||
|
||||
### 9.2 Check Nginx
|
||||
|
||||
```bash
|
||||
sudo systemctl status nginx
|
||||
sudo tail -f /var/log/nginx/webchecker_error.log
|
||||
```
|
||||
|
||||
### 9.3 Smoke-test the application
|
||||
|
||||
```bash
|
||||
curl -I https://your-domain.com/auth/login
|
||||
# Expect: HTTP/2 200
|
||||
```
|
||||
|
||||
Open a browser and navigate to `https://your-domain.com`. You should see the login page.
|
||||
|
||||
### 9.4 First login
|
||||
|
||||
Log in with the default admin credentials set during `initialize_database()`. **Change the password immediately** via Admin → Change Password.
|
||||
|
||||
---
|
||||
|
||||
## 10. Ongoing Maintenance
|
||||
|
||||
### Deploy a new version
|
||||
|
||||
```bash
|
||||
# 1. Copy/pull new files to /opt/webchecker
|
||||
# 2. Install any new dependencies
|
||||
sudo -u webchecker /opt/webchecker/venv/bin/pip install -r /opt/webchecker/requirements.txt
|
||||
|
||||
# 3. Reload Gunicorn (zero-downtime — workers are replaced one by one)
|
||||
sudo systemctl reload webchecker
|
||||
|
||||
# 4. If Nginx config changed:
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
### View application logs
|
||||
|
||||
```bash
|
||||
# Gunicorn access log
|
||||
sudo tail -f /var/log/webchecker/access.log
|
||||
|
||||
# Gunicorn error log
|
||||
sudo tail -f /var/log/webchecker/error.log
|
||||
|
||||
# systemd journal (includes startup errors)
|
||||
sudo journalctl -u webchecker -f
|
||||
```
|
||||
|
||||
### Backup the database
|
||||
|
||||
```bash
|
||||
mysqldump -u webchecker_user -p webchecker | gzip > ~/webchecker_$(date +%Y%m%d).sql.gz
|
||||
```
|
||||
|
||||
Automate with a daily cron:
|
||||
|
||||
```bash
|
||||
sudo crontab -e
|
||||
# Add:
|
||||
0 2 * * * mysqldump -u webchecker_user -pYourPassword webchecker | gzip > /var/backups/webchecker_$(date +\%Y\%m\%d).sql.gz
|
||||
```
|
||||
|
||||
### Rotate logs
|
||||
|
||||
Create `/etc/logrotate.d/webchecker`:
|
||||
|
||||
```
|
||||
/var/log/webchecker/*.log {
|
||||
daily
|
||||
missingok
|
||||
rotate 14
|
||||
compress
|
||||
delaycompress
|
||||
notifempty
|
||||
sharedscripts
|
||||
postrotate
|
||||
systemctl reload webchecker > /dev/null 2>&1 || true
|
||||
endscript
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Troubleshooting
|
||||
|
||||
| Symptom | Likely Cause | Fix |
|
||||
|---------|--------------|-----|
|
||||
| `502 Bad Gateway` | Gunicorn not running or socket missing | `sudo systemctl restart webchecker`; check `journalctl -u webchecker` |
|
||||
| `connect() to unix:/run/webchecker/webchecker.sock failed (13: Permission denied)` | Nginx user can't read socket | Add `nginx` to `webchecker` group: `sudo usermod -aG webchecker www-data`, then `sudo systemctl restart nginx` |
|
||||
| `OperationalError: Access denied for user` | Wrong DB credentials in `.env` | Verify `DB_USER`/`DB_PASSWORD` match what was set in MySQL |
|
||||
| `ImportError` or `ModuleNotFoundError` | Dependency not installed in venv | Run `pip install -r requirements.txt` with the venv active |
|
||||
| `cryptography.fernet.InvalidToken` | `CRYPTO_SECRET` changed after credentials were stored | Restore the original key from backup; never rotate without re-encrypting stored data |
|
||||
| Sessions expiring immediately | `SECRET_KEY` changed between restarts | Keep `SECRET_KEY` stable in `.env`; never regenerate it on a live system |
|
||||
| `413 Request Entity Too Large` | File upload exceeds `client_max_body_size` | Increase `client_max_body_size` in the Nginx config |
|
||||
| Static files returning 404 | Wrong `alias` path in Nginx | Confirm `/opt/webchecker/static/` exists and the `alias` directive ends with `/` |
|
||||
|
||||
---
|
||||
|
||||
*Generated for Website Checker Web — Flask/MySQL/Nginx/Gunicorn/Ubuntu deployment.*
|
||||
Reference in New Issue
Block a user