morethanadiagnosis-hub/DEPLOYMENT_GUIDE.md

# MoreThanADiagnosis - Production Deployment Guide

**Domain**: mtd.runfoo.run
**Server**: nexus-vector (100.95.3.92)
**Job ID**: MTAD-IMPL-2025-11-18-CL
**Date**: 2025-11-18

---

## 🚀 Pre-Deployment Checklist

- [ ] SSH access to nexus-vector configured
- [ ] Domain mtd.runfoo.run points to nexus-vector (100.95.3.92)
- [ ] All secrets prepared in `.env` file
- [ ] Docker and Docker Compose installed on nexus-vector
- [ ] At least 20GB free disk space
- [ ] Ports 80, 443, 8000, 8080 available
- [ ] Database backups tested and automated
- [ ] Monitoring and alerting configured

---

## 🔐 Secrets & Environment Variables

Create `.env` file in `/srv/containers/mtad-api/`:

```bash
# Database
DB_USER=admin
DB_PASSWORD=<generate-secure-password>

# Redis
REDIS_PASSWORD=<generate-secure-password>

# Security
SECRET_KEY=<generate-with: openssl rand -hex 32>
ALGORITHM=HS256

# CORS
CORS_ORIGINS=["https://mtd.runfoo.run"]

# Email (optional for future)
SMTP_PASSWORD=<if-needed>

# AWS S3 (optional for file uploads)
S3_ACCESS_KEY=<if-needed>
S3_SECRET_KEY=<if-needed>
```

**Generate secure passwords**:
```bash
# Password (32 chars)
openssl rand -base64 24

# Secret key (64 hex chars)
openssl rand -hex 32
```

---

## 📋 Step-by-Step Deployment

### 1. Prepare Server

```bash
# SSH to nexus-vector
ssh admin@nexus-vector

# Create deployment directory
sudo mkdir -p /srv/containers/mtad-api
cd /srv/containers/mtad-api

# Clone repository
git clone https://github.com/fullsizemalt/morethanadiagnosis-hub.git .
cd backend

# Copy production docker-compose
cp docker-compose.prod.yml docker-compose.yml
```

### 2. Create SSL Certificates (Let's Encrypt)

```bash
# Install certbot
sudo apt-get update
sudo apt-get install -y certbot python3-certbot-nginx

# Create certificate (interactive)
sudo certbot certonly --standalone \
  -d mtd.runfoo.run \
  -d www.mtd.runfoo.run \
  --agree-tos \
  --email admin@morethanadiagnosis.com

# Copy to nginx directory
sudo mkdir -p /srv/containers/mtad-api/certbot/conf
sudo cp -r /etc/letsencrypt /srv/containers/mtad-api/certbot/conf/

# Fix permissions
sudo chown -R admin:admin /srv/containers/mtad-api/certbot/
```

### 3. Configure Environment

```bash
# Copy .env template
cp backend/.env.example backend/.env

# Edit with production values
nano backend/.env

# Required values:
# DB_PASSWORD=...
# REDIS_PASSWORD=...
# SECRET_KEY=...
```

### 4. Build and Start Services

```bash
# Build Docker image
docker-compose build

# Start all services
docker-compose up -d

# Wait for startup
sleep 10

# Check status
docker-compose ps

# Expected output:
# NAME                  STATUS
# mtad-postgres        Up (healthy)
# mtad-redis           Up (healthy)
# mtad-api             Up (healthy)
# mtad-nginx           Up
```

### 5. Run Database Migrations

```bash
# Enter API container
docker-compose exec api bash

# Run migrations
cd /app
alembic upgrade head

# Exit container
exit
```

### 6. Verify Deployment

```bash
# Health check
curl https://mtd.runfoo.run/health

# Expected response:
# {"status":"healthy","version":"v1","env":"production"}

# API check
curl https://mtd.runfoo.run/api/v1/health

# Expected response:
# {"status":"healthy","service":"MoreThanADiagnosis API"}

# API docs
curl https://mtd.runfoo.run/docs
```

### 7. Set Up Automatic Backups

```bash
# Create backup script
cat > /srv/containers/mtad-api/backup.sh << 'EOF'
#!/bin/bash
BACKUP_DIR="/srv/containers/mtad-api/backups"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)

mkdir -p $BACKUP_DIR

# Backup database
docker-compose exec -T postgres pg_dump -U admin morethanadiagnosis \
  | gzip > $BACKUP_DIR/db_$TIMESTAMP.sql.gz

# Keep only last 30 days
find $BACKUP_DIR -name "db_*.sql.gz" -mtime +30 -delete

echo "Backup completed: $BACKUP_DIR/db_$TIMESTAMP.sql.gz"
EOF

chmod +x /srv/containers/mtad-api/backup.sh

# Schedule daily backups (crontab)
# Edit: crontab -e
# Add: 0 2 * * * /srv/containers/mtad-api/backup.sh >> /var/log/mtad-backup.log 2>&1
```

---

## 🔍 Monitoring & Maintenance

### Health Checks

```bash
# System health
curl https://mtd.runfoo.run/health

# API health
curl https://mtd.runfoo.run/api/v1/health

# Readiness check
curl https://mtd.runfoo.run/api/v1/ready
```

### Logs

```bash
# Real-time logs
docker-compose logs -f

# Specific service
docker-compose logs -f api
docker-compose logs -f postgres
docker-compose logs -f redis

# Last 100 lines
docker-compose logs --tail=100
```

### Resource Usage

```bash
# CPU and memory
docker stats

# Disk usage
df -h /srv/containers/mtad-api

# Database size
docker-compose exec postgres psql -U admin -d morethanadiagnosis -c \
  "SELECT pg_size_pretty(pg_database_size('morethanadiagnosis'));"
```

### Restart Services

```bash
# Restart single service
docker-compose restart api

# Restart all
docker-compose restart

# Stop services
docker-compose stop

# Start services
docker-compose start

# Full restart
docker-compose down && docker-compose up -d
```

---

## 🔄 SSL Certificate Renewal

Let's Encrypt certificates expire every 90 days. Set up automatic renewal:

```bash
# Test renewal
sudo certbot renew --dry-run

# Set up automatic renewal (cron)
# Edit: sudo crontab -e
# Add: 0 3 * * * certbot renew --quiet && docker-compose reload -s nginx
```

---

## 📊 Performance Tuning

### Database Optimization

```bash
# Connect to database
docker-compose exec postgres psql -U admin -d morethanadiagnosis

# Check index usage
SELECT * FROM pg_stat_user_indexes;

# Check table sizes
SELECT schemaname, tablename, pg_size_pretty(pg_total_relation_size(tablename))
FROM pg_tables
ORDER BY pg_total_relation_size(tablename) DESC;

# Vacuum and analyze
VACUUM ANALYZE;
```

### Redis Cache

```bash
# Check memory usage
docker-compose exec redis redis-cli INFO memory

# Clear cache (if needed)
docker-compose exec redis redis-cli FLUSHALL
```

### Nginx Performance

```bash
# Check worker processes
ps aux | grep nginx

# Monitor connections
netstat -an | grep ESTABLISHED | wc -l
```

---

## 🆘 Troubleshooting

### API Not Responding

```bash
# Check if containers are running
docker-compose ps

# Check logs
docker-compose logs api

# Restart API
docker-compose restart api

# Check connectivity
curl -v https://mtd.runfoo.run/health
```

### Database Connection Errors

```bash
# Check postgres status
docker-compose logs postgres

# Test database connection
docker-compose exec postgres psql -U admin -d morethanadiagnosis -c "SELECT 1"

# Check disk space
df -h

# Restart postgres (WARNING: will interrupt connections)
docker-compose restart postgres
```

### High Memory Usage

```bash
# Check which service
docker stats

# Redis memory
docker-compose exec redis redis-cli INFO memory

# Clear Redis cache (temporary)
docker-compose exec redis redis-cli FLUSHALL

# Permanent: update docker-compose.yml maxmemory-policy
```

### Certificate Issues

```bash
# Check certificate
sudo openssl x509 -in /etc/letsencrypt/live/mtd.runfoo.run/fullchain.pem -text

# Renew certificate
sudo certbot renew --force-renewal

# Restart nginx
docker-compose restart nginx
```

---

## 🔒 Security Best Practices

### Firewall

```bash
# Allow only necessary ports
sudo ufw allow 22/tcp    # SSH
sudo ufw allow 80/tcp    # HTTP
sudo ufw allow 443/tcp   # HTTPS
sudo ufw allow 8080/tcp  # Health check
sudo ufw enable
```

### Regular Updates

```bash
# Update base images monthly
docker pull postgres:15-alpine
docker pull redis:7-alpine
docker pull nginx:alpine

# Rebuild and restart
docker-compose build
docker-compose up -d
```

### Access Control

```bash
# Restrict file permissions
chmod 600 /srv/containers/mtad-api/.env
chmod 600 /srv/containers/mtad-api/certbot/conf/letsencrypt/*/privkey.pem

# Restrict directory
chmod 750 /srv/containers/mtad-api
```

---

## 📈 Scaling & Load Balancing

### Horizontal Scaling

For multiple API instances:

```yaml
# In docker-compose.yml
services:
  api:
    deploy:
      replicas: 3

  # Nginx will load balance automatically
```

### Database Replication

For high availability:
- Set up PostgreSQL primary-replica replication
- Use read replicas for analytics
- Configure automatic failover

---

## 🔄 CI/CD Integration

### GitHub Actions for Auto-Deploy

```yaml
name: Deploy to Production

on:
  push:
    branches: [main]
    paths:
      - 'backend/**'

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Deploy to nexus-vector
        run: |
          ssh admin@nexus-vector "cd /srv/containers/mtad-api && \
            git pull origin main && \
            docker-compose build && \
            docker-compose up -d"
```

---

## 📊 Monitoring Setup (Optional)

### With Prometheus & Grafana

```bash
# Add to docker-compose.yml for monitoring
prometheus:
  image: prom/prometheus
  volumes:
    - ./prometheus.yml:/etc/prometheus/prometheus.yml
  ports:
    - "9090:9090"

grafana:
  image: grafana/grafana
  ports:
    - "3000:3000"
```

### Sentry for Error Tracking

Update backend config to send errors to Sentry:

```python
import sentry_sdk
sentry_sdk.init(dsn="https://YOUR_KEY@sentry.io/PROJECT_ID")
```

---

## 📚 Documentation

- **API Docs**: https://mtd.runfoo.run/docs
- **ReDoc**: https://mtd.runfoo.run/redoc
- **OpenAPI Schema**: https://mtd.runfoo.run/openapi.json

---

## 🎯 Deployment Status

- **Server**: nexus-vector (100.95.3.92)
- **Domain**: mtd.runfoo.run
- **Service**: MoreThanADiagnosis API
- **Ports**: 80, 443, 8000, 8080
- **Database**: PostgreSQL 15 (in Docker)
- **Cache**: Redis 7 (in Docker)
- **Reverse Proxy**: Nginx (Alpine)
- **SSL**: Let's Encrypt (auto-renewal)

---

## 📞 Support

For issues:
1. Check logs: `docker-compose logs`
2. Verify health: `curl https://mtd.runfoo.run/health`
3. Check backups: `ls -lh /srv/containers/mtad-api/backups/`
4. Review documentation: `https://mtd.runfoo.run/docs`

---

**Job ID**: MTAD-IMPL-2025-11-18-CL
**Last Updated**: 2025-11-18
**Status**: Ready for deployment
**Maintainer**: Claude (Implementation Agent)

---

## Quick Reference Commands

```bash
# Common operations
docker-compose up -d              # Start all
docker-compose down               # Stop all
docker-compose logs -f api        # Follow API logs
docker-compose exec api bash      # Shell into API
docker-compose restart api        # Restart API
docker-compose build --no-cache   # Rebuild images
docker-compose ps                 # Check status

# Database
docker-compose exec postgres psql -U admin -d morethanadiagnosis  # Connect to DB
alembic upgrade head              # Run migrations
alembic downgrade -1              # Rollback last migration

# Monitoring
curl https://mtd.runfoo.run/health        # Health check
curl https://mtd.runfoo.run/api/v1/health # API health
docker stats                               # Resource usage
df -h                                      # Disk usage

# Backup
/srv/containers/mtad-api/backup.sh        # Manual backup
ls -lh /srv/containers/mtad-api/backups/  # List backups

# SSL
sudo certbot renew --dry-run              # Test renewal
sudo certbot certificates                 # List certificates
```

---

**Ready to deploy! 🚀**