quantum-ai/docs/POSTGRESQL_SETUP.md
Claude 4e7e43c436 Implement PostgreSQL development parity and fix migration conflicts
**PostgreSQL Development Setup:**
- Update .env with PostgreSQL configuration options
- Add comprehensive PostgreSQL setup guide (Docker + native)
- Configure development-production database parity

**Migration Conflict Resolution:**
- Create fix_migrations command to handle Railway migration conflicts
- Add reset_database command for clean development resets
- Update Railway deployment with migration conflict handling
- Add fake migration strategy for duplicate column errors

**New Management Commands:**
- `fix_migrations`: Diagnose and fix migration conflicts
- `reset_database`: Clean reset of migrations and database
- Support for both PostgreSQL and SQLite environments

**Railway Deployment Fixes:**
- Add migration conflict handling to railway.json
- Use --fake-initial and --fake strategies for deployment
- Better error recovery for existing schema conflicts

**Developer Experience:**
- Step-by-step PostgreSQL setup (Docker option for easy setup)
- Migration troubleshooting guide
- Development workflow documentation
- Local-production environment matching

**Fixes Railway Issue:**
- Resolves "column data_file already exists" error
- Handles existing database schema gracefully
- Prevents future migration conflicts

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-12 09:59:44 +05:30

5.6 KiB

PostgreSQL Local Development Setup

Why Use PostgreSQL Locally?

Using PostgreSQL locally matches your Railway production environment and prevents deployment failures caused by database engine differences.

Quick Setup (Option 1: Docker - Easiest)

1. Install Docker

Download Docker Desktop from: https://www.docker.com/products/docker-desktop/

2. Run PostgreSQL Container

# Create and start PostgreSQL container
docker run --name netcop-postgres \
  -e POSTGRES_DB=netcop_hub \
  -e POSTGRES_USER=netcop_user \
  -e POSTGRES_PASSWORD=netcop_pass \
  -p 5432:5432 \
  -d postgres:15

# Verify it's running
docker ps

3. Update Your .env File

The .env file is already configured for this setup:

DATABASE_URL=postgresql://netcop_user:netcop_pass@localhost:5432/netcop_hub

4. Start/Stop Database

# Start the database (if stopped)
docker start netcop-postgres

# Stop the database (when not needed)
docker stop netcop-postgres

# View logs (for debugging)
docker logs netcop-postgres

Full Setup (Option 2: Native PostgreSQL)

1. Install PostgreSQL

macOS (with Homebrew):

brew install postgresql@15
brew services start postgresql@15

Ubuntu/Debian:

sudo apt update
sudo apt install postgresql postgresql-contrib
sudo systemctl start postgresql
sudo systemctl enable postgresql

Windows: Download from: https://www.postgresql.org/download/windows/

2. Create Database and User

# Connect to PostgreSQL as superuser
sudo -u postgres psql

# Or on macOS/Windows:
psql postgres

# Create database and user
CREATE DATABASE netcop_hub;
CREATE USER netcop_user WITH PASSWORD 'netcop_pass';
GRANT ALL PRIVILEGES ON DATABASE netcop_hub TO netcop_user;
\q

3. Test Connection

psql -h localhost -U netcop_user -d netcop_hub
# Enter password: netcop_pass
# You should see: netcop_hub=>
\q

Django Setup

1. Install PostgreSQL Python Driver

pip install psycopg2-binary

2. Reset Migrations (Clean Start)

# Reset all migrations for clean PostgreSQL setup
python manage.py reset_database --action full --confirm

# Or manually:
python manage.py reset_database --action migrations --confirm
python manage.py makemigrations
python manage.py migrate
python manage.py populate_agents --create-admin

3. Test Your Setup

# Check database connection
python manage.py backup_users --action info

# Create test user
python manage.py create_user test@example.com testpass123 --balance 50

# Start development server
python manage.py runserver

Troubleshooting

Connection Refused Error

psycopg2.OperationalError: could not connect to server: Connection refused

Solution:

  • Ensure PostgreSQL is running: docker ps or brew services list
  • Check port 5432 is not in use: lsof -i :5432
  • For Docker: docker start netcop-postgres

Password Authentication Failed

psycopg2.OperationalError: FATAL: password authentication failed

Solution:

  • Check .env file has correct credentials
  • Recreate user with correct password:
DROP USER IF EXISTS netcop_user;
CREATE USER netcop_user WITH PASSWORD 'netcop_pass';
GRANT ALL PRIVILEGES ON DATABASE netcop_hub TO netcop_user;

Migration Conflicts

django.db.utils.ProgrammingError: column "data_file" already exists

Solution:

# Fix migration conflicts
python manage.py fix_migrations --app data_analyzer

# Or clean reset
python manage.py reset_database --action full --confirm

Database Permission Denied

django.db.utils.ProgrammingError: permission denied for relation

Solution:

# Grant all permissions to user
GRANT ALL PRIVILEGES ON DATABASE netcop_hub TO netcop_user;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO netcop_user;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO netcop_user;

Development Workflow

Daily Workflow

# 1. Start database (Docker)
docker start netcop-postgres

# 2. Start Django development server
python manage.py runserver

# 3. When done, stop database (optional)
docker stop netcop-postgres

Making Model Changes

# 1. Edit your models.py
# 2. Create migrations
python manage.py makemigrations

# 3. Test migration locally (PostgreSQL)
python manage.py migrate

# 4. Test your changes
python manage.py runserver

# 5. Commit and push (will deploy to Railway)
git add .
git commit -m "Update models"
git push origin main

Switching Between SQLite and PostgreSQL

To use SQLite (quick testing):

# In .env file:
DATABASE_URL=sqlite:///db.sqlite3

To use PostgreSQL (development/production parity):

# In .env file:
DATABASE_URL=postgresql://netcop_user:netcop_pass@localhost:5432/netcop_hub

Benefits You'll See

Reliable deployments - What works locally works on Railway
Early error detection - Catch PostgreSQL-specific issues
Consistent behavior - Same database engine everywhere
Better performance testing - Real PostgreSQL performance
Migration confidence - Test exact same migrations

Quick Commands Reference

# Database management
python manage.py backup_users --action info
python manage.py reset_database --action full --confirm
python manage.py fix_migrations --check-only

# User management
python manage.py create_user email@example.com password123 --superuser
python manage.py populate_agents --create-admin

# Docker PostgreSQL
docker start netcop-postgres
docker stop netcop-postgres
docker logs netcop-postgres

Your development environment now matches Railway production exactly! 🎉