Runbook — Twenty CRM Recovery

Recovery procedures for EchoHuman Twenty CRM when it becomes unresponsive or fails to start.

Symptoms

• CRM not accessible at https://crm.echohuman.org
• Container stuck in unhealthy state
• node dist/main process not binding port 3000
• Database migration errors
• Empty core schema in Postgres

Quick Checks

# Check container status
docker ps --filter name=echohuman-twenty

# Check logs
docker logs echohuman-twenty-server-1 -f --tail=100

# Check port binding
docker exec echohuman-twenty-server-1 ss -ltnp | grep 3000

# Check database
docker exec echohuman-twenty-db-1 psql -U twenty -d default \
  -c "SELECT count(*) FROM information_schema.tables WHERE table_schema='core';"

Recovery Procedure 1: Restart Container

cd /opt/stacks/echohuman-twenty
docker compose restart echohuman-twenty-server-1
docker logs echohuman-twenty-server-1 -f --tail=50

Wait 5–10 minutes for startup. If still unhealthy, proceed to Procedure 2.

Recovery Procedure 2: Rebuild Database

If database schema is corrupted (0 tables in core schema):

# Stop the stack
cd /opt/stacks/echohuman-twenty
docker compose down

# Drop corrupted schema
docker run --rm \
  --network echohuman-twenty_default \
  postgres:16 \
  psql -h echohuman-twenty-db-1 -U twenty -d default \
  -c "DROP SCHEMA core CASCADE;"

# Restart stack
docker compose up -d

# Watch startup
docker logs echohuman-twenty-server-1 -f --tail=100

Expected output after ~10 min:

Successfully registered all background sync jobs!

Then check port:

docker exec echohuman-twenty-server-1 ss -ltnp | grep 3000

Should show:

tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN

Recovery Procedure 3: Full Reset

If Procedure 2 fails, reset the entire stack:

cd /opt/stacks/echohuman-twenty

# Backup current state (optional)
docker exec echohuman-twenty-db-1 pg_dumpall -U postgres > /tmp/twenty_backup_$(date +%Y%m%d_%H%M%S).sql

# Stop and remove all containers and volumes
docker compose down -v

# Restart from scratch
docker compose up -d

# Watch startup
docker logs echohuman-twenty-server-1 -f --tail=100

Warning: This destroys all data. Only use if Procedure 2 fails.

Verification

After recovery, verify:

# 1. Container is healthy
docker ps --filter name=echohuman-twenty

# 2. Port is bound
docker exec echohuman-twenty-server-1 ss -ltnp | grep 3000

# 3. Database has schema
docker exec echohuman-twenty-db-1 psql -U twenty -d default \
  -c "SELECT count(*) FROM information_schema.tables WHERE table_schema='core';"

# 4. Public URL responds
curl -I https://crm.echohuman.org

Known Issues

Issue	Cause	Fix
APP_SECRET empty	Shell env overrides .env	Use env_file: in compose, not ${VAR}
0 tables in core schema	Partial migration	Drop schema and restart (Procedure 2)
Port 3000 not bound	Startup failure	Check logs for specific error
Database connection refused	DB container not ready	Wait longer, check DB logs

Escalation

If all procedures fail:

1. Check /opt/stacks/echohuman-twenty/.env for correct credentials
2. Verify network: docker network ls
3. Check disk space: df -h /var/lib/docker
4. Review full logs: docker logs echohuman-twenty-server-1 | tail -200
5. Contact infrastructure team with logs

Prevention

• Monitor container health regularly
• Keep backups of Postgres database
• Document any custom configuration changes
• Test recovery procedures quarterly