In-stack S3 backend (Garage)

Optional profile for deployments that don’t want to provision an external S3-compatible provider (AWS S3, Cloudflare R2, Backblaze B2, MinIO, etc.). Runs Garage as a single-node container inside the Compose stack and serves the buckets the api needs locally.

When to use this

• Self-hosted deployments without a paid S3 provider.
• Air-gapped or compliance-restricted environments where data must stay on the same host.
• Trial / staging stacks where you want zero external dependencies.

Single-node Garage = no DR

Single-node Garage stores all blobs on the same disk as Postgres. If the host dies, you lose both the database and the S3 backend. Snapshot ./data/garage/ and ./data/postgres/ to off-site storage if you care about disaster recovery. For production with real DR requirements, prefer an external S3 provider — the api works identically against both. See the S3 section of Installation and .env.example.

Single-node only

garage.toml ships with replication_factor = 1 and the container starts with --single-node. If you ever scale to a real cluster, change both together — Garage refuses to add a second node while replication_factor is still 1.

How it works

The garage Compose service runs garage server --single-node --default-access-key:

• --single-node auto-applies a one-node cluster layout on first boot.
• --default-access-key auto-creates an S3 access key from the GARAGE_DEFAULT_ACCESS_KEY / GARAGE_DEFAULT_SECRET_KEY env vars (the compose file maps them from S3_API_KEY / S3_API_SECRET in .env).

The api creates its own buckets on first boot — no manual bootstrap is required. When it calls CreateBucket on Garage via the S3 API, Garage grants the calling key ownership, so the api can read and write immediately.

Activate the profile

In .env:

COMPOSE_PROFILES=garage          # or backup,garage for both
S3_ENDPOINT=http://garage:3900

Without garage in COMPOSE_PROFILES, the service is defined in docker-compose.yaml but does not start.

Generate the three secrets

Garage needs an S3 access key pair (which the api reuses) and an internal RPC secret:

echo "S3_API_KEY=GK$(openssl rand -hex 16)"
echo "S3_API_SECRET=$(openssl rand -hex 32)"
echo "GARAGE_RPC_SECRET=$(openssl rand -hex 32)"

Paste the three lines into .env. The “GK” prefix is a Garage convention so the key is easy to spot in logs — any string works, but keep it.

First boot

docker compose pull
docker compose up -d

That’s the whole setup. On first start:

1. Garage applies a single-node cluster layout (no garage layout assign needed).

2. Garage creates the access key from GARAGE_DEFAULT_ACCESS_KEY / GARAGE_DEFAULT_SECRET_KEY.

3. The api runs migrations, creates the project bucket (sancho-env-${DEPLOY_ENV}), and then seeds the root account, which creates the account bucket (sancho-env-${DEPLOY_ENV}-account-1).

Both buckets are owned by the calling key, so the api can use them immediately.

Verify

docker compose exec garage /garage status        # node Up, layout applied
docker compose exec garage /garage key list      # the auto-imported key
docker compose exec garage /garage bucket list   # 2 buckets, both with the key allowed

# End-to-end: upload an avatar from the web app, then
ls ./data/garage/data                            # blob files appear

Backup profile on top

If you also set COMPOSE_PROFILES=backup,garage, the postgres-backup service writes daily dumps to Garage. It does not create its bucket on its own, so run this one-liner once:

docker compose exec garage /garage bucket create "$BACKUP_S3_BUCKET"

Garage grants the default key full access on bucket creation, so no bucket allow step is needed. After that, docker compose exec postgres-backup /backup.sh forces an immediate dump as a sanity check.

Caution

The backup lives on the same host as Postgres. That’s useful but not disaster recovery. Snapshot ./data/garage/ off-host for off-site safety, or point BACKUP_S3_BUCKET at an external S3 endpoint while keeping Garage for the api’s own buckets.

Operations

Ports

Garage binds 3900 (S3 API), 3901 (RPC), and 3903 (admin) inside the sancho network (garage.toml does not configure [s3_web], so the web port 3902 is not bound). None are exposed on the host. The api reaches it as http://garage:3900; for ad-hoc CLI use, run docker compose exec garage /garage ....

Persistent data

Path	Contents
./data/garage/meta	SQLite metadata, bucket/key state
./data/garage/data	Object blobs

Snapshot both directories together — splitting them between snapshots produces an inconsistent state. Stop the stack (docker compose down) before copy-style backups to avoid file-locking races.

Restoring

docker compose down
sudo rm -rf ./data/garage
sudo tar -xzf garage-data-YYYY-MM-DD.tar.gz   # extracts ./data/garage/{meta,data}
docker compose up -d

The access key, bucket list, and allow-list survive in the metadata snapshot — no extra steps after a restore.

Rotating S3 credentials

Garage tracks keys by access-key ID. Changing S3_API_KEY in .env makes the old key unusable on the api side but the old key still exists in Garage:

1. Generate a new pair:

   NEW_KEY="GK$(openssl rand -hex 16)"; NEW_SECRET="$(openssl rand -hex 32)"

2. Import as a new key in Garage:

   docker compose exec garage /garage key import "$NEW_KEY" "$NEW_SECRET" -n sancho-new --yes

3. Grant access to every bucket the old key could reach:

   for b in $(docker compose exec garage /garage bucket list | awk '/^ /{print $1}'); do
     docker compose exec garage /garage bucket allow --read --write --owner "$b" --key "$NEW_KEY"
   done

4. Update .env and restart api + postgres-backup:

   sed -i -e "s|^S3_API_KEY=.*|S3_API_KEY=$NEW_KEY|" \
          -e "s|^S3_API_SECRET=.*|S3_API_SECRET=$NEW_SECRET|" .env
   docker compose up -d api postgres-backup

5. (Optional) delete the old key once confirmed working:

   docker compose exec garage /garage key delete <OLD_KEY_ID> --yes

Troubleshooting

Api logs show NoSuchBucket on first request

The api’s CreateBucket call failed silently. Inspect docker compose logs api | grep -i 'create.*bucket\|s3' for the actual error. Common causes: S3_ENDPOINT mismatched, the access key was rotated between Garage’s auto-create and the api’s first boot.

Api logs show 'S3 key lacks CreateBucket permission'

You’re on Garage but the auto-key wasn’t created — usually because S3_API_KEY / S3_API_SECRET weren’t set before the first docker compose up -d. Stop the stack, set both in .env, docker compose down, then docker compose up -d again.

Garage is unhealthy / never reaches service_healthy

docker compose logs garage will show the cause. garage.toml is mounted read-only at /etc/garage.toml; if the file is missing or the operator edited it incorrectly, Garage exits at startup.