csi-s3-archiver
Install

S3 endpoints

Garage

A self-contained store aimed at small, self-hosted, geo-distributed clusters.

What it is#

Garage is a single static binary with no external database, designed to run on modest hardware across sites. It has the most complete feature coverage of the six here, and the most setup. A node has to be given a storage layout before it will serve.

Projecthttps://garagehq.deuxfleurs.fr
LicenceAGPL-3.0
Image testeddxflrs/garage:v1.0.1
S3 port3900

Run it#

A single-node setup, enough to archive into and to try the driver against. It is not a production topology for any of these products; each project's own documentation covers that.

docker-compose.yml
services:
  garage:
    image: dxflrs/garage:v1.0.1
    ports:
      - "3900:3900"   # S3 API
      - "3903:3903"   # admin API
    volumes:
      - ./garage.toml:/etc/garage.toml:ro
      - garage-meta:/var/lib/garage/meta
      - garage-data:/var/lib/garage/data

volumes:
  garage-meta:
  garage-data:
garage.toml — replication_factor 1 is a single-node setup; raise it for a real cluster
metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "sqlite"
replication_factor = 1

rpc_bind_addr = "[::]:3901"
rpc_public_addr = "127.0.0.1:3901"
# openssl rand -hex 32
rpc_secret = "<64 hex characters>"

[s3_api]
s3_region = "garage"
api_bind_addr = "[::]:3900"
root_domain = ".s3.garage"

[admin]
api_bind_addr = "[::]:3903"
admin_token = "<a token of your choosing>"

This one needs setting up before it will serve:

after the first start
# Garage will not serve until a storage layout exists.
docker compose exec garage /garage layout assign -z dc1 -c 1G "$(
  docker compose exec -T garage /garage node id -q | cut -d@ -f1)"
docker compose exec garage /garage layout apply --version 1

# Garage issues its own credentials; it will not accept one you pick.
docker compose exec garage /garage key create csi-archiver
docker compose exec garage /garage key allow --create-bucket csi-archiver
docker compose exec garage /garage bucket create archives
docker compose exec garage /garage bucket allow --read --write archives --key csi-archiver

Point the driver at it#

Endpoint and credentials go on the volume; nothing about the driver's installation changes. pathStyle is on because a container reached by address has no per-bucket DNS, which is the usual shape outside AWS.

a volume archiving into Garage
apiVersion: v1
kind: Secret
metadata:
  name: garage-credentials
  namespace: default
stringData:
  accessKeyId: archiver-key
  secretAccessKey: archiver-secret
---
apiVersion: v1
kind: Pod
metadata:
  name: writer
spec:
  containers:
    - name: app
      image: busybox:1.36
      command: ["sh", "-c", "echo hello > /dumps/first.txt; sleep 3600"]
      volumeMounts:
        - { name: dumps, mountPath: /dumps }
  volumes:
    - name: dumps
      csi:
        driver: s3archiver.csi.trion.de
        nodePublishSecretRef:
          name: garage-credentials
        volumeAttributes:
          bucket: archives
          prefix: "{namespace}/{podName}/"
          endpoint: http://garage.storage.svc.cluster.local:3900
          pathStyle: "true"
          region: garage

To make it the default for every volume instead, set S3A_ENDPOINT, S3A_PATH_STYLE and S3A_REGION on the DaemonSet and leave them off the volumes. The configuration reference lists both halves.

What works#

Measured, not claimed. An opt-in test suite runs every one of these against Garage through the driver's own code paths.

CapabilityWhat it gives you
Single PutObjectyesArchiving anything at all.
Multipart uploadyesFiles over the 64 MiB threshold. A heapdump is almost always over it.
ListObjectsV2yesSegment compaction and durable volumes. Ephemeral archiving never lists.
DeleteObjectyesCompaction removes fragments it has assembled; durable volumes mirror deletions.
GetObjectyesRestoring a durable volume at pod start. Ephemeral volumes never read back.
UploadPartCopyyesServer-side append and segment assembly. Without it a growing file is re-uploaded whole.
Offset append 1noThe cheapest append, one request carrying only the new bytes. An S3 Express feature.
SSE-S3yesRequesting AES256 encryption per volume. A bucket default covers you regardless.
SSE-KMSyesPer-volume encryption with a customer-managed key.
Presigned PUTyesPresigned credential mode, where the node holds no S3 keys.
Presigned POST policynoSigner-less mode, with one prefix-scoped policy in the volume Secret.

9 of 11 supported. Missing: Offset append, Presigned POST policy. The driver degrades rather than failing for all of these except where noted below.

Worth knowing#