Comparison
Compared to the alternatives
Where this driver fits among S3 FUSE mounts, sidecars, log shippers and backup tools, including, at the end, the cases where you should pick one of those instead.
The short version#
Most tools in this space answer the question "how do I make S3 look like a filesystem?" This one answers a narrower question: "how do I get files out of a pod and into a bucket without touching the workload?" Narrower means it does less, and that is why it is cheap to reason about.
- yes
- partial or conditional
- no
| Approach | Nothing in the pod | Local write speed | Reads from S3 | Zero app changes | Survives termination | Any file type | Reach for it when |
|---|---|---|---|---|---|---|---|
| csi-s3-archiver | Write-once files you may want later | ||||||
| S3 SDK in the application | The app really does need object storage | ||||||
Sidecar + shared emptyDir | You already run one for other reasons | ||||||
preStop hook script | Nothing; see below | ||||||
| s3fs / goofys | Legacy code that must see a mounted bucket | ||||||
| Mountpoint for Amazon S3 (CSI) | Read-heavy AWS workloads | ||||||
| csi-s3 (FUSE-backed CSI) | You want a PVC that is a bucket | ||||||
| JuiceFS CSI driver | Shared POSIX storage across pods | ||||||
| Fluent Bit / Vector | Log records, parsed and routed | ||||||
| Velero | Disaster recovery of the whole cluster |
"Survives termination" means: A file written moments before the pod is killed still reaches the bucket, without you arranging it. That column is the one that decides most real cases.
The FUSE mounts: S3fs, goofys, Mountpoint, csi-s3, JuiceFS#
These present a bucket as a mounted filesystem. Reads and writes go over the network, translated into S3 requests by a FUSE layer. That is a different product category, and if you need to read objects, one of them is the answer rather than this driver.
What that costs when all you wanted was to archive a heapdump:
- Write latency is object-store latency. The application's
write(2)is now a network operation. A JVM writing a 4 GiB heapdump onto a FUSE mount is a very different event from one writing to local disk. - Object stores are not filesystems. Rename is copy-then-delete; partial overwrites, appends
and random writes range from expensive to unsupported. Mountpoint for Amazon S3, the
best-engineered of the group, supports sequential writes to new objects; appends only work
on S3 Express One Zone directory buckets, and overwriting an existing object needs an explicit
--allow-overwriteand a full sequential rewrite. - A FUSE mount can fail underneath a running pod. When the mount goes away, the application sees I/O errors on a path it thought was a disk.
- Mounting needs privilege. There is a real
mount(2), a FUSE device and usually mount propagation. This driver has none of that: The volume is an ordinary directory, so the container runs unprivileged with a read-only root filesystem and an empty RBAC. - JuiceFS additionally needs a metadata engine, Redis or a SQL database, that you now operate and back up. In exchange it gives real POSIX semantics and shared access across pods, which none of the others do.
Sidecars and in-app SDKs#
The two most common hand-rolled answers, and the reason this driver exists.
An S3 SDK in the application
Correct if the application's job really does involve object storage. Wrong as a way to get diagnostics out. It puts an SDK, a credential, a retry policy and a bucket name into every service that might ever produce a dump, in whatever languages those services happen to be written in, and the code runs inside the process that is currently dying of an OutOfMemoryError.
A sidecar sharing an emptyDir
Better: The application stays clean and one image does the uploading. What you pay:
- a container, its image pulls, its requests and limits, and its CVE surface, on every pod, on every node, rather than once per node;
- S3 credentials mounted into the workload's own pod, where the application container can often reach them;
- termination ordering. Kubernetes has native sidecars now, so a sidecar can outlive the main container, but you still have to configure it, and getting it wrong is silent until the day it matters.
The DaemonSet form of the same idea is one pod per node instead of one per workload, with the credentials outside the workload's namespace and the termination ordering guaranteed by the CSI contract rather than by your YAML.
Log shippers: Fluent Bit, Vector, Promtail#
These tail files, parse them into records, and route the records somewhere, including S3. If what you have is logs, and what you want is searchable structured events, they are the right tool and this driver is not competing with them.
They are the wrong tool when the artefact is a file rather than a stream of lines: A 4 GiB
heapdump, a pg_dump tarball, a core dump, a JFR recording. A log shipper either mangles those
into "records" or ignores them.
The dividing line is simple: if you would ever want to open the thing in a viewer rather than grep it, it is a file, not a log stream. Many clusters want both, and they compose fine: A log shipper for the log stream, this driver for the dumps.
Velero#
Velero backs up Kubernetes API objects and persistent volumes for disaster recovery, on a schedule, with restore as the whole point. Different job, different failure model, different retention story.
It has no opinion about a file that appeared in an ephemeral scratch directory ninety seconds ago, and this driver has no opinion about restoring your cluster. Run both.
Ease of setup#
Throughput and features get compared; the number of moving parts usually does not, and it is the one an operator lives with. Counted as objects to install and things that can be misconfigured, not as lines of YAML.
| Approach | To install | Needs cluster permissions? | Per workload |
|---|---|---|---|
| csi-s3-archiver | One CSIDriver, a namespace, a DaemonSet | None for the driver. Its ServiceAccount has no Role at all | Six lines in the pod spec |
| s3fs / goofys in a sidecar | Nothing cluster-wide | Usually privileged or SYS_ADMIN for FUSE | A sidecar, a shared volume, a lifecycle ordering problem |
| CSI FUSE drivers (csi-s3, JuiceFS) | A CSIDriver, a DaemonSet, a controller, RBAC, a StorageClass, often a metadata service | RBAC for the provisioner; the node plugin is usually privileged | A PVC |
| Mountpoint for Amazon S3 CSI | A DaemonSet, a controller, RBAC, IRSA setup | RBAC, plus an AWS IAM role per service account | A PVC |
| S3 SDK in the application | Nothing | None | A dependency, credentials, retry logic, and a code change per application |
| Fluent Bit / Vector | A DaemonSet, a ConfigMap, RBAC to read pod metadata | RBAC to list pods | Annotations or a parser entry |
| Velero | A CRD set, a controller, RBAC, a BackupStorageLocation | Cluster-admin-shaped permissions | A Backup or Schedule resource |
What "no RBAC" actually means
It is unusual enough to be worth showing. This is the whole of the driver's access to the Kubernetes API:
apiVersion: v1
kind: ServiceAccount
metadata:
name: csi-s3-archiver
namespace: csi-s3-archiver
# and that is the entire file: no Role, no RoleBinding, no ClusterRoleIt works because kubelet pushes everything the driver needs into the CSI calls: The pod's name,
namespace, UID and service account, and the contents of nodePublishSecretRef. Nothing has to be
looked up, so nothing has to be granted. If you deploy the optional Controller for PVC-declared
volumes, the RBAC that adds belongs to the upstream external-provisioner sidecar; the driver
container in that pod still talks to nobody.
The first volume, end to end
From nothing to an archived file, assuming a bucket and credentials exist:
kubectl apply -k deploy/base # 1. the driver
kubectl create secret generic s3-creds \ # 2. credentials
--from-literal=accessKeyId=... \
--from-literal=secretAccessKey=...
kubectl apply -f my-pod.yaml # 3. a pod with six extra linesThere is no operator to install first, no CRDs to reconcile, no cert-manager dependency and no webhook to fail closed. The failure mode of a misconfigured volume is a pod event naming the attribute that is wrong, at pod start, rather than a bucket that quietly stays empty.
When not to use this driver#
Stated plainly, because a comparison page that only lists wins is an advertisement:
- You need to read objects back. There is no restore, ever. Use a FUSE mount or an SDK.
- You need shared storage between pods. The volume is pod-local and dies with the pod. Use JuiceFS or a real network filesystem.
- You need the object to be up to date continuously. Archiving is asynchronous.
appendOnlynarrows the window toappendSyncSeconds, but it is still a window. - You need a PVC. This is ephemeral inline volumes only: No StorageClass and no dynamic provisioning in v1.
- Your files are rewritten in place constantly. Change detection is size + mtime, and every change means a full re-upload of that file.
- You need it today, in production, with a support contract. There has been no tagged release yet.
Where it does fit, which is write-once files that somebody may want later produced by applications that should not know about S3, it is a DaemonSet, an empty RBAC, and six lines of YAML in the pod spec.