S3 endpoints
Zenko CloudServer
Scality's S3 server, the storage engine behind Zenko.
What it is#
CloudServer is a mature implementation with the widest S3 surface of the six, and the one whose behaviour is closest to AWS on the features this driver uses. The in-memory backend below is for trying it out; use the file backend or a real Scality RING for anything you intend to keep.
| Project | https://github.com/scality/cloudserver |
|---|---|
| Licence | Apache-2.0 |
| Image tested | zenko/cloudserver:latest-7.70.10 |
| S3 port | 8000 |
Run it#
A single-node setup, enough to archive into and to try the driver against. It is not a production topology for any of these products; each project's own documentation covers that.
services:
cloudserver:
image: zenko/cloudserver:latest-7.70.10
environment:
# "mem" keeps nothing across a restart. Use S3BACKEND=file with a volume to persist.
S3BACKEND: mem
SCALITY_ACCESS_KEY_ID: archiver-key
SCALITY_SECRET_ACCESS_KEY: archiver-secret
REMOTE_MANAGEMENT_DISABLE: "1"
ports:
- "8000:8000"Point the driver at it#
Endpoint and credentials go on the volume; nothing about the driver's installation
changes. pathStyle is on because a container reached by address has no
per-bucket DNS, which is the usual shape outside AWS.
apiVersion: v1
kind: Secret
metadata:
name: cloudserver-credentials
namespace: default
stringData:
accessKeyId: archiver-key
secretAccessKey: archiver-secret
---
apiVersion: v1
kind: Pod
metadata:
name: writer
spec:
containers:
- name: app
image: busybox:1.36
command: ["sh", "-c", "echo hello > /dumps/first.txt; sleep 3600"]
volumeMounts:
- { name: dumps, mountPath: /dumps }
volumes:
- name: dumps
csi:
driver: s3archiver.csi.trion.de
nodePublishSecretRef:
name: cloudserver-credentials
volumeAttributes:
bucket: archives
prefix: "{namespace}/{podName}/"
endpoint: http://cloudserver.storage.svc.cluster.local:8000
pathStyle: "true"
region: us-east-1To make it the default for every volume instead, set S3A_ENDPOINT,
S3A_PATH_STYLE and S3A_REGION on the DaemonSet and leave them off
the volumes. The configuration
reference lists both halves.
What works#
Measured, not claimed. An opt-in test suite runs every one of these against Zenko CloudServer through the driver's own code paths.
| Capability | What it gives you | |
|---|---|---|
| Single PutObject | yes | Archiving anything at all. |
| Multipart upload | yes | Files over the 64 MiB threshold. A heapdump is almost always over it. |
| ListObjectsV2 | yes | Segment compaction and durable volumes. Ephemeral archiving never lists. |
| DeleteObject | yes | Compaction removes fragments it has assembled; durable volumes mirror deletions. |
| GetObject | yes | Restoring a durable volume at pod start. Ephemeral volumes never read back. |
| UploadPartCopy | yes | Server-side append and segment assembly. Without it a growing file is re-uploaded whole. |
| Offset append 1 | no | The cheapest append, one request carrying only the new bytes. An S3 Express feature. |
| SSE-S3 | yes | Requesting AES256 encryption per volume. A bucket default covers you regardless. |
| SSE-KMS | yes | Per-volume encryption with a customer-managed key. |
| Presigned PUT | yes | Presigned credential mode, where the node holds no S3 keys. |
| Presigned POST policy | no | Signer-less mode, with one prefix-scoped policy in the volume Secret. |
9 of 11 supported. Missing: Offset append, Presigned POST policy. The driver degrades rather than failing for all of these except where noted below.