This is the fourth post in our Kubernetes fundamentals series, building on Pods and Deployments and ConfigMaps and Secrets.
We covered this exact problem for plain Docker containers earlier in this blog: a container's own writable filesystem layer disappears the moment the container is removed. Kubernetes inherits that same disposability at the Pod level — a replaced Pod gets a brand new, empty filesystem — and adds its own set of objects for solving it properly at cluster scale.
The simplest case: emptyDir
apiVersion: v1
kind: Pod
metadata:
name: cache-pod
spec:
containers:
- name: app
image: my-app:1.0
volumeMounts:
- name: cache-volume
mountPath: /tmp/cache
volumes:
- name: cache-volume
emptyDir: {}An emptyDir volume is created empty when the Pod starts and shares that storage across every container in the Pod — genuinely useful for temporary scratch space or for one container in a Pod to pass data to another. It is not a durable storage solution: it's deleted the moment the Pod itself is removed, exactly like the underlying container filesystem it's meant to improve on. If a Pod restarts (rather than being removed and recreated), the emptyDir typically does survive — but that distinction is fragile enough that it shouldn't be relied on for anything that actually matters.
PersistentVolumes and PersistentVolumeClaims: storage that outlives a Pod
For real durability, Kubernetes separates the request for storage from the actual provisioning of it — two objects, working together:
# PersistentVolumeClaim (PVC) — the request
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: db-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10GiapiVersion: apps/v1
kind: StatefulSet
metadata:
name: db
spec:
serviceName: db
replicas: 1
selector:
matchLabels:
app: db
template:
metadata:
labels:
app: db
spec:
containers:
- name: postgres
image: postgres:16
volumeMounts:
- name: db-storage
mountPath: /var/lib/postgresql/data
volumes:
- name: db-storage
persistentVolumeClaim:
claimName: db-dataA PersistentVolumeClaim (PVC) is a request: "I need 10Gi of storage, readable and writable by one Pod at a time." A PersistentVolume (PV) is the actual underlying storage that gets bound to satisfy that request — an AWS EBS volume, a GCP persistent disk, an NFS share, depending on the cluster's environment. Crucially, the PVC — and the real data behind it — survives the Pod being deleted and recreated. A new Pod referencing the same PVC reattaches to the exact same underlying storage, data intact.
Why database Pods use StatefulSets, not Deployments
Notice the example above uses a StatefulSet, not a Deployment — this is the detail most people miss the first time they try to run a database in Kubernetes. A Deployment's replicas are meant to be interchangeable — any replica can be replaced by any other, with no individual identity. A database replica very much has an identity: it has specific data attached to it, and swapping one for an identically-configured but empty replacement is not the same operation at all.
A StatefulSet gives each replica a stable, unique identity (db-0, db-1, not an arbitrary generated suffix) and — this is the important part — its own dedicated PVC, created automatically per replica and reattached to that same specific replica if it's ever recreated. This is why stateful workloads (databases, message queues, anything where individual replica identity and attached storage genuinely matter) use StatefulSets, while stateless application servers use Deployments.
Access modes: how many Pods can actually use a volume at once
The accessModes field in a PVC matters beyond just being required boilerplate — it determines a real constraint on how the resulting volume can be used:
ReadWriteOnce(RWO): mountable read-write by a single node at a time — the common case for a database's own data directory, and the mode used in the example above.ReadWriteMany(RWX): mountable read-write by multiple nodes simultaneously — needed for genuinely shared storage (several Pods writing to the same file share), but not every storage backend supports it; a plain AWS EBS volume, notably, does not —ReadWriteManyon AWS typically requires EFS instead.ReadOnlyMany(ROX): mountable read-only by multiple nodes — useful for distributing a shared, static dataset to many Pods without any of them needing write access.
Requesting ReadWriteMany against a StorageClass backed by a provisioner that only supports ReadWriteOnce fails at PVC creation time with a clear error — worth checking a storage backend's supported access modes before assuming a given multi-Pod-write pattern will actually work on it.
StorageClasses: how the actual provisioning happens
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: fast-ssd
provisioner: ebs.csi.aws.com
parameters:
type: gp3
iops: "4000"
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumerapiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: db-data
spec:
storageClassName: fast-ssd
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10GiA StorageClass tells Kubernetes how to dynamically provision storage when a PVC requests it — which cloud disk type, which performance tier — rather than requiring a cluster administrator to manually pre-create PersistentVolumes for every possible request in advance. Referencing storageClassName: fast-ssd in a PVC triggers automatic provisioning of exactly the kind of underlying disk that StorageClass describes, the moment the PVC is created.
reclaimPolicy deserves specific attention: Delete (the common default) means the underlying cloud disk is destroyed when its PVC is deleted — appropriate for genuinely disposable data, actively dangerous for anything that isn't. Retain keeps the underlying storage even after the PVC referencing it is deleted, at the cost of needing manual cleanup later. Getting this backwards in either direction is a real, recoverable-only-with-a-backup mistake — worth checking deliberately rather than assuming the default matches your intent.
volumeBindingMode: WaitForFirstConsumer is worth understanding too: it delays actually provisioning the underlying disk until a Pod using the PVC is actually scheduled, rather than provisioning immediately when the PVC is created. This matters specifically in multi-availability-zone clusters — provisioning eagerly risks creating a disk in a different availability zone than the Pod eventually gets scheduled to, which then can't attach at all. Waiting for the Pod's scheduling decision first guarantees the disk gets provisioned in the correct zone.
Backing up what actually matters
None of the mechanisms above are a backup strategy on their own — a PVC surviving Pod deletion protects against Pod churn, not against a corrupted database, an accidental DROP TABLE, or a StorageClass misconfigured with reclaimPolicy: Delete on data that mattered. Tools like Velero back up both Kubernetes object state and the actual PV contents (via cloud-provider snapshot APIs) on a schedule, independent of the PVC/PV mechanism itself — worth treating as a separate, deliberate practice rather than assuming durable storage and backed-up storage are the same thing.
Expanding a PVC's size after the fact
A PVC's requested storage size isn't necessarily fixed forever — for StorageClasses that support it, a PVC can be expanded in place without recreating it or losing data:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: fast-ssd
provisioner: ebs.csi.aws.com
allowVolumeExpansion: truekubectl patch pvc db-data -p '{"spec":{"resources":{"requests":{"storage":"20Gi"}}}}'allowVolumeExpansion: true on the StorageClass is what makes this possible at all — without it, growing a PVC requires provisioning an entirely new, larger volume and migrating data manually. Checking this setting before a workload's storage needs actually grow, rather than after, avoids discovering the hard way that the StorageClass in use doesn't support the expansion path at all.
Static provisioning: pre-created PersistentVolumes
Everything covered so far assumes dynamic provisioning — a StorageClass creates a new PV automatically when a PVC requests one. Static provisioning is the alternative: a cluster administrator pre-creates a PV directly, which a PVC then binds to by matching criteria rather than triggering new provisioning:
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-manual-001
spec:
capacity:
storage: 10Gi
accessModes: [ReadWriteOnce]
awsElasticBlockStore:
volumeID: vol-0abcd1234efgh5678This is worth knowing specifically for the case of adopting existing, already-provisioned storage into Kubernetes — an EBS volume that existed before the cluster did, for instance — the same "bring existing infrastructure under management, rather than only ever creating things fresh" scenario covered for Terraform's import command earlier in this blog, applied here to storage specifically.
What to actually remember from this post
emptyDiris temporary scratch space, tied to the Pod's lifetime — not a durable storage solution.- PVCs are requests; PVs are the actual provisioned storage — a PVC and its underlying data survive a Pod being deleted and recreated.
- StatefulSets, not Deployments, are for workloads with real per-replica identity and attached storage — most obviously, databases.
- Access modes constrain how a volume can actually be used —
ReadWriteManyisn't supported by every storage backend, and needs checking before you assume it'll work. reclaimPolicyon a StorageClass decides whether deleting a PVC destroys the underlying disk — verify it matches what you actually intend for genuinely important data.- Durable storage is not the same as backed-up storage — a real backup tool, run on a schedule, is a separate and necessary practice.
Next in the series: Autoscaling and Resource Management in Kubernetes, the final post — covering how Kubernetes decides how much CPU and memory a Pod actually gets, and when to add more of them.
