Point in time Snapshot of Persistent Volume Data with Kubernetes’ Volume Snapshots

Published in

Linux Shots

7 min readDec 24, 2022

When we perform an upgrade, or make a configuration change in an application, there is always little risk that the application may behave unexpected. Upgraded/updated version of application may make changes in lots of files which resides in persistent volume, which may later become difficult to revert it back, in case of unexpected result.

Kubernetes has a feature which can be used to create snapshots from an existing PVC or to clone a PVC. This is supported in CSI drivers based persistent volumes with snapshot feature. This can be used to create backup of persistent volumes before performing an upgrade or any changes to application.

This feature is supported by most cloud providers and third-party solutions who provide CSI drivers. To list some, here are the below:

GCE persistent disk CSI driver(pd.csi.storage.gke.io)
AWS EBS CSI driver (ebs.csi.aws.com)
Azure Disk CSI driver (disk.csi.azure.com)
Portworx CSI driver (pxd.portworx.com)
NetApp Trident CSI driver (csi.trident.netapp.io)
Host path CSI driver (hostpath.csi.k8s.io)

Why it works on my machine?
Enable support for volume snapshot
Deploy a sample application with PVC
Create snapshot from PVC
Restore from snapshot
Clone a PVC

Why it works on my machine?

I am using:

K8s cluster 1.23.15, but it should work on any cluster above v1.17
Master and worker nodes on Ubuntu 22.04 LTS OS, But It should work on any Linux based OS
CRI-O container runtime 1.24.3, Any container runtime like containerd, docker, etc. should also be fine

Enable support for volume snapshots

Cluster must have volume snapshot CRDs, and snapshot controller deployed on it for this to work.

Create volume snapshots CRDs on cluster.

Check if volume snapshot’s CRDs are present on cluster.

kubectl get crds | grep snapshot.storage.k8s.io

Above command must list below three CRDs.

volumesnapshotclasses.snapshot.storage.k8s.io
volumesnapshotcontents.snapshot.storage.k8s.io
volumesnapshots.snapshot.storage.k8s.io

If they are not listed, run below command to create these CRDs. Here we are deploying CRDs from release-6.1.

kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-6.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-6.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-6.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml

VolumeSnapshotClass is equivalent to storage class in K8s. It is used to define provisioner and different attributes which can be used to create volumesnapshot. It creates VolumeSnapshotContent when a request received through VolumeSnapshot.

VolumeSnapshotContentis equivalent to PV. This is a piece of storage which is provisioned by cluster admin or dynamically provisioned by VolumeSnapshotClass to store data from source volume.

VolumeSnapshot is equivalent to PVC. It is a request for storage snapshot made by user.

2. Install Snapshot controller

Check if snapshot controller is available on cluster

kubectl -n kube-system get po --selector app=snapshot-controller

If it's not available, use below command to install snapshot controller

kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-6.1/deploy/kubernetes/snapshot-controller/rbac-snapshot-controller.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-6.1/deploy/kubernetes/snapshot-controller/setup-snapshot-controller.yaml

3. You must have CSI driver running on your machine. If you are using cloud native K8s cluster (e.g. EKS, GKE or AKS), This must be available on cluster.

In our demo, I am using hostpath-csi driver to use node’s path as volume. If you want to use this driver for testing this demo, you can install it using below commands. Host path is not recommended in live/production cluster.

git clone https://github.com/kubernetes-csi/csi-driver-host-path.git
cd csi-driver-host-path
deploy/kubernetes-latest/deploy.sh

# Deploy storage class using below command

cat <<EOF | kubectl apply -f -
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: csi-hostpath-sc
provisioner: hostpath.csi.k8s.io
reclaimPolicy: Delete
volumeBindingMode: Immediate
EOF

To remove it after testing

# To remove it after demo
cd csi-driver-host-path
deploy/kubernetes-latest/destroy.sh
kubectl delete sc csi-hostpath-sc

Deploy a sample application with a PVC

Deploy PVC

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: csi-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: csi-hostpath-sc
EOF

2. Deploy application which uses this volume claim

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-csi-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-csi-app
  template:
    metadata:
      labels:
        app: my-csi-app
    spec:
      containers:
        - name: my-frontend
          image: busybox
          volumeMounts:
          - mountPath: "/data"
            name: my-csi-volume
          command: [ "sleep", "1000000" ]
      volumes:
        - name: my-csi-volume
          persistentVolumeClaim:
            claimName: csi-pvc
EOF

Persistent Volume Claim and Persistent Volume

3. Create a file inside pod volume

kubectl exec -it <pod-name-of-app> -- sh
echo "This is version 1" > /data/version.txt

Create snapshot from PVC

Before we create PVC, we will need to create VolumeSnapshotClass which is similar to StorageClass for volume snapshots.

cat <<EOF | kubectl apply -f -
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: csi-hostpath-sc
driver: hostpath.csi.k8s.io
deletionPolicy: Delete
EOF

In above manifest YAML, you may have noticed, it is very similar to storage class where we have driver in place of provisioner in storage class and instead of reclaimPolicy here we have deletionPolicy.

Create snapshot from PVC

cat <<EOF | kubectl apply -f -
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: new-snapshot-demo
spec:
  volumeSnapshotClassName: csi-hostpath-sc
  source:
    persistentVolumeClaimName: csi-pvc
EOF

In above manifest, we have spec.source field where we define source of this volume snapshot. Also, We define spec.volumeSnapshotClassName to let cluster know which respective VolumeSnapshotClass to use. If we don’t define this value, It will use provisioner of PersistentVolumeClaim to create volume snapshot.

Check for snapshot

kubectl get VolumeSnapshot new-snapshot-demo
kubectl get VolumeSnapshotContent

Wait until READYTOUSE column becomes true.

Check volume snapshot and volume snapshot content

Restore from snapshot

Let's make change in file which resides in PV of application. Then we will restore it back from snapshot.

kubectl exec -it <pod-name-of-app> -- sh
echo "This is version 2" > /data/version.txt

Now, Let's restore to previous version from snapshot. We will name restored PVC as csi-pvc-restored. To restore a PVC from volume snapshot, we use normal PVC manifest. We add snapshot details in spec.dataSource to create a PVC from a Volume Snapshot.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: csi-pvc-restored
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: csi-hostpath-sc
  dataSource:
    name: new-snapshot-demo
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
EOF

Here, when creating a new PVC, we are using spec.dataSource where we define source volume snapshot to use to create new restored PVC. This will create PVC and restore data from snapshot.

You should be able to see restored PVC in list. kubectl get pvc

Now, we can update application deployment file to use restored PVC and apply it.

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-csi-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-csi-app
  template:
    metadata:
      labels:
        app: my-csi-app
    spec:
      containers:
        - name: my-frontend
          image: busybox
          volumeMounts:
          - mountPath: "/data"
            name: my-csi-volume
          command: [ "sleep", "1000000" ]
      volumes:
        - name: my-csi-volume
          persistentVolumeClaim:
            claimName: csi-pvc-restored
EOF

Update application deployment to use restored PVC

Now, if we get inside persistent volume and check our file, it should be of previous version.

kubectl exec -it <pod-name> -- cat /data/version.txt

Create clone of a PVC

To create clone of a PVC, we will create another PVC with name csi-pvc-clone. We add source PVC details in spec.dataSource of PVC manifest.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: csi-pvc-clone
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: csi-hostpath-sc
  dataSource:
    name: csi-pvc
    kind: PersistentVolumeClaim
EOF

Here, instead of using VolumeSnapshot as data source, we are using PersistentVolumeClaim to create new PVC which will be clone of it.

This will clone the existing PVC and create new PVC.

Thanks for reading. I hope this article will be helpful to you. You may learn more about Volume Snapshot on K8s official document https://kubernetes.io/docs/concepts/storage/volume-snapshots/

If you learned something new and liked it, don’t forget to join your hands to clap. You may also share a cup of coffee with me on https://www.buymeacoffee.com/linuxshots or by scanning below QR code.