Shared file volumes

This page covers shared volumes: claims that Pods on several Nodes mount at the same time. LKS backs each one with a Latitude File Storage filesystem, mounted over NFSv4.1. For the model behind it, see Persistent storage. For volumes attached to one Node at a time, see Persistent volumes.

StorageClasses

Every Cluster with persistent storage has two StorageClasses for shared volumes. Both use the file.csi.latitude.sh provisioner.

StorageClass	Default	Reclaim policy	When the claim is deleted
`latitude-file`	No	`Delete`	The filesystem is deleted.
`latitude-file-retain`	No	`Retain`	The filesystem is kept in your project.

Both classes create the filesystem as soon as you create the claim (volumeBindingMode: Immediate) and allow expansion. Neither is the Cluster default, so set storageClassName in every claim that needs a shared volume. A claim that names no StorageClass gets a block volume from latitude-block. Don’t edit or delete the built-in classes. For other mount options, create your own StorageClass. See Mount options.

Create a shared volume

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: shared
spec:
  storageClassName: latitude-file
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi

The filesystem is created with the claim, so the claim is Bound within seconds, before any Pod uses it:

kubectl get pvc shared

Every Pod that uses the claim mounts the same filesystem, on whichever Node it runs. This Deployment runs two replicas on different Nodes, and each writes to the shared directory:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: web
      containers:
        - name: web
          image: busybox:1.37.0
          command: ['sh', '-c', 'echo "$(hostname) was here" >> /shared/log; sleep 3600']
          volumeMounts:
            - name: shared
              mountPath: /shared
      volumes:
        - name: shared
          persistentVolumeClaim:
            claimName: shared

Both replicas see each other’s lines:

kubectl exec deploy/web -- cat /shared/log

To keep the filesystem after its claim is deleted, set storageClassName: latitude-file-retain in the claim.

Read-only access

A new filesystem starts empty. To let some Pods read data that others write, use one ReadWriteMany claim and mount it read-only in the readers:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: reader
spec:
  replicas: 2
  selector:
    matchLabels:
      app: reader
  template:
    metadata:
      labels:
        app: reader
    spec:
      containers:
        - name: reader
          image: busybox:1.37.0
          command: ['sh', '-c', 'cat /shared/log; sleep 3600']
          volumeMounts:
            - name: shared
              mountPath: /shared
              readOnly: true
      volumes:
        - name: shared
          persistentVolumeClaim:
            claimName: shared
            readOnly: true

Writes from these Pods fail with Read-only file system, while the web Pods above keep writing.

Size and expansion

Filesystems are sized in whole GiB, rounded up. The default and minimum is 1 GiB, and the maximum is 20480 GiB. To grow a filesystem, increase the claim’s request:

kubectl patch pvc shared -p '{"spec":{"resources":{"requests":{"storage":"20Gi"}}}}'

The filesystem grows while Pods keep using it, and they see the new size within seconds. Filesystems can’t shrink.

Access modes

Access mode	Supported	Behavior
`ReadWriteMany`	Yes	Mounted read-write by Pods on any number of Nodes.
`ReadOnlyMany`	Yes	Mounted read-only on every Node. A new filesystem starts empty, so for read-only sharing use a `ReadWriteMany` claim and mount it read-only.
`ReadWriteOnce`	Yes	Mounted on one Node at a time. Pods on that Node can share it.
`ReadWriteOncePod`	Yes	Used by a single Pod in the Cluster.

Raw block volumes (volumeMode: Block) aren’t supported for shared volumes. Use latitude-block for those.

File ownership

Shared volumes honor the Pod’s securityContext.fsGroup. When a Pod sets it, Kubernetes changes the group of the files on the volume when it mounts it, as it does for block volumes. On a filesystem with many files, set fsGroupChangePolicy: OnRootMismatch so later mounts skip the change when the top directory already matches.

Mount options

Shared volumes are mounted with nfsvers=4.1,hard,proto=tcp,retry=1. With hard, a process waits for the filesystem to answer instead of getting an I/O error. To add mount options, create a StorageClass that sets mountOptions. They’re added after the defaults, and win where they overlap. For example, nconnect=8 opens eight connections to the filesystem instead of one, which can raise throughput (see File Storage performance):

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: latitude-file-nconnect
provisioner: file.csi.latitude.sh
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true
mountOptions:
  - nconnect=8

Then set storageClassName: latitude-file-nconnect in your claims. Filesystems are served over NFSv4 only, so don’t set nfsvers=3. StorageClass parameters aren’t supported: a claim against a class that sets one stays Pending.

Lifecycle

Event	`latitude-file` (Delete)	`latitude-file-retain` (Retain)
Pod moves or restarts	The new Pod mounts the same filesystem.	The new Pod mounts the same filesystem.
Claim deleted	The filesystem is deleted.	The filesystem is kept in your project.
Cluster deleted	The filesystem is deleted.	The filesystem is kept in your project.

Each claim is its own filesystem. Filesystems appear in the dashboard under Storage > File storage, named lks-<cluster-id>-<hash>, where the hash comes from the claim’s PersistentVolume. Don’t rename or delete one there while its claim exists: a deleted filesystem is gone for the Pods using it, and a renamed one is no longer deleted with its claim. A kept filesystem stays in your project until you delete it there.

Deleting a claim that uses latitude-file, or deleting the Cluster, deletes those filesystems. There are no snapshots yet, so back up anything you need first.

The storage network

Nodes reach filesystems over your project’s private storage network, the same one block volumes use. The first shared volume mounted on a Node puts that Node on the storage network, which adds about 20 to 30 seconds to the first Pod’s start there. Later Pods on that Node mount at once. A Node stays on the storage network until it’s deleted. Nodes need no setup: the NFS client ships with the LKS storage driver. Access is limited to your project’s storage network. Any server on it, including every Node of every Cluster in the project, can mount any filesystem in the project with read and write access. A claim’s access mode controls how Kubernetes mounts it, not who can reach it. Treat the project as the security boundary for your filesystems.

Limits

Each claim is one filesystem, and filesystems count toward your team’s Latitude File Storage filesystem limit. To raise it, contact support.
A filesystem can be up to 20480 GiB.
Filesystems are created in the Cluster’s location only.

Pricing

Filesystems are billed as Latitude File Storage, on provisioned capacity while the filesystem exists. That includes filesystems kept by latitude-file-retain after their claim or Cluster is deleted. Delete them when you no longer need them. Traffic between your Nodes and filesystems is free.

Troubleshooting

The Cluster records what’s happening as events. Read them with kubectl describe pvc <name>, kubectl describe pod <name>, or kubectl get events. A shared claim is created right away, so a claim that stays Pending means the filesystem couldn’t be created. Its events give the reason.

What you see	What it means
`ProvisioningFailed: Latitude team filesystem limit reached`	Your team is at its filesystem limit. Delete unused filesystems or ask support to raise the limit.
`ProvisioningFailed: … is more than the largest filesystem, 20480 GiB`	The claim asks for more than 20480 GiB. Request less.
`ProvisioningFailed: Latitude refused to create filesystem …`	Latitude refused the filesystem’s name. Contact support.
`ProvisioningFailed: StorageClass parameter … is not supported`	The claim’s StorageClass sets `parameters`. Use a class without them.
`ProvisioningFailed: raw block is not supported by file volumes`	The claim sets `volumeMode: Block`. Use `latitude-block` for raw block volumes.
Pod `ContainerCreating` for about 30 seconds	Expected for the first shared volume on a Node, while the Node joins the storage network.
`Joining node … to the Latitude storage network` on a VolumeAttachment	The Node is taking more than a minute to join the storage network. The Pod starts once it has joined.
`FailedAttachVolume: … Latitude failed to attach server … to the storage network`	The Node couldn’t join the storage network. The event gives the reason. Contact support if it persists.
`FailedAttachVolume: Multi-Attach error`	A `ReadWriteOnce` claim is used by Pods on two Nodes. Use `ReadWriteMany` to share it across Nodes.
`FailedMount: … mounting …`	The Node couldn’t mount the filesystem. Kubernetes retries. Check the mount options of a custom StorageClass, and contact support if it persists.

Docs

Getting started

Compute

Storage & data

Networking

Resource management

Platform settings

Usage & billing

Support

Legal

StorageClasses

Create a shared volume

Read-only access

Size and expansion

Access modes

File ownership

Mount options

Lifecycle

The storage network

Limits

Pricing

Troubleshooting

​StorageClasses

​Create a shared volume

​Read-only access

​Size and expansion

​Access modes

​File ownership

​Mount options

​Lifecycle

​The storage network

​Limits

​Pricing

​Troubleshooting

StorageClasses

Create a shared volume

Read-only access

Size and expansion

Access modes

File ownership

Mount options

Lifecycle

The storage network

Limits

Pricing

Troubleshooting