Skip to main content

Mount CVMFS repositories on Kubernetes

image-20220509154116581

danger

The CVMFS CSI Plugin isn't stable yet, so we have to do it the old way: using hostPath.

Helm and Docker resources

The Helm resources are stored on ClusterFactory Git Repository.

The Dockerfile is described in the git repository cvmfs/cvmfs.

A Docker image can be pulled with:

docker pull docker.io/cvmfs/cvmfs:latest

1. AppProject

Apply the AppProject:

user@local:/ClusterFactory
kubectl apply -f argo/cvmfs/app-project.yml

2. Secrets

Create a SealedSecret which contains the keys of the repositories:

  1. Create a -secret.yml.local file:
argo/cvmfs/secrets/cvmfs-keys-secret.yml.local
apiVersion: v1
kind: Secret
metadata:
name: cvmfs-keys-secret
namespace: cvmfs
type: Opaque
stringData:
software.sion.csquare.run.pub: |
-----BEGIN PUBLIC KEY-----
...
-----END PUBLIC KEY-----
  1. Seal the secret:
user@local:/ClusterFactory
cfctl kubeseal
  1. Apply the SealedSecret:
user@local:/ClusterFactory
kubectl apply -f argo/cvmfs/secrets/cvmfs-keys-sealed-secret.yml

3. Editing cvmfs-service-app.yml to use the fork

Change the repoURL to the URL used to pull the fork. Also add the values-production.yaml file to customize the values.

argo/provisioning/apps/cvmfs-service-app.yml > spec > source
source:
# You should have forked this repo. Change the URL to your fork.
repoURL: git@github.com:<your account>/ClusterFactory.git
targetRevision: HEAD
path: helm/cvmfs-service
helm:
releaseName: cvmfs-service

# Create a values file inside your fork and change the values.
valueFiles:
- values-production.yaml

4. Adding custom values to the chart

tip

Read the values.yaml to see all the default values.

4.a. Create the values file

Create the values file values-production.yaml inside the helm/cvmfs-service/ directory.

4.b. Select the CVMFS repositories

helm/cvmfs-service/values-production.yaml
repositories:
- name: software-sion-csquare-run
repository: software.sion.csquare.run

4.c. Configure the CVMFS client

helm/cvmfs-service/values-production.yaml
# ...
configs:
default.local:
mountPath: default.local
contents: |
CVMFS_QUOTA_LIMIT=-1
CVMFS_USE_GEOAPI=no
CVMFS_HTTP_PROXY="DIRECT"
CVMFS_KEYS_DIR="/etc/cvmfs/keys"
CVMFS_SERVER_URL="http://cvmfs.ch1.deepsquare.run/cvmfs/@fqrn@"
CVMFS_USER=root

4.d. Configure the keys

helm/cvmfs-service/values-production.yaml
# ...
keys:
secretName: 'cvmfs-keys-secret'

The keys will be mounted on the /etc/cvmfs/keys directory. If you wish to change the path of each key:

helm/cvmfs-service/values-production.yaml
#...
keys:
secretName: 'cvmfs-keys-secret'
items:
- key: software.sion.csquare.run.pub
path: sion.csquare.run/software.sion.csquare.run.pub

The key will be moved to the path /etc/cvmfs/keys/sion.csquare.run/software.sion.csquare.run.pub.

5. Deploy the CVMFS service

Commit and push:

user@local:/ClusterFactory
git add .
git commit -m "Added CVMFS service"
git push

And deploy the Argo CD application:

user@local:/ClusterFactory
kubectl apply -f argo/provisioning/apps/cvmfs-service-app.yml

6. Mount the repositories to a container

job.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: list-cvmfs
spec:
template:
spec:
containers:
- name: list-cvmfs
image: busybox
command: ['ls', '/cvmfs/software.sion.csquare.run']
volumeMounts:
- mountPath: /cvmfs/software.sion.csquare.run
name: software-sion-csquare-run
readOnly: true
restartPolicy: Never
volumes:
- name: software-sion-csquare-run
hostPath:
path: /cvmfs/cvmfs-k8s/mounts/software.sion.csquare.run
type: Directory
backoffLimit: 0