Skip to main content

How to deploy xCAT

xCAT deployment graph

Helm and Docker resources​

The Helm resources are stored on ClusterFactory Git Repository.

The Dockerfile is described in the git repository SquareFactory/xcat-rocky.

An Docker image can be pulled with:

docker pull ghcr.io/squarefactory/xcat-rocky:latest

1. Deploy Namespace and AppProject​

user@local:/ClusterFactory
kubectl apply -f argo/provisioning/

2. Creating a StorageClass or PersistentVolume​

We will use NFS. Feel free to use another type of storage. We recommend at least 100 GB since the storage is used to store the root file system of the operating system images.

argo/provisioning/volumes/storage-class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: xcat-nfs
  namespace: provisioning
  labels:
    app: xcat
    topology.kubernetes.io/region: <FILL ME> # <country code>-<city>
    topology.kubernetes.io/zone: <FILL ME> # <country code>-<city>-<index>
provisioner: nfs.csi.k8s.io
parameters:
  server: <FILL ME> # IP or host
  share: <FILL ME> # /srv/nfs/k8s/xcat
  mountPermissions: '0775'
mountOptions:
  - hard
  - nfsvers=4.1
  - noatime
  - nodiratime
volumeBindingMode: Immediate
reclaimPolicy: Retain
allowedTopologies:
  - matchLabelExpressions:
      - key: topology.kubernetes.io/region
        values:
          - <FILL ME> # <country code>-<city>
user@local:/ClusterFactory
kubectl apply -f argo/provisioning/volumes/storage-class.yaml

3. Editing xcat-app.yaml to use the fork​

argo.example/provisioning/apps/xcat-app.yaml > spec > source
source:
  # You should have forked this repo. Change the URL to your fork.
  repoURL: git@github.com:<your account>/ClusterFactory.git
  # You should use your branch too.
  targetRevision: HEAD
  path: helm/xcat
  helm:
    releaseName: xcat

    valueFiles:
      - values-production.yaml

4. Adding custom values​

tip

Read the values.yaml to see all the default values.

4.a. Add the values file to the chart​

Create values-production.yaml inside the helm/xcat/ directory.

4.b. Selecting a zone​

xCAT will use the host network. Make sure that xCAT stays on the same network by using the nodeSelector.

Your nodes should be annotated with topology.kubernetes.io/region and topology.kubernetes.io/zone.

helm/xcat/values-production.yaml
nodeSelector:
  topology.kubernetes.io/region: <FILL ME> # <country code>-<city>
  topology.kubernetes.io/zone: <FILL ME> # <country code>-<city>-<index>

4.c. Network configuration​

xCAT will be connected to the host network using the IPVLAN CNI plugin. Make sure that Multus CNI is already installed.

tip

Check with:

> kubectl get daemonset -n kube-system kube-multus-ds
NAME             DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
kube-multus-ds   1         1         1       1            1           <none>          28d

If it isn't already deployed, you can deploy Multus with:

kubectl apply -f https://raw.githubusercontent.com/k8snetworkplumbingwg/multus-cni/master/deployments/multus-daemonset-thick-plugin.yaml

xcat network diagram

Similar to configuring the network interface of a Virtual Machine, you must change these fields:

title=
# ...

net:
  # Kubernetes host interface
  masterInterface: eth0
  mode: l2
  type: ipvlan

  # https://www.cni.dev/plugins/current/ipam/static/
  ipam:
    type: static
    addresses:
      - address: 192.168.0.3/24
        gateway: 192.168.0.1
    routes:
      - dst: 0.0.0.0/0

  # https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-dns-config
  dns:
    nameservers:
      - 127.0.0.1

Use IPAM static since xCAT has a ISC DHCP server.

4.d. Volume configuration​

Edit the values accordingly:

helm/xcat/values-production.yaml
# ...
persistence:
  storageClassName: 'xcat-nfs'
  accessModes: ['ReadWriteOnce']
  size: 50Gi

5. Deploy xCAT​

Commit and push:

user@local:/ClusterFactory
git add .
git commit -m "Added xCAT application and values"
git push

And deploy:

user@local:/ClusterFactory
kubectl apply -f argo/provisioning/apps/xcat-app.yaml