Skip to main content

How to deploy xCAT

xCAT deployment graph

Helm and Docker resources

The Helm resources are stored on ClusterFactory Git Repository.

The Dockerfile is described in the git repository SquareFactory/xcat-rocky.

An Docker image can be pulled with:

docker pull ghcr.io/squarefactory/xcat-rocky:latest

1. Deploy Namespace and AppProject

user@local:/ClusterFactory
kubectl apply -f argo/provisioning/

2. Creating a StorageClass or PersistentVolume

We will use NFS. Feel free to use another type of storage. We recommend at least 100 GB since the storage is used to store the root file system of the operating system images.

argo/provisioning/volumes/storage-class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: xcat-nfs
namespace: provisioning
labels:
app: xcat
topology.kubernetes.io/region: <FILL ME> # <country code>-<city>
topology.kubernetes.io/zone: <FILL ME> # <country code>-<city>-<index>
provisioner: nfs.csi.k8s.io
parameters:
server: <FILL ME> # IP or host
share: <FILL ME> # /srv/nfs/k8s/xcat
mountPermissions: '0775'
mountOptions:
- hard
- nfsvers=4.1
- noatime
- nodiratime
volumeBindingMode: Immediate
reclaimPolicy: Retain
allowedTopologies:
- matchLabelExpressions:
- key: topology.kubernetes.io/zone
values:
- <FILL ME> # <country code>-<city>-<index>
user@local:/ClusterFactory
kubectl apply -f argo/provisioning/volumes/storage-class.yaml

3. Editing xcat-app.yml to use the fork

argo.example/provisioning/apps/xcat-app.yml > spec > source
source:
# You should have forked this repo. Change the URL to your fork.
repoURL: git@github.com:<your account>/ClusterFactory.git
targetRevision: HEAD
path: helm/xcat
helm:
releaseName: xcat

valueFiles:
- values-production.yaml

4. Adding custom values

tip

Read the values.yaml to see all the default values.

4.a. Add the values file to the chart

Create values-production.yaml inside the helm/xcat/ directory.

4.b. Selecting a zone

xCAT will use the host network. Make sure that xCAT stays on the same network by using the nodeSelector.

Your nodes should be annotated with topology.kubernetes.io/region and topology.kubernetes.io/zone.

helm/xcat/values-production.yaml
nodeSelector:
topology.kubernetes.io/region: <FILL ME> # <country code>-<city>
topology.kubernetes.io/zone: <FILL ME> # <country code>-<city>-<index>

4.c. Network configuration

xCAT will be connected to the host network using the IPVLAN CNI plugin. Make sure that Multus CNI is already installed.

tip

Check with:

> kubectl get daemonset -n kube-system kube-multus-ds
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-multus-ds 1 1 1 1 1 <none> 28d

If it isn't already deployed, you can deploy Multus with:

kubectl apply -f https://raw.githubusercontent.com/k8snetworkplumbingwg/multus-cni/master/deployments/multus-daemonset-thick-plugin.yml

xcat network diagram

Similar to configuring the network interface of a Virtual Machine, you must change these fields:

title=
# ...

net:
# Kubernetes host interface
masterInterface: eth0
mode: l2
type: ipvlan

# https://www.cni.dev/plugins/current/ipam/static/
ipam:
type: static
addresses:
- address: 192.168.0.3/24
gateway: 192.168.0.1
routes:
- dst: 0.0.0.0/0
- dst: 192.168.0.0/24
gw: 192.168.0.1

# https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-dns-config
dns:
nameservers:
- 127.0.0.1

Use IPAM static since xCAT has a ISC DHCP server.

4.d. Volume configuration

Edit the values accordingly:

helm/xcat/values-production.yaml
# ...
persistence:
storageClassName: 'xcat-nfs'
accessModes: ['ReadWriteOnce']
size: 50Gi

5. Deploy xCAT

Commit and push:

user@local:/ClusterFactory
git add .
git commit -m "Added xCAT application and values"
git push

And deploy:

user@local:/ClusterFactory
kubectl apply -f argo/provisioning/apps/xcat-app.yml