Rancher Kubernetes Engine 2 (RKE2) Installation

Prerequisites

TBD

Setup Kubernetes “First” Master Node

XXXXXXXXXXXXXXXXXXXXX

1. XXXX

2. XXXX

   Note

   XXXXXX sudo <command>, XXXXXXXXXXXX

   sudo -i

3. XXXX

   XXXXXXXXXXXXXXXXXX

   export INSTALL_RKE2_VERSION="v1.32.5+rke2r1"

   XXXXXXXXXXXXXXXXXX

   curl -sfL https://get.rke2.io | sh -

4. XXXX

   cp -f /usr/local/share/rke2/rke2-cis-sysctl.conf /etc/sysctl.d/60-rke2-cis.conf
   systemctl restart systemd-sysctl

5. XXXX

   useradd -r -c "etcd user" -s /sbin/nologin -M etcd -U
   mkdir -p /var/lib/rancher/rke2/server/db/etcd
   chown etcd:etcd /var/lib/rancher/rke2/server/db/etcd

6. Create Configuration for Master Node

   Create necessary directory

   mkdir -p /etc/rancher/rke2/

   Note

   You are about to perform File Configuration on RKE2.

   See Server Configuration Reference, for any extra configurations that you may need.

   Danger

   Also see Critical Configuration Values which are important values that it must be the same value on all servers in the cluster.

   Failure to do so will cause new servers to fail to join the cluster.

   • agent-token
   • service-cidr
   • cluster-cidr
   • cluster-dns
   • cluster-domain
   • disable-cloud-controller
   • disable-kube-proxy
   • egress-selector-mode

   These are also critical values but not mentioned in the document.

   • tls-san

   Anything not mentioned below will accept defaults from RKE2, See Server Configuration Reference.

   The following configuration will:

   • Add Node Taint to Master Nodes.
   • Using Container Network Interface (CNI) Plugin with calico.
   • Disable Cloud Controller Manager
   • Validate system configuration against the selected benchmark with CIS Benchmark.
   • Write Kubeconfig file with the 600 file permission.

   cat <<EOF > /etc/rancher/rke2/config.yaml
   node-taint:
     - "CriticalAddonsOnly=true:NoExecute"
     - "node-role.kubernetes.io/control-plane:NoSchedule"
     - "node-role.kubernetes.io/etcd:NoExecute"
   
   cni: calico
   
   disable-cloud-controller: true
   
   profile: "cis"
   write-kubeconfig-mode: "0600"
   
   EOF

   IF you will have multiple Master Nodes and distribute traffic through a Load Balancer. (See Load Balancers section)

   You need to also include that endpoint/domain in TLS as Subject Alternative Name (SAN) as well.

   export KUBERNETES_API_HA_ENDPOINT="XXX.YYY.ZZZ.AAA"

   cat <<EOF >> /etc/rancher/rke2/config.yaml
   tls-san:
     - "${KUBERNETES_API_HA_ENDPOINT}"
   
   EOF

   Tip

   You can double check the configuration with cat /etc/rancher/rke2/config.yaml

7. XXXXX

   systemctl enable rke2-server.service
   systemctl start rke2-server.service

8. WAIT XXXXXXXXXXXXXXXXXXXXXXXXX

   XXXXXXXXXXXXXXXXXXXXXXXXXX

   systemctl status rke2-server.service

   ● rke2-server.service - Rancher Kubernetes Engine v2 (server)
    Loaded: loaded (/usr/local/lib/systemd/system/rke2-server.service; enabled; preset: enabled)
    Active: active (running) since Tue XXXX-XX-XX XX:XX:XX UTC; XXs ago
      Docs: https://github.com/rancher/rke2#readme
   Process: 23765 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
   Process: 23767 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
   Process: 23770 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   ... (deducted)

   journalctl -xe -u rke2-server.service

   Note

   You can optionally append -f to the command to be

   journalctl -xe -u rke2-server.service -f

   So you can continuously follow the log from rke2-server.service

   ... (deducted)
   MON XX XX:XX:XX <your-master-node-name> systemd[1]: Started rke2-server.service - Rancher Kubernetes Engine v2 (server).
   ░░ Subject: A start job for unit rke2-server.service has finished successfully
   ░░ Defined-By: systemd
   ░░ Support: http://www.ubuntu.com/support
   ░░
   ░░ A start job for unit rke2-server.service has finished successfully.
   ░░
   ░░ The job identifier is XXXX.
   XXX XX XX:XX:XX <your-master-node-name> rke2[23773]: time="XXXX-XX-XXTXX:XX:XXZ" level=info msg="Reconciliation of ETCDSnapshotFile resources complete"
   XXX XX XX:XX:XX <your-master-node-name> rke2[23773]: time="XXXX-XX-XXTXX:XX:XXZ" level=info msg="Reconciling ETCDSnapshotFile resources"
   XXX XX XX:XX:XX <your-master-node-name> rke2[23773]: time="XXXX-XX-XXTXX:XX:XXZ" level=info msg="Starting k3s.cattle.io/v1, Kind=Addon controller"
   XXX XX XX:XX:XX <your-master-node-name> rke2[23773]: time="XXXX-XX-XXTXX:XX:XXZ" level=info msg="Creating deploy event broadcaster"
   XXX XX XX:XX:XX <your-master-node-name> rke2[23773]: time="XXXX-XX-XXTXX:XX:XXZ" level=info msg="Reconciliation of ETCDSnapshotFile resources complete"
   ... (deducted)

9. XXXXXX

   export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
   /var/lib/rancher/rke2/bin/kubectl get nodes

   NAME                      STATUS   ROLES                       AGE    VERSION
   <your-master-node-name>   Ready    control-plane,etcd,master   XmXs   v1.32.5+rke2r1

10. XXXXXXXXXXXX

    cat /var/lib/rancher/rke2/server/node-token

    Caution

    XXXXXXXXXXXXXXXXXXXXXXX

Setup Kubernetes Master Nodes (Peer Nodes)

XXXXXXXXXXXXXXXXXXXXX

1. XXXX

2. XXXX

   Note

   XXXXXX sudo <command>, XXXXXXXXXXXX

   sudo -i

3. XXXX

   XXXXXXXXXXXXXXXXXX

   export INSTALL_RKE2_VERSION="v1.32.5+rke2r1"

   XXXXXXXXXXXXXXXXXX

   curl -sfL https://get.rke2.io | sh -

4. XXXX

   cp -f /usr/local/share/rke2/rke2-cis-sysctl.conf /etc/sysctl.d/60-rke2-cis.conf
   systemctl restart systemd-sysctl

5. XXXX

   useradd -r -c "etcd user" -s /sbin/nologin -M etcd -U
   mkdir -p /var/lib/rancher/rke2/server/db/etcd
   chown etcd:etcd /var/lib/rancher/rke2/server/db/etcd

6. Create Configuration for Master Node

   Create necessary directory

   mkdir -p /etc/rancher/rke2/

   Note

   You are about to perform File Configuration on RKE2.

   See Server Configuration Reference, for any extra configurations that you may need.

   Danger

   Also see Critical Configuration Values which are important values according to RKE2 that it must be the same value on all servers in the cluster.

   Failure to do so will cause new servers to fail to join the cluster.

   • agent-token
   • service-cidr
   • cluster-cidr
   • cluster-dns
   • cluster-domain
   • disable-cloud-controller
   • disable-kube-proxy
   • egress-selector-mode

   These are also critical values but not mentioned in the document.

   • tls-san

   Anything not mentioned below will accept defaults from RKE2, See Server Configuration Reference.

   The following configuration will:

   • Add Node Taint to Master Nodes.
   • Using Container Network Interface (CNI) Plugin with calico.
   • Disable Cloud Controller Manager
   • Validate system configuration against the selected benchmark with CIS Benchmark.
   • Write Kubeconfig file with the 600 file permission.

   cat <<EOF > /etc/rancher/rke2/config.yaml
   node-taint:
     - "CriticalAddonsOnly=true:NoExecute"
     - "node-role.kubernetes.io/control-plane:NoSchedule"
     - "node-role.kubernetes.io/etcd:NoExecute"
   
   cni: calico
   
   disable-cloud-controller: true
   
   profile: "cis"
   write-kubeconfig-mode: "0600"
   
   EOF

   IF you will have multiple Master Nodes and distribute traffic through a Load Balancer. (See Load Balancers section)

   You need to also include that endpoint/domain in TLS as Subject Alternative Name (SAN) as well.

   export KUBERNETES_API_HA_ENDPOINT="XXX.YYY.ZZZ.AAA"

   cat <<EOF >> /etc/rancher/rke2/config.yaml
   tls-san:
     - "${KUBERNETES_API_HA_ENDPOINT}"
   
   server: https://${KUBERNETES_API_HA_ENDPOINT}:9345
   
   EOF

7. XXXXX

   export KUBERNETES_JOIN_TOKEN="CHANGEME-to-RandomSecret"

   Caution

   Example of Secret: <67-Chars-Long>::server:<32-Chars-Long>

   cat <<EOF >> /etc/rancher/rke2/config.yaml
   token: ${KUBERNETES_JOIN_TOKEN}
   
   EOF

   Tip

   You can double check the configuration with cat /etc/rancher/rke2/config.yaml

8. XXXXX

   systemctl enable rke2-server.service
   systemctl start rke2-server.service

9. WAIT XXXXXXXXXXXXXXXXXXXXXXXXX

   XXXXXXXXXXXXXXXXXXXXXXXXXX

   systemctl status rke2-server.service

   ● rke2-server.service - Rancher Kubernetes Engine v2 (server)
    Loaded: loaded (/usr/local/lib/systemd/system/rke2-server.service; enabled; preset: enabled)
    Active: active (running) since Tue XXXX-XX-XX XX:XX:XX UTC; XXs ago
      Docs: https://github.com/rancher/rke2#readme
   Process: 23765 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
   Process: 23767 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
   Process: 23770 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   ... (deducted)

   journalctl -xe -u rke2-server.service

   Note

   You can optionally append -f to the command to be

   journalctl -xe -u rke2-server.service -f

   So you can continuously follow the log from rke2-server.service

   ... (deducted)
   MON XX XX:XX:XX <your-master-node-name> systemd[1]: Started rke2-server.service - Rancher Kubernetes Engine v2 (server).
   ░░ Subject: A start job for unit rke2-server.service has finished successfully
   ░░ Defined-By: systemd
   ░░ Support: http://www.ubuntu.com/support
   ░░
   ░░ A start job for unit rke2-server.service has finished successfully.
   ░░
   ░░ The job identifier is XXXX.
   XXX XX XX:XX:XX <your-master-node-name> rke2[23773]: time="XXXX-XX-XXTXX:XX:XXZ" level=info msg="Reconciliation of ETCDSnapshotFile resources complete"
   XXX XX XX:XX:XX <your-master-node-name> rke2[23773]: time="XXXX-XX-XXTXX:XX:XXZ" level=info msg="Reconciling ETCDSnapshotFile resources"
   XXX XX XX:XX:XX <your-master-node-name> rke2[23773]: time="XXXX-XX-XXTXX:XX:XXZ" level=info msg="Starting k3s.cattle.io/v1, Kind=Addon controller"
   XXX XX XX:XX:XX <your-master-node-name> rke2[23773]: time="XXXX-XX-XXTXX:XX:XXZ" level=info msg="Creating deploy event broadcaster"
   XXX XX XX:XX:XX <your-master-node-name> rke2[23773]: time="XXXX-XX-XXTXX:XX:XXZ" level=info msg="Reconciliation of ETCDSnapshotFile resources complete"
   ... (deducted)

10. XXXXXX

    export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
    /var/lib/rancher/rke2/bin/kubectl get nodes

    NAME                        STATUS   ROLES                       AGE    VERSION
    <your-master-node-name-1>   Ready    control-plane,etcd,master   XmXs   v1.32.5+rke2r1
    <your-master-node-name-2>   Ready    control-plane,etcd,master   XmXs   v1.32.5+rke2r1
    <your-master-node-name-3>   Ready    control-plane,etcd,master   XmXs   v1.32.5+rke2r1

Setup Kubernetes Worker Nodes

XXXXXXXXXXXXXXXXXXXXX

1. XXXX

2. XXXX

   Note

   XXXXXX sudo <command>, XXXXXXXXXXXX

   sudo -i

3. XXXX

   XXXXXXXXXXXXXXXXXX

   export INSTALL_RKE2_VERSION="v1.32.5+rke2r1"
   export INSTALL_RKE2_TYPE="agent"

   XXXXXXXXXXXXXXXXXX

   curl -sfL https://get.rke2.io | sh -

4. XXXX

   sudo cp -f /usr/local/share/rke2/rke2-cis-sysctl.conf /etc/sysctl.d/60-rke2-cis.conf
   sudo systemctl restart systemd-sysctl

5. XXXX

   mkdir -p /etc/rancher/rke2/

6. XXXXX

   Note

   XXXXXXXXXXXXXXXXXX

   export KUBERNETES_API_HA_ENDPOINT="XXX.YYY.ZZZ.AAA"

   export KUBERNETES_JOIN_TOKEN="CHANGEME-to-RandomSecret"

   Caution

   Example of Secret: <67-Chars-Long>::server:<32-Chars-Long>

   cat <<EOF >> /etc/rancher/rke2/config.yaml
   server: https://${KUBERNETES_API_HA_ENDPOINT}:9345
   
   token: ${KUBERNETES_JOIN_TOKEN}
   
   EOF

   Tip

   You can double check the configuration with cat /etc/rancher/rke2/config.yaml

7. XXXXX

   systemctl enable rke2-agent.service
   systemctl start rke2-agent.service

8. WAIT XXXXXXXXXXXXXXXXXXXXXXXXX

   XXXXXXXXXXXXXXXXXXXXXXXXXX

   systemctl status rke2-agent.service

   TBD
   ... (deducted)

   journalctl -xe -u rke2-agent.service

   Note

   You can optionally append -f to the command to be

   journalctl -xe -u rke2-agent.service -f

   So you can continuously follow the log from rke2-agent.service

   ... (deducted)
   TBD
   ... (deducted)

Ensure Connection for All Master Nodes and Worker Nodes

1. XXXXXX

2. XXXXXX

   export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
   /var/lib/rancher/rke2/bin/kubectl get nodes

   NAME                        STATUS   ROLES                       AGE    VERSION
   <your-master-node-name-1>   Ready    control-plane,etcd,master   XmXs   v1.32.5+rke2r1
   <your-master-node-name-2>   Ready    control-plane,etcd,master   XmXs   v1.32.5+rke2r1
   <your-master-node-name-3>   Ready    control-plane,etcd,master   XmXs   v1.32.5+rke2r1
   <your-worker-node-name-1>   Ready    <none>                      XmXs   v1.32.5+rke2r1
   <your-worker-node-name-2>   Ready    <none>                      XmXs   v1.32.5+rke2r1
   <your-worker-node-name-3>   Ready    <none>                      XmXs   v1.32.5+rke2r1
   ... (deducted)

Finished?

Next step, will begin the Kubernetes Preparations, Good luck! … 🚀

Kubernetes Preparations