Skip to content

This is the Experimental version (Latest). It is under active development and may change. For the most reliable documentation, use the version selector in the top-right to switch to Stable, or click here to go to the Stable version's homepage.

Opstella Workers

Last updated:

🟢 Management

Opstella Workers are Microservices to integrate with DevSecOps Tools, Observability Tools, Keycloak, and Kubernetes.

As you have determined number of tools/instruments in Software Resources Preparation/Determine List of Opstella Workers Section

Number of Opstella Workers will depend on DevSecOps Tools, Observability Tools enabled within the entire platform as it could be vary by your requirement.

For Example:

Hard Requirement: Opstella Keycloak Service - Install worker-keycloak

Hard Requirement: Kubernetes - Install worker-kubernetes

DevSecOps:

  • ArgoCD: Install worker-argocd
  • DefectDojo: Install worker-defectdojo
  • GitLab: Install worker-gitlab
  • Headlamp: Install worker-headlamp
  • Harbor: Install worker-harbor
  • SonarQube: Install worker-sonarqube
  • Vault: Install worker-vault

Observability:

  • Grafana LGTM Stack: Install worker-grafana, worker-loki, worker-tempo

In conclusion, this is the list of Opstella Workers that required to install

  • worker-keycloak
  • worker-kubernetes
  • worker-argocd
    • Due to Opstella Architecture and Kubernetes Clusters requirements that we followed the Reference Architecture, it must have worker-argocd for 🟦 Non-Production Workload and 🟥 Production Workload.
      • worker-argocd-nonprod
      • worker-argocd-prod
  • worker-defectdojo
  • worker-gitlab
  • worker-headlamp
  • worker-harbor
  • worker-sonarqube
  • worker-vault
  • worker-grafana
  • worker-loki
  • worker-tempo

List of Opstella Workers will be

#!/bin/bash
export OPSTELLA_ENABLED_INSTRUMENTS=(keycloak kubernetes argocd-nonprod argocd-prod defectdojo gitlab headlamp-nonprod headlamp-prod harbor sonarqube vault grafana loki tempo)

  1. Connect to 🟢 Management Kubernetes Cluster ; i.e w/ Kubeconfig File

    Ensure you have defined and loaded your Global Shell Variables as described in Shell Variables.

    Terminal window
    source $BASE_WORKING_DIR/shell-values/kubernetes/management_cluster.vars.sh

    Ensure BASE_DOMAIN is defined as per the Shell Variables guide.

  2. Helm Values Preparation

    Important Configurations

    • image.repository, image.tag: Set your Container Image Location, Version

    (Ensure OPSTELLA_REGISTRY is loaded from your variables)

    Terminal window
    for KEY in "${OPSTELLA_ENABLED_INSTRUMENTS[@]}"
    do
    if echo $KEY | grep -e "^argocd"; then
    export WORKER_NAME=$KEY
    export IMAGE_NAME=argocd
    elif echo $KEY | grep -e "^headlamp"; then
    export WORKER_NAME=$KEY
    export IMAGE_NAME=kubernetes
    elif echo $KEY | grep -e "^kubernetes"; then
    export WORKER_NAME=$KEY
    export IMAGE_NAME=kubernetes
    else
    export WORKER_NAME=$KEY
    export IMAGE_NAME=$KEY
    fi
    cat <<EOF > $BASE_WORKING_DIR/helm-values/opstella-worker-${WORKER_NAME}-full-values.yaml
    image:
    repository: ${OPSTELLA_REGISTRY}/platform/worker-${IMAGE_NAME}
    tag: ${OPSTELLA_WORKER_IMAGE_TAG}
    pullPolicy: Always
    nameOverride: worker-${WORKER_NAME}
    fullnameOverride: worker-${WORKER_NAME}
    serviceAccount:
    name:
    imagePullSecrets:
    - name: registry-secret
    env:
    - name: WORKER_NAME
    value: ${WORKER_NAME}
    - name: INVOKE_URL
    value: http://localhost:3500/v1.0/invoke/opstella-core/method
    - name: STATE_STORE_URL
    value: http://localhost:3500/v1.0/state/statestore
    - name: INVOKE_URL_WORKER
    value: http://localhost:3500/v1.0/invoke
    containerPorts: 3000
    service:
    port: 3000
    # Waiting time in seconds for shutting down pod after sent SIGTERM
    # terminationGracePeriodSeconds: 30
    healthCheck:
    enabled: true
    liveness:
    httpGet:
    path: "/healthcheck"
    port: 3000
    initialDelaySeconds: 180
    periodSeconds: 30
    readiness:
    httpGet:
    path: "/"
    port: 3000
    initialDelaySeconds: 10
    periodSeconds: 5
    podAnnotations:
    dapr.io/enabled: "true"
    dapr.io/app-id: "${WORKER_NAME}"
    dapr.io/app-port: "3000"
    dapr.io/enable-api-logging: "true"
    dapr.io/config: "config"
    dapr.io/sidecar-seccomp-profile-type: "RuntimeDefault"
    podSecurityContext:
    fsGroup: 1000
    securityContext:
    seccompProfile:
    type: RuntimeDefault
    capabilities:
    drop: ["ALL"]
    runAsNonRoot: true
    privileged: false
    allowPrivilegeEscalation: false
    runAsGroup: 1000
    runAsUser: 1000
    EOF
    done

    So that’s why worker-argocd are needed to deploy a separate instances as

    • worker-argocd-nonprod
    • worker-argocd-prod

    However, its container image can be shared.

    In the previous Helm Values Configuration section, we have intented to ease the operation but mistakenly created the incorrect configuration. This step will fix it.

    Terminal window
    sed -i "s#repository:.*#repository: $OPSTELLA_REGISTRY/worker-argocd#g" $BASE_WORKING_DIR/helm-values/opstella-worker-argocd-nonprod-full-values.yaml
    sed -i "s#repository:.*#repository: $OPSTELLA_REGISTRY/worker-argocd#g" $BASE_WORKING_DIR/helm-values/opstella-worker-argocd-prod-full-values.yaml
  1. Install Opstella Workers

    Install Helm Release using local opstella-platform Helm Chart

    Terminal window
    for key in "${OPSTELLA_ENABLED_INSTRUMENTS[@]}"
    do
    helm upgrade --install worker-${key} \
    oci://asia-southeast1-docker.pkg.dev/opstella-dev/opstella-charts/generic-deployment \
    --version 0.3.15 \
    --namespace opstella-system \
    -f $BASE_WORKING_DIR/helm-values/opstella-worker-${key}-full-values.yaml
    done
  1. Get Pod Status - Opstella Workers

    Terminal window
    kubectl get pods -n opstella-system

    Opstella Workers should be Running

    NAME READY STATUS RESTARTS AGE
    ... (deducted)
    worker-argocd-nonprod-XXXXXXX-YYYYY 1/1 Running 0 XdXh
    worker-argocd-prod-XXXXXXX-YYYYY 1/1 Running 0 XdXh
    worker-defectdojo-XXXXXXX-YYYYY 1/1 Running 0 XdXh
    worker-gitlab-XXXXXXX-YYYYY 1/1 Running 0 XdXh
    worker-grafana-XXXXXXX-YYYYY 1/1 Running 0 XdXh
    worker-harbor-XXXXXXX-YYYYY 1/1 Running 0 XdXh
    ... (deducted)

When the tools use a private or self-signed certificate

Section titled “When the tools use a private or self-signed certificate”

The workers call every tool over HTTPS and verify the certificate, but they fail in ways that do not name TLS:

WorkerWhat it looks like
worker-argocd-*panic: nil pointer in usecase/argocd.GetToken — the login returned no token, and the result is dereferenced anyway
worker-harborPod stays 1/2 and restarts; the log does show x509: certificate signed by unknown authority
othersno visible symptom until a task fails

Build one CA bundle from the system store plus every certificate the workers have to trust, and mount it over the system bundle:

Terminal window
# Start from the system bundle so the public CAs keep working
kubectl run ca-dump --rm -i --restart=Never --image=alpine:3.20 -- \
cat /etc/ssl/certs/ca-certificates.crt > /tmp/ca-bundle.crt
# Append each internal certificate the workers must trust
cat /path/to/wildcard.crt >> /tmp/ca-bundle.crt
kubectl create configmap opstella-ca-bundle --namespace opstella-system \
--from-file=ca-certificates.crt=/tmp/ca-bundle.crt \
--dry-run=client -o yaml | kubectl apply -f -

Add the mount to the values file of every worker, so it survives helm upgrade. The generic-deployment chart takes it in the same shape opstella-core-values.yaml already uses for its custom-trusted-ca volume:

volumes: true
volumesConfigMap:
- name: ca-bundle
configMap:
name: opstella-ca-bundle
volumeMounts:
- name: ca-bundle
mountPath: /etc/ssl/certs/ca-certificates.crt
subPath: ca-certificates.crt

💡 Mount over ca-certificates.crt itself. Adding a separate file next to it changes nothing — the Go and OpenSSL defaults read the bundle, not every file in the directory.

Verify from inside a worker:

Terminal window
kubectl exec --namespace opstella-system deploy/worker-harbor -c generic-deployment -- \
wc -l /etc/ssl/certs/ca-certificates.crt

💡 The container in every Opstella Deployment is named generic-deployment, not after the worker — kubectl exec -c and kubectl set image both need that name.

Vault token Secret is read from the vault Namespace

Section titled “Vault token Secret is read from the vault Namespace”

Each worker that talks to Vault reads a Secret named vault-token-<company-slug> from the Namespace vault, which is not configurable. If Vault itself was installed elsewhere — the Opstella guide puts it in devsecops-system — create the Namespace and copy the Secret across:

Terminal window
kubectl create namespace vault --dry-run=client -o yaml | kubectl apply -f -
kubectl get secret vault-token-${ORGANIZATION_NAME} --namespace devsecops-system -o yaml \
| sed 's/namespace: devsecops-system/namespace: vault/' \
| kubectl apply --namespace vault -f -

Finished?

Use the below navigation to proceed