Skip to main content
Version: GAI 26.1

Deploying Eggplant Generator with Kubernetes

This page describes how to deploy Eggplant Generator in a Kubernetes environment. It includes system and software requirements specific to Kubernetes deployments. You will also need to install Eggplant IAM (Keycloak).

tip

Before proceeding with the installation of Eggplant Generator in Kubernetes containers, you should ensure the engineers in your organization are Certified Kubernetes Administrators (https://www.cncf.io/training/certification/cka/) or have equivalent experience.

Software Recommendations for Eggplant Generator Deployments with Kubernetes​

note

See the Prerequisites page for information about the required hardware and memory for Eggplant Generator.

Before installation, you will need to meet the following software requirements:

SoftwareConfiguration
Kubernetes cluster1.29/1.32
helmv3.8.0
ingress-nginx1.10.0 (chart version 4.10.0)
NVIDIA CUDA Driver12.x. See CUDA Installation Guide for Linux for more information.
NVIDIA Container Toolkit1.17.3 See Installing the NVIDIA Container Toolkit — NVIDIA Container Toolkit for more information.
k8s device pluginv0.17.0. See GitHub - NVIDIA/k8s-device-plugin: NVIDIA device plugin for Kubernetes for more information.
Eggplant IAMSee install Eggplant IAM (Keycloak).
info

Eggplant Generator currently requires 9 persistent volumes for storing data, each of which is configured with 8Gi (Gigabytes) by default. If your Kubernetes cluster does not have dynamic volume provisioning in place, you will need to manually create the required volumes before proceeding.

The versions of the software shown in the table above are the versions tested with Eggplant Generator.

Deploy Eggplant Generator with Kubernetes​

tip

Eggplant Generator container images are large. Please be sure to allow time to pull them during deployment.

  1. Download the required software. Refer to the Software Requirements table above for the list of what you need.

  2. Create a new namespace to install Eggplant Generator:

    kubectl create ns gai

  3. Create a secret called keycloak within the namespace:

    info

    If Eggplant IAM is installed in the same namespace as where you are installing Generator then this secret should already exist and this step can be skipped.

    Otherwise if Eggplant IAM has been installed in another namespace you can use the commands below to obtain the secret from the Eggplant IAM namepace and create the equivalent secret in the Eggplant Generator namespace.

    This secret is used to configure Eggplant IAM (Keycloak) with the Eggplant Generator specific configuration. It needs to match the value from when Keycloak was installed.

    kubectl get secret keycloak -n <your keycloak namespace> -o json | jq .data.password -r | base64 -d
    kubectl create secret generic keycloak \
       --from-literal=password=<your secret> \
       --namespace=gai

  4. Create a secret called postgres-credentials for PostgreSQL database access:

    info

    The PostgreSQL secret contains credentials required for both the PostgreSQL database and the Eggplant Generator application to communicate with the database.

    First, set the PostgreSQL password as an environment variable (use a strong, randomly generated password):

    export POSTGRES_PASSWORD=<secure-postgres-password>

    Then create the secret:

    kubectl create secret generic postgres-credentials \
       --from-literal=password=$POSTGRES_PASSWORD \
       --from-literal=username=postgres \
       --from-literal=database=eggplant_skill \
       --namespace=gai
    warning

    Use a strong, randomly generated password for the PostgreSQL instance. This same POSTGRES_PASSWORD environment variable will be used later for backup and restore operations.

  5. Eggplant Generator requires Transport Layer Security (TLS) / HTTPS and will not function correctly using plain HTTP. Create a Kubernetes secret with the certificate and key path from the local machine in the target namespace using the following command:

    kubectl --namespace gai create secret tls gai-tls-secret --cert=path/to/cert/file --key=path/to/key/file
    tip

    Any public TLS certificates can be used. If you do not have a preferred TLS certificate provider we suggest Lets Encrypt which provides free TLS certs. Please refer to the documentation on options for obtaining and automatically renewing certificates though Lets Encrypt.

    note

    If you intend to offload the TLS termination elsewhere (e.g. on a load balancer) or have default TLS certs configured on the ingress controller then you can skip this step and omit the two set global.ingress.tls lines below.

  6. Deploy Eggplant Generator with the default configuration using the script below.

    helm upgrade --install --namespace gai \
gai \
oci://harbor.dai.eggplant.cloud/charts/gai \
--version 0.12.11 \
--set global.ingress.host="genai.mydomain.com" \
--set realmConfigInstaller.keycloakHostname="keycloak.mydomain.com" \
--set global.ingress.tls[0].secretName=gai-tls-secret \
--set global.ingress.tls[0].hosts[0]="genai.mydomain.com" \
--set postgres.existingSecret=postgres-credentials \
--set postgres.existingSecretKey=password

Optional Customization​

Configuring Persistent Volume Size​

Eggplant Generator currently uses 9 persistent volumes, each with a default size of 8Gi (Gigabytes). Update the size based on your needs.

helm upgrade --install --namespace gai \
--create-namespace gai \
--repo oci://harbor.dai.eggplant.cloud/charts/gai \
--version 0.12.11 \
--set global.ingress.host="genai.mydomain.com" \
--set realmConfigInstaller.keycloakHostname="keycloak.mydomain.com" \
--set realmConfigInstaller.keycloakExistingSecret="secretname" \
--set realmConfigInstaller.keycloakExistingSecretKey="secretkey" \
--set postgres.existingSecret=postgres-credentials \
--set postgres.existingSecretKey=password \
--set global.persistence.neo4jVolumeSize=8Gi \
--set global.persistence.postgresVolumeSize=8Gi \
--set global.persistence.gaiContextDocsVolumeSize=8Gi \
--set global.persistence.gaiDownloadsVolumeSize=8Gi \
--set global.persistence.gaiSpacyVolumeSize=8Gi \
--set global.persistence.gaiRequirementsVolumeSize=8Gi

Launch Generator in Kubernetes​

See Launching Eggplant Generator for information about how to launch Eggplant Generator.

Backup and Restore​

You must regularly back up configuration and results data from your Generator installation. Data that needs to be backed up is stored in PostgreSQL as well as a PVC attached to the gai deployment.

How you back up this data will depend on how you've configured your deployment, but here we provide an example of how both can be backed up in the default installation shown at the start of this document.

info

For backup operations, you will need to extract the PostgreSQL password from the postgres-credentials secret and set it as an environment variable. You can do this with the following command:

export POSTGRES_PASSWORD=$(kubectl -n gai get secret postgres-credentials -o jsonpath='{.data.password}' | base64 -d)

Then use $POSTGRES_PASSWORD in all the backup and restore commands below.

Backup Generator data​

  1. Create a backup of the keycloak database.

    kubectl -n <backup_namespace> exec "sts/eggplant-iam-postgres" \
    -- /bin/sh -c \
    'export PGPASSWORD=$POSTGRES_PASSWORD && pg_dump --username postgres -d keycloak --clean --create' \
    >keycloak.dump

  2. Create a backup of the eggplant_skill database.

    info

    This database backup may contain large amounts of data and can take a significant amount of time to complete.

    kubectl -n <backup_namespace> exec "deploy/postgres-db" \
    -- /bin/sh -c \
    'export PGPASSWORD=$POSTGRES_PASSWORD && pg_dump --username postgres -d eggplant_skill --clean --create' \
    >eggplant_skill.dump

  3. Create a backup of all the Generator asset files.

    kubectl -n <backup_namespace> exec "deploy/gai" -- tar czf - -C /etc/sde downloads requirement spacy upload_docs >assets.tar.gz

Restore Generator data​

info

The below assumes you have the three files (keycloak.dump, eggplant_skill.dump and assets.tar.gz) generated as part of the backup procedure above in your working directory.

You will also need the PostgreSQL password from the postgres-credentials secret as described above.

  1. Scale down pods to clear database connections.

    kubectl -n <restore_namespace> scale "sts/keycloak" --replicas=0
    kubectl -n <restore_namespace> wait --for=delete pod -l "app.kubernetes.io/component=keycloak" --timeout=30m
    kubectl -n <restore_namespace> scale "deploy/gai" --replicas=0
    kubectl -n <restore_namespace> wait --for=delete pod -l "app=gai" --timeout=30m

  2. Restore the `keycloak database.

    kubectl -n <restore_namespace> exec -i "sts/eggplant-iam-postgres" \
    -- /bin/sh -c \
    'export PGPASSWORD=$POSTGRES_PASSWORD && psql --username=postgres \
    --dbname postgres \
    --file -' <keycloak.dump

  3. Restore the eggplant_skill database.

    tip

    This database can take a very long time to restore. Make sure you have set the POSTGRES_PASSWORD environment variable as described in the backup section above.

    kubectl -n <restore_namespace> exec -i "deploy/postgres-db" \
    -- /bin/sh -c \
    'export PGPASSWORD=$POSTGRES_PASSWORD && psql --username postgres \
    --dbname postgres \
    --file -' <eggplant_skill.dump

  4. Scale the pods backup.

    kubectl -n <restore_namespace> scale "sts/keycloak" --replicas=1
    kubectl -n <restore_namespace> scale "deploy/gai" --replicas=1
    kubectl -n <restore_namespace> rollout status "deploy/gai" --timeout=30m

  5. Restore assets to gai PVC.

    kubectl -n <restore_namespace> exec -i deploy/gai \
    -- /bin/sh -c \
    "tar -xzv -C /etc/sde --file -" <./assets.tar.gz

Upgrading Generator in Kubernetes Deployments​

info

The general procedure for upgrading is the same as any Helm release which is to run helm upgrade. Before upgrading, please refer to the Backup and Restore section on how to create and restore backups.

Each release may have specific additional steps. So before applying this procedure, please review the notes below for the upgrade you're performing.

Upgrading 1.2.0 to 26.1.0​

  1. Upgrade Eggplant Generator by running the following command:
helm upgrade --install --namespace gai \
--create-namespace gai \
--repo oci://harbor.dai.eggplant.cloud/charts/gai \
--version 0.12.11 \
--set global.ingress.host="genai.mydomain.com" \
--set realmConfigInstaller.keycloakHostname="keycloak.mydomain.com" \
--set realmConfigInstaller.keycloakExistingSecret="secretname" \
--set realmConfigInstaller.keycloakExistingSecretKey="secretkey" \
--set postgres.existingSecret=postgres-credentials \
--set postgres.existingSecretKey=password \
--set global.persistence.neo4jVolumeSize=8Gi \
--set global.persistence.postgresVolumeSize=8Gi \
--set global.persistence.gaiContextDocsVolumeSize=8Gi \
--set global.persistence.gaiDownloadsVolumeSize=8Gi \
--set global.persistence.gaiSpacyVolumeSize=8Gi \
--set global.persistence.gaiRequirementsVolumeSize=8Gi

Uninstall Generator​

You can uninstall by running the command below:

helm delete gai --namespace gai
Last updated on