Deploying Eggplant Generator with Kubernetes

This page describes how to deploy Eggplant Generator in a Kubernetes environment. It includes system and software requirements specific to Kubernetes deployments. You will also need to install Eggplant IAM (Keycloak).

tip

Before proceeding with the installation of Eggplant Generator in Kubernetes containers, you should ensure the engineers in your organization are Certified Kubernetes Administrators (https://www.cncf.io/training/certification/cka/) or have equivalent experience.

Software Recommendations for Eggplant Generator Deployments with Kubernetes

note

See the Prerequisites page for information about the required hardware and memory for Eggplant Generator.

Before installation, you will need to meet the following software requirements:

Software	Configuration
Kubernetes cluster	1.29/1.32
helm	v3.8.0
ingress-nginx	1.10.0 (chart version 4.10.0)
NVIDIA CUDA Driver	12.x. See CUDA Installation Guide for Linux for more information.
NVIDIA Container Toolkit	1.17.3 See Installing the NVIDIA Container Toolkit — NVIDIA Container Toolkit for more information.
k8s device plugin	v0.17.0. See GitHub - NVIDIA/k8s-device-plugin: NVIDIA device plugin for Kubernetes for more information.
Eggplant IAM	See install Eggplant IAM (Keycloak).

info

Eggplant Generator currently requires 9 persistent volumes for storing data, each of which is configured with 8Gi (Gigabytes) by default. If your Kubernetes cluster does not have dynamic volume provisioning in place, you will need to manually create the required volumes before proceeding.

The versions of the software shown in the table above are the versions tested with Eggplant Generator.

Deploy Eggplant Generator with Kubernetes

tip

Eggplant Generator container images are large. Please be sure to allow time to pull them during deployment.

1. Download the required software. Refer to the Software Requirements table above for the list of what you need.

2. Create a new namespace to install Eggplant Generator:

   kubectl create ns gai

3. Create a secret called keycloak-realm-installer within the namespace:

   info

   If Eggplant IAM is installed in the same namespace as where you are installing Generator then this secret should already exist and this step can be skipped.

   Otherwise if Eggplant IAM has been installed in another namespace you can use the commands below to obtain the secret from the Eggplant IAM namepace and create the equivalent secret in the Eggplant Generator namespace.

   This secret is used to configure Eggplant IAM (Keycloak) with the Eggplant Generator specific configuration. It needs to match the auto generated value from when Keycloak was installed.

   kubectl get secret keycloak-realm-installer -n <your keycloak namespace> -o json | jq .data.REALM_INSTALLER_CLIENT_SECRET -r | base64 -d

   kubectl create secret generic keycloak-realm-installer \
      --from-literal=REALM_INSTALLER_CLIENT_SECRET=<your secret> \
      --namespace=gai

4. Create a secret called keycloak within the namespace:

   info

   If Eggplant IAM is installed in the same namespace as where you are installing Generator then this secret should already exist and this step can be skipped.

   Otherwise if Eggplant IAM has been installed in another namespace you can use the commands below to obtain the secret from the Eggplant IAM namepace and create the equivalent secret in the Eggplant Generator namespace.

   This secret is used to configure Eggplant IAM (Keycloak) with the Eggplant Generator specific configuration. It needs to match the auto generated value from when Keycloak was installed.

   kubectl get secret keycloak -n <your keycloak namespace> -o json | jq .data.password -r | base64 -d

   kubectl create secret generic keycloak \
      --from-literal=password=<your secret> \
      --namespace=gai

5. Eggplant Generator requires Transport Layer Security (TLS) / HTTPS and will not function correctly using plain HTTP. Create a Kubernetes secret with the certificate and key path from the local machine in the target namespace using the following command:

   kubectl --namespace gai create secret tls gai-tls-secret --cert=path/to/cert/file --key=path/to/key/file
   tip

   Any public TLS certificates can be used. If you do not have a preferred TLS certificate provider we suggest Lets Encrypt which provides free TLS certs. Please refer to the documentation on options for obtaining and automatically renewing certificates though Lets Encrypt.

   note

   If you intend to offload the TLS termination elsewhere (e.g. on a load balancer) or have default TLS certs configured on the ingress controller then you can skip this step and omit the two set global.ingress.tls lines below.

6. Deploy Eggplant Generator with the default configuration using the script below.

   helm upgrade --install --namespace gai \
   gai \
   oci://harbor.dai.eggplant.cloud/charts/gai \
   --version 0.10.11 \
   --set global.ingress.host="genai.mydomain.com" \
   --set realmConfigInstaller.keycloakHostname="keycloak.mydomain.com" \
   --set realmConfigInstaller.keycloakExistingSecret="keycloak" \
   --set realmConfigInstaller.keycloakExistingSecretKey="password" \
   --set global.ingress.tls[0].secretName=gai-tls-secret \
   --set global.ingress.tls[0].hosts[0]="genai.mydomain.com"

info

The value for realmConfigInstaller.keycloakExistingSecret should be the name of an existing Kubernetes secret within the namespace that contains the Eggplant IAM / Keycloak password. realmConfigInstaller.keycloakExistingSecretKey should be the name of the key with the secret that contains the password.

Optional Customization

Configuring Persistent Volume Size

Eggplant Generator currently uses 9 persistent volumes, each with a default size of 8Gi (Gigabytes). Update the size based on your needs.

helm upgrade --install --namespace gai \
--create-namespace gai \
--repo oci://harbor.dai.eggplant.cloud/charts/gai \
--version 0.10.11 \
--set global.ingress.host="genai.mydomain.com" \
--set realmConfigInstaller.keycloakHostname="keycloak.mydomain.com" \
--set realmConfigInstaller.keycloakExistingSecret="secretname" \
--set realmConfigInstaller.keycloakExistingSecretKey="secretkey" \
--set global.persistence.neo4jVolumeSize=8Gi \
--set global.persistence.postgresVolumeSize=8Gi \
--set global.persistence.gaiContextDocsVolumeSize=8Gi \
--set global.persistence.gaiDownloadsVolumeSize=8Gi \
--set global.persistence.gaiSpacyVolumeSize=8Gi \
--set global.persistence.gaiRequirementsVolumeSize=8Gi

Launch Generator in Kubernetes

See Launching Eggplant Generator for information about how to launch Eggplant Generator.

Backup and Restore

You must regularly back up configuration and results data from your Generator installation. Data that needs to be backed up is stored in PostgreSQL as well as a PVC attached to the gai deployment.

How you back up this data will depend on how you've configured your deployment, but here we provide an example of how both can be backed up in the default installation shown at the start of this document.

Backup Generator data

1. Create a backup of the keycloak database.

   kubectl -n <backup_namespace> exec "sts/postgres" \
   -- /bin/sh -c \
   'export PGPASSWORD=$POSTGRES_PASSWORD && pg_dump --username postgres -d keycloak --clean --create' \
   >keycloak.dump

2. Create a backup of the eggplant_skill database.

   kubectl -n <backup_namespace> exec "deploy/postgres-db" \
   -- /bin/sh -c \
   'export PGPASSWORD=$POSTGRES_PASSWORD && pg_dump --username postgres -d eggplant_skill --clean --create' \
   >eggplant_skill.dump

3. Create a backup of all the Generator asset files.

   kubectl -n <backup_namespace> exec "deploy/gai" -- tar czf - -C /etc/sde downloads requirement spacy upload_docs >assets.tar.gz

Restore Generator data

info

The below assumes you have the three files (keycloak.dump, eggplant_skill.dump and assets.tar.gz) generated as part of the backup procedure above in your working directory.

1. Scale down pods to clear database connections.

   kubectl -n <restore_namespace> scale "sts/keycloak" --replicas=0
   kubectl -n <restore_namespace> wait --for=delete pod -l "app.kubernetes.io/component=keycloak" --timeout=30m
   kubectl -n <restore_namespace> scale "deploy/gai" --replicas=0
   kubectl -n <restore_namespace> wait --for=delete pod -l "app=gai" --timeout=30m

2. Restore the `keycloakk database.

   kubectl -n <restore_namespace> exec -i "sts/postgres" \
   -- /bin/sh -c \
   'export PGPASSWORD=$POSTGRES_PASSWORD && psql --username=postgres \
   --dbname postgres \
   --file -' <keycloak.dump

3. Restore the eggplant_skill database.

   tip

   This database can take a very long time to restore.

   kubectl -n <restore_namespace> exec -i "deploy/postgres-db" \
   -- /bin/sh -c \
   'export PGPASSWORD=$POSTGRES_PASSWORD && psql --username postgres \
   --dbname postgres \
   --file -' <eggplant_skill.dump

4. Scale the pods backup.

   kubectl -n <restore_namespace> scale "sts/keycloak" --replicas=1
   kubectl -n <restore_namespace> scale "deploy/gai" --replicas=1
   kubectl -n <restore_namespace> rollout status "deploy/gai" --timeout=30m

5. Restore assets to gai PVC.

   kubectl -n <restore_namespace> exec -i deploy/gai \
   -- /bin/sh -c \
   "tar -xzv -C /etc/sde --file -" <./assets.tar.gz

Upgrading Generator in Kubernetes Deployments

info

The general procedure for upgrading is the same as any Helm release which is to run helm upgrade. Before upgrading, please refer to the Backup and Restore section on how to create and restore backups.

Each release may have specific additional steps. So before applying this procedure, please review the notes below for the upgrade you're performing.

Upgrading 1.1.0 to 1.2.0

1. Upgrade Eggplant Generator by running the following command:

helm upgrade --install --namespace gai \
--create-namespace gai \
--repo oci://harbor.dai.eggplant.cloud/charts/gai \
--version 0.10.11 \
--set global.ingress.host="genai.mydomain.com" \
--set realmConfigInstaller.keycloakHostname="keycloak.mydomain.com" \
--set realmConfigInstaller.keycloakExistingSecret="secretname" \
--set realmConfigInstaller.keycloakExistingSecretKey="secretkey" \
--set global.persistence.neo4jVolumeSize=8Gi \
--set global.persistence.postgresVolumeSize=8Gi \
--set global.persistence.gaiContextDocsVolumeSize=8Gi \
--set global.persistence.gaiDownloadsVolumeSize=8Gi \
--set global.persistence.gaiSpacyVolumeSize=8Gi \
--set global.persistence.gaiRequirementsVolumeSize=8Gi

Uninstall Generator

You can uninstall by running the command below:

helm delete gai --namespace gai