Skip to main content

Deploying Eggplant Generator with Kubernetes

This page describes how to deploy Eggplant Generator in a Kubernetes environment. It includes system and software requirements specific to Kubernetes deployments. You will also need to install Eggplant IAM (Keycloak).

tip

Before proceeding with the installation of Eggplant Generator in Kubernetes containers, you should ensure the engineers in your organization are Certified Kubernetes Administrators (https://www.cncf.io/training/certification/cka/) or have equivalent experience.

Software Recommendations for Eggplant Generator Deployments with Kubernetes​

note

See the Prerequisites page for information about the required hardware and memory for Eggplant Generator.

Before installation, you will need to meet the following software requirements:

SoftwareConfiguration
Kubernetes cluster1.29/1.32
helmv3.8.0
ingress-nginx1.10.0 (chart version 4.10.0)
NVIDIA CUDA Driver12.x. See CUDA Installation Guide for Linux for more information.
NVIDIA Container Toolkit1.17.3 See Installing the NVIDIA Container Toolkit — NVIDIA Container Toolkit for more information.
k8s device pluginv0.17.0. See GitHub - NVIDIA/k8s-device-plugin: NVIDIA device plugin for Kubernetes for more information.
Eggplant IAMSee install Eggplant IAM (Keycloak).
info

Eggplant Generator currently requires 9 persistent volumes for storing data, each of which is configured with 8Gi (Gigabytes) by default. If your Kubernetes cluster does not have dynamic volume provisioning in place, you will need to manually create the required volumes before proceeding.

The versions of the software shown in the table above are the versions tested with Eggplant Generator.

Deploy Eggplant Generator with Kubernetes​

tip

Eggplant Generator container images are large. Please be sure to allow time to pull them during deployment.

  1. Download the required software. Refer to the Software Requirements table above for the list of what you need.

  2. Create a new namespace to install Eggplant Generator:

    kubectl create ns gai

  3. Create a secret called keycloak-realm-installer within the namespace:

    kubectl create secret generic keycloak-realm-installer \
       --from-literal=REALM_INSTALLER_CLIENT_SECRET=<your secret> \
       --namespace=gai
    info

    This secret is used to configure Eggplant IAM (Keycloak) with the Eggplant Generator specific configuration. It needs to match the auto generated value from when Keycloak was installed. If Eggplant IAM is installed in the same namespace as where you are installing Generator then this secret should already exist and this step can be skipped. Otherwise if Eggplant IAM has been installed in another namespace you can use the command below to obtain it:

    kubectl get secret keycloak-realm-installer -n <your keycloak namespace> -o json | jq .data.REALM_INSTALLER_CLIENT_SECRET -r | base64 -d

  4. Eggplant Generator requires Transport Layer Security (TLS) / HTTPS and will not function correctly using plain HTTP. Create a Kubernetes secret with the certificate and key path from the local machine in the target namespace using the following command:

    kubectl --namespace gai create secret tls gai-tls-secret --cert=path/to/cert/file --key=path/to/key/file
    tip

    Any public TLS certificates can be used. If you do not have a preferred TLS certificate provider we suggest Lets Encrypt which provides free TLS certs. Please refer to the documentation on options for obtaining and automatically renewing certificates though Lets Encrypt.

    note

    If you intend to offload the TLS termination elsewhere (e.g. on a load balancer) or have default TLS certs configured on the ingress controller then you can skip this step and omit the two set global.ingress.tls lines below.

  5. Deploy Eggplant Generator with the default configuration using the script below.

    helm upgrade --install --namespace gai \
gai \
oci://harbor.dai.eggplant.cloud/charts/gai \
--version 0.7.1 \
--set global.ingress.host="genai.mydomain.com" \
--set realmConfigInstaller.keycloakHostname="keycloak.mydomain.com" \
--set realmConfigInstaller.keycloakExistingSecret="secretname" \
--set realmConfigInstaller.keycloakExistingSecretKey="secretkey" \
--set global.ingress.tls[0].secretName=gai-tls-secret \
--set global.ingress.tls[0].hosts[0]="genai.mydomain.com"
info

The value for realmConfigInstaller.keycloakExistingSecret should be the name of an existing Kubernetes secret within the namespace that contains the Eggplant IAM / Keycloak password. realmConfigInstaller.keycloakExistingSecretKey should be the name of the key with the secret that contains the password.

Optional Customization​

Configuring Persistent Volume Size​

Eggplant GenAI currently uses 9 persistent volumes, each with a default size of 8Gi (Gigabytes). Update the size based on your needs.

helm upgrade --install --namespace gai \
--create-namespace gai \
--repo oci://harbor.dai.eggplant.cloud/charts/gai \
--version 0.7.1 \
--set global.ingress.host="genai.mydomain.com" \
--set realmConfigInstaller.keycloakHostname="keycloak.mydomain.com" \
--set realmConfigInstaller.keycloakExistingSecret="secretname" \
--set realmConfigInstaller.keycloakExistingSecretKey="secretkey" \
--set global.persistence.neo4jVolumeSize=8Gi \
--set global.persistence.postgresVolumeSize=8Gi \
--set global.persistence.gaiContextDocsVolumeSize=8Gi \
--set global.persistence.gaiDownloadsVolumeSize=8Gi \
--set global.persistence.gaiSpacyVolumeSize=8Gi \
--set global.persistence.gaiRequirementsVolumeSize=8Gi

Launch Eggplant Generator​

Prior to using Eggplant Generator for the first time you will need to configure a user. You can use the initial username and password that was created as part of the Keycloak installation to login to https://keycloak.mydomain.com/auth/admin/eggplant/console/#/eggplant/users. From there you can create a user with the generator:admin: or generator:user roles.

Launch Gen AI by opening your web browser and navigating to the following URL (for example, https://genai.mydomain.com/collections).

Last updated on