How to Build a Custom Kubernetes Image for Deploying Imply

Since its debut in 2014, Kubernetes has grown to become the leading container orchestration technology today, used by nearly 5.6 million developers worldwide as of 2021. Given its popularity, it was only natural for Imply to enable the deployment and automation of its database products on Kubernetes.

This tutorial will demonstrate how to build customized Imply images to deploy Imply applications within a Kubernetes environment. By building on the official documentation, we hope to provide a comprehensive, step-by-step approach to integrating custom files into images.

Overview and requirements

For this walkthrough, assume that we have encountered a scenario where a customer has just deployed Imply in their environment, but needs to execute a series of custom steps. If they wish to automate this process, they can use the custom files feature offered within Imply Manager.

Before they begin, however, there are a number of prerequisites, including:

An existing Imply image in repository /local machine
Their desired custom user scripts that will be executed at the initialization of the application
Access to the image repository, used to store the new image and where the system can pull the new image from at the time of deployment
A Kubernetes environment in which to run the application

The process

Step 1

In this case, we are using the standard image from imply/manager:2023.08, which lives on the local machine, as a template to build the custom Imply image. First, use vi Dockerfile to create a Docker file, and then use the below commands to both create a new image.

FROM imply/manager:2023.08
COPY user-init /mnt/var/user/user-init
COPY user-init /home/user-init

Now, add user-init files into the image under the path /mnt/var/ and /home/ using the following steps:

#!/bin/bash
sudo mkdir -p /home/venkat
sudo chmod 777 /home/venkat
cd /home/venkat
touch test.txt
chmod 777 test.txt

Step 2

Build the Docker image file by using the Docker file content from step 1. Here’s how I accessed existing Docker images in my local machine.

venkat@Venkata-Ramana-Reddy-Bheemcherla ~ % docker images
REPOSITORY                    TAG       IMAGE ID       CREATED        SIZE
venky2023/manager_new         2023.08   0f22a1f90379   2 weeks ago    2.21GB
venky2023/manager             2023      b287c9edd8bc   2 weeks ago    2.21GB
nginx                         latest    ab73c7fd6723   3 months ago   192MB
venky2023/manager             2023.08   9cd94c63e2e3   4 months ago   2.21GB
imply/manager                 2023.08   9cd94c63e2e3   4 months ago   2.21GB
gcr.io/k8s-minikube/kicbase   v0.0.40   f52519afe5f6   5 months ago   1.1GB
kindest/node                  <none>    6e360fda99b5   5 months ago   850MB

In order to create a new image, use the Docker build command with the tag docker build -t venky2023/manager_custom:2023.08 . This ensures that we are tagging the image with repository details so that I can push the image to the repository.

venkat@Venkata-Ramana-Reddy-Bheemcherla ~ % docker build -t venky2023/manager_custom:2023.08 . 
[+] Building 4.0s (9/9) FINISHED                                                                                                                                                                                                docker:desktop-linux

 => [internal] load .dockerignore                                                                                                                                                                                                               0.0s

 => => transferring context: 2B                                                                                                                                                                                                                 0.0s

 => [internal] load build definition from Dockerfile                                                                                                                                                                                            0.0s

 => => transferring dockerfile: 134B                                                                                                                                                                                                            0.0s

 => [internal] load metadata for docker.io/imply/manager:2023.08                                                                                                                                                                                3.9s

 => [auth] imply/manager:pull token for registry-1.docker.io                                                                                                                                                                                    0.0s

 => CACHED [1/3] FROM docker.io/imply/manager:2023.08@sha256:648b5e802d8770c029e705bb564247a56e5635b6a6a9d9be3c4b46c84bb1652e                                                                                                                   0.0s

 => [internal] load build context                                                                                                                                                                                                               0.0s

 => => transferring context: 1.09kB                                                                                                                                                                                                             0.0s
 => [2/3] COPY user-init /mnt/var/user/user-init                                                                                                                                                                                                0.0s

 => [3/3] COPY user-init /home/user-init                                                                                                                                                                                                        0.0s

 => exporting to image                                                                                                                                                                                                                          0.0s

 => => exporting layers                                                                                                                                                                                                                         0.0s

 => => writing image sha256:05a42e5715bc1b9aa9cebf4260bd76643339888ff3772e6197b4d0a6ec14c1e3                                                                                                                                                    0.0s

 => => naming to docker.io/venky2023/manager_custom:2023.08                                                                                                                                                                                     0.0s

venkat@Venkata-Ramana-Reddy-Bheemcherla ~ %

As you can see, following the successful implementation of the previous build command, the new image now lives alongside the local Docker images.

venkat@Venkata-Ramana-Reddy-Bheemcherla ~ % docker images 
REPOSITORY                    TAG       IMAGE ID       CREATED         SIZE
venky2023/manager_custom      2023.08   05a42e5715bc   5 minutes ago   2.21GB
venky2023/manager_new         2023.08   0f22a1f90379   2 weeks ago     2.21GB
venky2023/manager             2023      b287c9edd8bc   2 weeks ago     2.21GB
nginx                         latest    ab73c7fd6723   3 months ago    192MB
venky2023/manager             2023.08   9cd94c63e2e3   4 months ago    2.21GB
imply/manager                 2023.08   9cd94c63e2e3   4 months ago    2.21GB
gcr.io/k8s-minikube/kicbase   v0.0.40   f52519afe5f6   5 months ago    1.1GB
kindest/node                  <none>    6e360fda99b5   5 months ago    850MB
venkat@Venkata-Ramana-Reddy-Bheemcherla ~ %

Step 3

Now, you can push the custom image to repository using docker push

venky2023/manager_custom:2023.08

venkat@Venkata-Ramana-Reddy-Bheemcherla ~ % docker push venky2023/manager_custom:2023.08

The push refers to repository [docker.io/venky2023/manager_custom]

7a23ab508be9: Pushed 
314986644c85: Pushed 
67ba3c3c959f: Mounted from venky2023/manager_new 
263b24eaac83: Mounted from venky2023/manager_new 
7c65d38ed9f4: Mounted from venky2023/manager_new 
6a4e2aca3307: Mounted from venky2023/manager_new 
80af3d6c88bb: Mounted from venky2023/manager_new 
572be9cf6e3e: Mounted from venky2023/manager_new 
6e8805e33669: Mounted from venky2023/manager_new 
bced7f97b85b: Mounted from venky2023/manager_new 
be28649c5aed: Mounted from venky2023/manager_new 
a04da79dd562: Mounted from venky2023/manager_new 
e73158310fc7: Mounted from venky2023/manager_new 
2023.08: digest: sha256:f1bdb5b705282d86b63c99f55b1097d8ca33a50ae46870cdf939673131fbccf3 size: 3039

Following this, you should also be able to see the image in your docker hub repository.

Step 4

Now, you can modify the Imply helm chart values.yaml file in order to pull the custom image from the repository where it is stored. For the purposes of this article, the repository is located at Venky2023/manager_custom:2023:09.

Here is a brief code snippet of what it should look like:

nameOverride:
fullnameOverride:
images:
  manager:
    repository: venky2023/manager_custom
    tag: "2023.08"
  agent:
    repository: imply/agent
    tag: "v20"
  pullPolicy: IfNotPresent

Step 5

By adding the below variable, changing it from /mnt/var/ to /home, we are overriding the default location in the Imply manager section of values.yaml.

extraEnv:
  - name: IMPLY_MANAGER_USER_RESOURCE_DIR
    value: /home

Step 6

Now, you can install the Imply application in the Kubernetes environment using the helm install imply ./imply command—and validate the results.

venkat@Venkata-Ramana-Reddy-Bheemcherla imply-kubernetes % helm install imply ./imply

NAME: imply
LAST DEPLOYED: Mon Dec 11 11:35:17 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:

Release name: imply

It may take a few minutes for the deployment to become available.

Manager
-------
Once all the pods are running, you can access the Manager console at http://127.0.0.1:9097 by running:

  kubectl --namespace default port-forward svc/imply-manager-int 9097

Pivot and Druid Console
-----------------------
Once the Imply Manager reports that the cluster is running, you can set up port forwarding by running:

  kubectl --namespace default port-forward svc/imply-query 8888 9095

Then connect to:

  - Pivot: http://127.0.0.1:9095
  - Druid: http://127.0.0.1:8888

To connect through the node port:

  - Find the node IP address:

      kubectl get nodes --namespace default -o jsonpath='{.items[0].status.addresses[?(@.type=="ExternalIP")].address}'

  - Find the node ports for Pivot and the Druid console:

      kubectl get --namespace default -o jsonpath="{.spec.ports[*].nodePort}" services imply-query

  - Connect to:

      http://{NODE_IP}:{NODE_PORT}

Lastly, update the user-init details in Imply Manager, and restart the server.

In order to validate the results, you should log into the master pod, data pod, and query pod and check whether the commands from the user-init file were executed. If the user-init file executed successfully, then you will be able to see the /home/venkat folder in all three pods with full permissions.

As you can see in the screencap below, the master pod has the /home/venkat folder, as well as the test.txt file—which means that the custom script file executed successfully.

A quick note: one common failure occurs during the execution of the Docker build command, in the form of an error message like

failed to solve: imply/manager:2023.08: error getting credentials - err: exec: "docker-credential-desktop": executable file not found in $PATH, out: ``

In order to resolve this issue, you should remove credsStore from ~/.docker/config.json.

Conclusion

With the explosive growth of containers worldwide, Kubernetes has become a vital component for orchestrating, or controlling, fleets of these discrete software units. By using the process outlined above, you can not only deploy your Imply products through Kubernetes, but even automate custom steps, thus freeing up time and energy for you to try other things.