Deploy on Kubernetes

Learn how to deploy the complete Kafkorama ecosystem including Portal, Gateway, and Apache Kafka® on Kubernetes using a local Minikube setup.

This tutorial shows how to deploy the Kafkorama Portal — a web-based management interface for Kafkorama Gateway clusters, in conjunction with Apache Kafka®, using Kubernetes.

Prerequisites

Before deploying Kafkorama Portal, ensure that you have installed Minikube, a tool for quickly setting up local Kubernetes clusters.

Start Minikube as follows:

minikube start

Check the Kubernetes dashboard as follows:

minikube dashboard

Create Namespace

To organize the Kubernetes resources for Kafkorama, create a dedicated namespace kafkorama as follows.

kubectl create namespace kafkorama

Deploy

Deploy Apache Kafka®

To enable communication between the Kafkorama components and Apache Kafka®, we'll deploy a minimal, single-node Apache Kafka® cluster using the following Kubernetes manifest:

apiVersion: v1
kind: Service
metadata:
  name: kafka-service
  namespace: kafkorama
spec:
  ports:
    - port: 9092
      name: kafka-port
  selector:
    app: kafka
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: kafka
  namespace: kafkorama
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka
  template:
    metadata:
      labels:
        app: kafka
    spec:
      containers:
        - name: kafka
          image: apache/kafka:latest
          ports:
            - containerPort: 9092
          env:
            - name: KAFKA_NODE_ID
              value: "1"
            - name: KAFKA_PROCESS_ROLES
              value: "broker,controller"
            - name: KAFKA_LISTENERS
              value: "PLAINTEXT://:9092,CONTROLLER://:9093"
            - name: KAFKA_ADVERTISED_LISTENERS
              value: "PLAINTEXT://kafka-service.kafkorama.svc.cluster.local:9092"
            - name: KAFKA_CONTROLLER_LISTENER_NAMES
              value: "CONTROLLER"
            - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
              value: "CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT"
            - name: KAFKA_CONTROLLER_QUORUM_VOTERS
              value: "1@127.0.0.1:9093"
            - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
              value: "1"
            - name: KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR
              value: "1"
            - name: KAFKA_TRANSACTION_STATE_LOG_MIN_ISR
              value: "1"
            - name: KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS
              value: "0"
            - name: KAFKA_NUM_PARTITIONS
              value: "3"
            - name: ALLOW_PLAINTEXT_LISTENER
              value: "yes"
            - name: KAFKA_AUTO_CREATE_TOPICS_ENABLE
              value: "true"
          volumeMounts:
            - name: data
              mountPath: /kafka
      volumes:
        - name: data
          emptyDir: {}

Save this manifest as kafka.yaml and run:

kubectl apply -f kafka.yaml

Deploy Kafkorama Portal

The Kafkorama Portal provides a web-based interface to manage Kafkorama Gateway clusters, configure topics, monitor performance, and control access. The following manifest deploys the Portal with persistent storage for the SQLite database:

apiVersion: v1
kind: Service
metadata:
  namespace: kafkorama
  name: kafkorama-portal-external
  labels:
    app: kafkorama
spec:
  type: LoadBalancer
  ports:
    - name: portal-port
      port: 8080
      protocol: TCP
      targetPort: 8080
  selector:
    app: kafkorama-portal
---
apiVersion: v1
kind: Service
metadata:
  name: kafkorama-portal-internal
  namespace: kafkorama
spec:
  type: ClusterIP
  ports:
  - port: 8080
    targetPort: 8080
    protocol: TCP
    name: http
  selector:
    app: kafkorama-portal
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: kafkorama-portal-sqlite-pvc
  namespace: kafkorama
  labels:
    app: kafkorama-portal
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: local-path # Adjust based on your cluster's storage classes
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: kafkorama-portal-config
  namespace: kafkorama
  labels:
    app: kafkorama-portal
data:
  kafkorama-portal.conf: |
    memory=512 MB

    log.folder=logs

    web.listen.ip=0.0.0.0
    web.listen.port=8080

    web.email.verification=false
    #web.email.address=contact@example.com
    #web.email.password=changeme123
    #web.url=https://kafkorama.example.com

    portal.password=my-password

    # the server address used by the web console to connect clients to MigratoryData cluster.
    gateway.servers=192.168.5.190:8800
    gateway.servers.protocol=http://

    # default user
    admin.email=admin@admin.com
    admin.password=password
    admin.organization=kafkorama

    demo.apis = true
    demo.topic = demo
    demo.bootstrap.servers=kafka-service:9092

    db.name=./data/kafkorama-portal.db
    db.type=sqlite
    db.driver.classpath=org.sqlite.JDBC

    renewTokenBeforeSeconds=60
    signature.type=hmac
    # generate a random base64-encoded 32-byte secret key using the command:
    #    openssl rand -base64 32
    signature.hmac.secret=He39zDQW7RdkOcxe3L9qvoSQ/ef40BG6Ro4hrHDjE+U=

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafkorama-portal
  namespace: kafkorama
  labels:
    app: kafkorama-portal
spec:
  replicas: 1  # Single pod 
  selector:
    matchLabels:
      app: kafkorama-portal
  template:
    metadata:
      labels:
        app: kafkorama-portal
    spec:
      containers:
      - name: kafkorama-portal
        image: kafkorama/kafkorama-portal:1.0.2
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
          name: http
        volumeMounts:
        - name: sqlite-storage
          mountPath: /data
        - name: config-volume
          mountPath: /kafkorama-portal/kafkorama-portal.conf
          subPath: kafkorama-portal.conf
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /admin/healthz
            port: 8080
          initialDelaySeconds: 5
        readinessProbe:
          httpGet:
            path: /admin/healthz
            port: 8080
          initialDelaySeconds: 5
      volumes:
      - name: sqlite-storage
        persistentVolumeClaim:
          claimName: kafkorama-portal-sqlite-pvc
      - name: config-volume
        configMap:
          name: kafkorama-portal-config
          items:
          - key: kafkorama-portal.conf
            path: kafkorama-portal.conf
      restartPolicy: Always

This manifest includes:

  • External Service: LoadBalancer to expose the Portal web interface
  • Internal Service: ClusterIP for internal cluster communication
  • PersistentVolumeClaim: 5GB storage for the SQLite database
  • ConfigMap: Portal configuration including JWT settings
  • Deployment: Single instance of the Portal with health checks

Save this manifest as portal.yaml and run:

kubectl apply -f portal.yaml

Deploy Kafkorama Gateway

We need to deploy the Kafkorama Gateway cluster that the Portal will manage:

apiVersion: v1
kind: Service
metadata:
  namespace: kafkorama
  name: kafkorama-gateway-external
  labels:
    app: kafkorama
spec:
  type: LoadBalancer
  ports:
    - name: client-port
      port: 8800
      protocol: TCP
      targetPort: 8800
  selector:
    app: kafkorama
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: kafkorama-gateway-config
  namespace: kafkorama
  labels:
    app: kafkorama
data:
  kafkorama-gateway.conf: |
    LicenseKey = zczuvikp41d2jb2o7j6n

    Memory = 128MB

    Listen = *:8800

    LogLevel=INFO
    LogRotateLimit = 10 MB
    LogFolder = logs

    DocumentRoot = html

    Entitlement=Portal
    Portal.Url = http://kafkorama-portal-internal:8080
    Portal.Password = my-password

    Rest.Enable = true

    Stats.LogInterval=5
    Stats.PrettyPrint=true
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafkorama-gateway
  namespace: kafkorama
spec:
  selector:
    matchLabels:
      app: kafkorama
  replicas: 1 # Desired number of cluster nodes
  template:
    metadata:
      labels:
        app: kafkorama
    spec:
      containers:
        - name: kafkorama-cluster
          imagePullPolicy: Always
          image: kafkorama/kafkorama-gateway:6.0.24
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: KAFKORAMA_GATEWAY_EXTRA_OPTS
              value: "-DNodeName=$(NODE_NAME)"
            - name: KAFKORAMA_GATEWAY_JAVA_GC_LOG_OPTS
              value: "-XX:+PrintCommandLineFlags -XX:+PrintGC -XX:+PrintGCDetails -XX:+DisableExplicitGC -Dsun.rmi.dgc.client.gcInterval=0x7ffffffffffffff0 -Dsun.rmi.dgc.server.gcInterval=0x7ffffffffffffff0 -verbose:gc"
          resources:
            requests:
              memory: "256Mi"
              cpu: "0.5"
          ports:
            - name: client-port
              containerPort: 8800
          readinessProbe:
            tcpSocket:
              port: 8800
            initialDelaySeconds: 20
            failureThreshold: 5
            periodSeconds: 5
          livenessProbe:
            tcpSocket:
              port: 8800
            initialDelaySeconds: 10
            failureThreshold: 5
            periodSeconds: 5
          volumeMounts:
          - name: config-volume
            mountPath: /kafkorama-gateway/kafkorama-gateway.conf
            subPath: kafkorama-gateway.conf
      volumes:
      - name: config-volume
        configMap:
          name: kafkorama-gateway-config
          items:
          - key: kafkorama-gateway.conf
            path: kafkorama-gateway.conf
      restartPolicy: Always

Save this manifest as gateway.yaml and run:

kubectl apply -f gateway.yaml

Deploy Demo Applications (Optional)

To test the complete setup, you can deploy demo applications that showcase real-time messaging capabilities:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-stocks
  namespace: kafkorama
  labels:
    app: demo-stocks
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-stocks
  template:
    metadata:
      labels:
        app: demo-stocks
    spec:
      containers:
      - name: demo-stocks
        image: kafkorama/demo-stocks:latest
        imagePullPolicy: Always
        env:
        - name: SERVER
          value: "kafkorama-gateway-external:8800"
        - name: TOKEN
          value: "eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJkNzcyMTUiLCJwZXJtaXNzaW9ucyI6eyJhbGwiOlsiL2thZmtvcmFtYS9kZW1vLyoiXX0sImFwcCI6ImthZmtvcmFtYS1hcHAtaWQiLCJpYXQiOjE3NDU5MTY2OTgsImV4cCI6MTkwMzcwMTMyOH0.JBomOnpTUXJqwP04leBaHAvc1D5W75TV1vq8pHWPHOs"
        - name: GET_SYMBOLS
          value: "/kafkorama/demo/stocks/symbols"
        - name: TOPICS
          value: "/kafkorama/demo/stocks/AWERQ,/kafkorama/demo/stocks/WERZF,/kafkorama/demo/stocks/QWZAF,/kafkorama/demo/stocks/TEYDF,/kafkorama/demo/stocks/TYUII,/kafkorama/demo/stocks/XCVSD,/kafkorama/demo/stocks/POUVB,/kafkorama/demo/stocks/TYEWD,/kafkorama/demo/stocks/WYWUI"
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "256Mi"
            cpu: "200m"
      restartPolicy: Always
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-traffic
  namespace: kafkorama
  labels:
    app: demo-traffic
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-traffic
  template:
    metadata:
      labels:
        app: demo-traffic
    spec:
      containers:
      - name: demo-traffic
        image: kafkorama/demo-traffic:latest
        imagePullPolicy: Always
        env:
        - name: SERVER
          value: "kafkorama-gateway-external:8800"
        - name: TOKEN
          value: "eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJkNzcyMTUiLCJwZXJtaXNzaW9ucyI6eyJhbGwiOlsiL2thZmtvcmFtYS9kZW1vLyoiXX0sImFwcCI6ImthZmtvcmFtYS1hcHAtaWQiLCJpYXQiOjE3NDU5MTY2OTgsImV4cCI6MTkwMzcwMTMyOH0.JBomOnpTUXJqwP04leBaHAvc1D5W75TV1vq8pHWPHOs"
        - name: SUBJECT
          value: "/kafkorama/demo/traffic/brussels"
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "256Mi"
            cpu: "200m"
      restartPolicy: Always
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-cryptocurrency
  namespace: kafkorama
  labels:
    app: demo-cryptocurrency
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-cryptocurrency
  template:
    metadata:
      labels:
        app: demo-cryptocurrency
    spec:
      containers:
      - name: demo-cryptocurrency
        image: kafkorama/demo-cryptocurrency:latest
        imagePullPolicy: Always
        env:
        - name: SERVER
          value: "kafkorama-gateway-external:8800"
        - name: TOKEN
          value: "eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJkNzcyMTUiLCJwZXJtaXNzaW9ucyI6eyJhbGwiOlsiL2thZmtvcmFtYS9kZW1vLyoiXX0sImFwcCI6ImthZmtvcmFtYS1hcHAtaWQiLCJpYXQiOjE3NDU5MTY2OTgsImV4cCI6MTkwMzcwMTMyOH0.JBomOnpTUXJqwP04leBaHAvc1D5W75TV1vq8pHWPHOs"
        - name: SUBJECT
          value: "/kafkorama/demo/cryptocurrency/rates"
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "256Mi"
            cpu: "200m"
      restartPolicy: Always
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-parking
  namespace: kafkorama
  labels:
    app: demo-parking
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-parking
  template:
    metadata:
      labels:
        app: demo-parking
    spec:
      containers:
      - name: demo-parking
        image: kafkorama/demo-parking:latest
        imagePullPolicy: Always
        env:
        - name: SERVER
          value: "kafkorama-gateway-external:8800"
        - name: TOKEN
          value: "eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJkNzcyMTUiLCJwZXJtaXNzaW9ucyI6eyJhbGwiOlsiL2thZmtvcmFtYS9kZW1vLyoiXX0sImFwcCI6ImthZmtvcmFtYS1hcHAtaWQiLCJpYXQiOjE3NDU5MTY2OTgsImV4cCI6MTkwMzcwMTMyOH0.JBomOnpTUXJqwP04leBaHAvc1D5W75TV1vq8pHWPHOs"
        - name: SUBJECT
          value: "/kafkorama/demo/parking"
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "256Mi"
            cpu: "200m"
      restartPolicy: Always
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-seismic
  namespace: kafkorama
  labels:
    app: demo-seismic
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-seismic
  template:
    metadata:
      labels:
        app: demo-seismic
    spec:
      containers:
      - name: demo-seismic
        image: kafkorama/demo-seismic:latest
        imagePullPolicy: Always
        env:
        - name: SERVER
          value: "kafkorama-gateway-external:8800"
        - name: TOKEN
          value: "eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJkNzcyMTUiLCJwZXJtaXNzaW9ucyI6eyJhbGwiOlsiL2thZmtvcmFtYS9kZW1vLyoiXX0sImFwcCI6ImthZmtvcmFtYS1hcHAtaWQiLCJpYXQiOjE3NDU5MTY2OTgsImV4cCI6MTkwMzcwMTMyOH0.JBomOnpTUXJqwP04leBaHAvc1D5W75TV1vq8pHWPHOs"
        - name: SUBJECT
          value: "/kafkorama/demo/seismic/info"
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "256Mi"
            cpu: "200m"
      restartPolicy: Always

Save this as demos.yaml and apply it:

kubectl apply -f demos.yaml

Namespace Switch

Since the deployment uses the kafkorama namespace, switch to it with the following command:

kubectl config set-context --current --namespace=kafkorama

To switch back to the default namespace, run:

kubectl config set-context --current --namespace=default

Verify Installation

Check that all pods are running:

kubectl get pods

The output should look similar to the following:

NAME                                 READY   STATUS    RESTARTS   AGE
kafka-0                                1/1     Running   0          9m36s
kafkorama-portal-5c94794f9-gzxbs       1/1     Running   0          8m43s
kafkorama-gateway-bd4c75658-5rhx8      1/1     Running   0          8m15s
demo-seismic-76479b5d55-8m9j6          1/1     Running   0          7m31s
demo-traffic-6d4d5d4868-vkrv9          1/1     Running   0          7m31s
demo-stocks-7bdd96c4fc-6lps5           1/1     Running   0          7m31s
demo-cryptocurrency-846789989b-wctzl   1/1     Running   0          7m31s
demo-parking-8679cc64fb-xtl2h          1/1     Running   0          7m31s

Check the services:

kubectl get svc

The output should show:

NAME                        TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)          AGE
kafka-service               ClusterIP      10.43.244.197   <none>         9092/TCP         5m45s
kafkorama-cs                LoadBalancer   10.43.237.196   127.0.0.1      8888:31735/TCP   4m28s
kafkorama-portal-external   LoadBalancer   10.43.158.123   127.0.0.1      8080:30856/TCP   3m01s
kafkorama-portal-internal   ClusterIP      10.43.199.88    <none>         8080/TCP         3m01s
kafkorama-demos-service     LoadBalancer   10.43.145.67    127.0.0.1      3000:32189/TCP   1m35s

To view the logs of the Portal pod, run:

kubectl logs kafkorama-portal-5c94794f9-gzxbs

Test Installation

To expose the LoadBalancer Services, use the Minikube tunnel command:

minikube tunnel

Access Kafkorama Portal

Open your browser and navigate to the Portal's external IP address (typically http://127.0.0.1:8080). You should see the Kafkorama Portal login page.

Use the default admin credentials specified in the ConfigMap:

  • Email: admin@admin.com
  • Password: password

Access Kafkorama Gateway

The Gateway is accessible at ip 127.0.0.1:8800.

View Logs

Portal logs:

kubectl logs -f deployment/kafkorama-portal

Gateway logs:

kubectl logs -f deployment/kafkorama-gateway

Kafka logs:

kubectl logs -f statefulset/kafka

Check Pod Status

kubectl describe pod <pod-name>

Access Pod Shell

kubectl exec -it <pod-name> -- /bin/bash

Uninstall

To remove all Kubernetes resources created for this deployment, run:

kubectl delete namespace kafkorama

Then, switch back to the default namespace:

kubectl config set-context --current --namespace=default

Build Real-time Apps

With your Kafkorama ecosystem deployed, you can:

  1. Explore the Portal: Use the web interface to manage clusters, topics, and users
  2. Connect Clients: Use Kafkorama SDKs to build real-time applications
  3. Integrate with Kafka: Publish messages to Kafka topics that will be delivered to connected clients
  4. Monitor Performance: Use the Portal's dashboard to track system metrics

The Portal serves as your central control hub for managing the entire Kafkorama infrastructure, making it easy to deploy, monitor, and scale your real-time messaging platform.

© 2025 MigratoryData. All rights reserved.

Table of Contents