Logo Event-Driven Architecture & Industrial Edge EN ES

Kaoto + Apache Camel

What is Apache Camel?

Apache Camel is an open source integration framework that implements Enterprise Integration Patterns (EIP). It lets you connect diverse systems using a broad library of components.

Red Hat build of Apache Camel for Quarkus is the supported distribution for OpenShift — cloud-native, fast startup, GraalVM native.

Red Hat Camel K (serverless) reached End of Life in June 2025. The migration path is Camel for Quarkus, which offers the same route model but with traditional deployment.

What is Kaoto?

Kaoto is a low-code visual designer for Apache Camel routes. It lets you create and edit integrations by dragging components — without writing YAML/XML by hand.

Kaoto is available as a VS Code extension (redhat.vscode-kaoto) and comes pre-installed in the DevSpaces workspaces in this cluster.

Camel Route for the CDC Pipeline

The integration route consumes CDC events from Kafka and sends email notifications to Mailpit:

- route:
    id: cdc-customer-notification
    from:
      uri: kafka:cdc.public.customers
      parameters:
        brokers: cdc-cluster-kafka-bootstrap.kafka-cdc.svc:9092
        groupId: camel-cdc-consumer
        autoOffsetReset: latest
    steps:
      - log:
          message: "CDC event received: ${body}"
      - unmarshal:
          json:
            library: jackson
      - choice:
          when:
            - jsonpath:
                expression: "$.payload.op"
              simple: "${body[payload][op]} == 'c'"
              steps:
                - setHeader:
                    name: subject
                    constant: "New Customer Created"
                - setBody:
                    simple: |
                      {"from":{"name":"CDC Pipeline","email":"cdc@neuralbank.io"},
                       "to":[{"name":"Admin","email":"admin@neuralbank.io"}],
                       "subject":"New Customer Created",
                       "text":"Customer created in the database."}
          otherwise:
            steps:
              - setBody:
                  simple: |
                    {"from":{"name":"CDC Pipeline","email":"cdc@neuralbank.io"},
                     "to":[{"name":"Admin","email":"admin@neuralbank.io"}],
                     "subject":"Customer Record Changed",
                     "text":"A change was detected on the customers table."}
      - marshal:
          json:
            library: jackson
      - to:
          uri: http://n8n-mailpit.openshift-lightspeed.svc:8025/api/v1/send

Using Kaoto in DevSpaces

  1. Open DevSpaces: https://devspaces.

  2. Create a new workspace or open an existing one

  3. The Kaoto extension is pre-installed automatically

  4. Open any .camel.yaml file — Kaoto shows the visual view

  5. You can drag components, change parameters, and save — the YAML updates automatically

Camel components used in this pipeline

Component Usage

camel-kafka

Consumes messages from the CDC topic

camel-jackson

Marshal/unmarshal JSON

camel-http

HTTP POST to the Mailpit API

camel-jsonpath

JSONPath expression evaluation for content-based routing

camel-micrometer

Exports Prometheus metrics for processing

How it Works

Processing a CDC event in Camel

When a new CDC event arrives on the topic cdc.public.customers, the Camel route runs the following internal steps:

  1. Poll — The camel-kafka component calls poll() on the leader broker for the partitions assigned to the consumer group camel-cdc-consumer. Kafka returns a batch of records. Each record is processed individually as a Camel Exchange.

  2. Unmarshal — The Exchange body (JSON string of the Debezium event) is deserialized to a Map<String,Object> using Jackson. This lets you access fields such as payload.op, payload.after.first_name, etc.

  3. Content-Based Routing — The choice() evaluates $.payload.op via JSONPath:

    • If op == 'c' (INSERT): generates a "New Customer Created" email

    • For any other operation (UPDATE, DELETE): generates a generic change email

  4. Marshal + HTTP POST — The transformed body is serialized back to JSON and sent via HTTP POST to the Mailpit API (/api/v1/send). The camel-http component handles connection pooling, retries, and timeouts.

  5. Error Handling — If any step fails, onException catches the exception:

    • Marks the error as handled (does not propagate the failure upstream)

    • Sends the original message to the DLQ topic dlq.cdc-camel-errors

    • Logs the error for diagnosis

    • The Kafka offset is committed — the message is not reprocessed

Camel Exchange Model

Each message processed by Camel is an Exchange with:

  • Body: the message content (transformed at each step)

  • Headers: metadata (Kafka topic, partition, offset, timestamp, HTTP headers for the POST)

  • Properties: internal Exchange state (not visible to external components)

The Exchange flows through the route as a pipeline — each step’s output is the next step’s input. This lets you chain transformations without shared state.

Camel processor metrics

The Camel processor exposes Prometheus metrics at /q/metrics:

curl -s http://camel-cdc-processor.kafka-cdc.svc:8080/q/metrics | grep camel

These metrics are collected by the PodMonitor and displayed in the Grafana dashboard "Kafka CDC Pipeline".

Screenshots

Kaoto — Visual Camel Route Designer in DevSpaces

Official Documentation