Categories
SAP Products

Kafka Adapter for ABAP

SAP ABAP

Kafka Adapter for ABAP

Native, bidirectional connectivity between your ABAP applications and Apache Kafka — over TCP/TLS, without a separate Kafka REST proxy.

Integration architecture

ABAP applications
Publish / Consume
KaTe GmbHKaTe Kafka Adapter for ABAPNative TCP/TLS
Kafka records
Apache Kafka
TCP/TLSConnectionAvroSchema RegistryJSONData formats
Technical building blocks
IDocs
Custom ABAP types
JSON
Avro
Schema Registry

Architecture overview

ABAP applications, KaTe Kafka Adapter for ABAP and Apache Kafka

ABAP applications
KaTe GmbHKaTe Kafka Adapter for ABAPNative ABAP integration
Apache Kafka
Publish / ConsumeNative TCP/TLS
Direct connection between ABAP and Kafka

Features

All features at a glance

Bring native Kafka connectivity into your ABAP applications with native producer and consumer APIs, configurable data binding and central administration.

Core functions: publish and consume

Publish application data as Kafka records and consume incoming records directly in standard S/4HANA and ECC functions and ABAP.

  • Flexible publishing: publish Kafka records from IDocs, RAP Events (TBA) or custom ABAP programs, using blocking sends or non-blocking outbox enqueueing with Kafka broker acknowledgements (acks).
  • Controlled consumption: read Kafka records and submit them to SAP integration objects, such as IDocs and ABAP calls, through managed consumer channels with consumer groups. Consumers process partitions in parallel and commit Kafka offsets only after successful SAP receiver processing, with retries for failed records within their partition.
  • Configurable setup: use one or more topics, control offsets and work with Kafka keys, values and headers.

Security and connection management

Secure your Kafka connections with TLS encryption, protected credentials and centrally managed connection profiles.

  • Native TCP/TLS connectivity with SASL/PLAIN authentication.
  • Encrypted credentials for Kafka brokers and Schema Registry.

Advanced features: SAP integration, data formats and error handling

SAP integration, data formats and error handling

SAP integration and data formats

Integrate SAP objects with Kafka

  • IDocs: exchange complete IDocs as Kafka records, with SAP acting as sender or receiver.
  • Custom ABAP data types and tables: bind application data to explicitly configured or generic ABAP types, structures and internal tables.
  • RAP Events (TBA): integration with events and powerful reference data lookup from the ABAP RESTful Application Programming Model.

Supported serializers and deserializers

  • Standard Kafka formats: String, Boolean, Short, Integer, Long, Float, Double, UUID, Void and binary formats (ByteArray, ByteBuffer and Bytes).
  • Confluent Avro: native binary encoding and decoding, Schema Registry integration and supported schema evolution.
  • Confluent JSON Schema: serialization and validation with Schema Registry integration within a documented Draft 7 subset.

Key and Value formats can be freely configured with respective serializers/deserializers and binding to data structures.

Configurable error handling and retries

Keep processing failures visible and control how failed records are retried.

  • Configure retry limits and delays.
  • Review retry status and failure details in monitoring.
  • Retry failed records within their partition while other partitions continue.
  • Review uncertain producer delivery outcomes before deciding on recovery.

Advantages

Why connect to Kafka directly from ABAP?

1 · Connect directly to Kafka brokers

Use Kafka’s binary protocol from ABAP. A separate Kafka REST proxy is unnecessary for broker communication; Schema Registry uses its own HTTPS API.

2 · Work with your application’s data

Integrate IDocs and RAP Events (TBA), or use generic and typed interfaces for custom ABAP data. Select the serialization format independently for keys and values.

3 · Keep configuration and diagnostics together

Manage connection profiles and channels in one user-friendly web console, and investigate operational failures through correlated runtime diagnostics.

Comparison

Choose the integration approach that fits your scenario

Kafka Adapter for ABAP

Direct ABAP APIs and SAP data binding

Application-controlled table extraction

Integration logic within your SAP application

Kafka Adapter for SAP Integration Suite

Integration through Cloud Integration iFlows

Orchestration of data exposed by SAP interfaces

Central transformation and routing across systems

Setup

Set up your Kafka integration in three steps

1

1 · Discuss your scenario

Identify your SAP environment, Kafka topics, data formats and processing requirements with our team.

2

2 · Configure the adapter

Prepare the SAP prerequisites, configure protected connection profiles and define producer or consumer channels with the required data bindings.

3

3 · Validate and start processing

Check connectivity, test your application integration and use the administration console to control and monitor processing.

Compatibility

Find the right setup for your SAP environment

Contact us to discuss your SAP release, Kafka environment and installation prerequisites. Avro, Schema Registry and JSON Schema support follow documented mapping, schema and resource limits.

Administration console

More control during setup and operations

Operations and monitoring

Manage Kafka channels centrally and investigate processing issues with runtime status, application logs and targeted diagnostics.

  • Start, stop, pause and resume managed channels from the administration console, with consumer startup from the beginning or latest position of the stream.
  • Inspect consumer group offsets, processing progress and partition status.
  • Monitor producer outbox entries and review uncertain delivery outcomes.
  • Investigate failures through correlated application logs and temporary Kafka protocol traces.

FAQ

Frequently asked questions

Answers to common questions about the Kafka Adapter for ABAP.

What is the Kafka Adapter for ABAP?

The adapter connects SAP applications directly to Kafka using native ABAP producer and consumer APIs. It brings IDocs and custom ABAP data into Kafka integration, with configurable serialization and a web console for administration. RAP Events integration is TBA.

How are Kafka connections secured?

Broker communication uses TCP/TLS with SASL/PLAIN authentication. Broker and Schema Registry credentials are stored under separate encrypted aliases, keeping connection configuration separate from secrets.

How does the adapter preserve processing order?

Records are processed in order within each assigned partition. A failed record is retried before later records in that partition proceed, while other partitions can continue. Ordering across different partitions is determined by your application design.

Can consumers process records in parallel?

Managed consumer groups distribute partitions across configured consumers. This allows parallel processing across partitions while retaining ordered processing within each partition. Capacity depends on partition count, SAP resources and receiver workload.

Can I choose where consumption starts?

You can configure consumption from the beginning or latest position of the stream. Explicit-offset read APIs also support selected stream positions. Existing consumer-group offsets affect where an established group resumes.

How can I monitor and troubleshoot my integration?

The web console brings channel controls, processing progress, consumer offsets and producer outbox inspection together. Application logs show error and retry details; temporary Kafka protocol traces support targeted troubleshooting. Retry limits and delays are configurable.

When should I use the ABAP adapter instead of a Kafka adapter on middleware?

Use the ABAP adapter when integration logic belongs in SAP — for example, application data integration or custom ABAP extraction of table data to Kafka. Publish and consume Kafka records directly within your ABAP applications, with control over data binding and processing, without routing each exchange through a middleware layer. Use the KaTe Kafka Adapter for SAP Integration Suite when you need central orchestration, transformations or routing across multiple systems. The two adapters complement each other within your integration landscape.

Do I need a Kafka REST proxy?

Unlike other solutions, no separate Kafka REST proxy is needed for broker communication. The adapter speaks Kafka’s binary protocol directly over TCP/TLS from ABAP. Schema Registry integration uses HTTPS separately.

Can I replicate tables to Kafka?

Yes. Publish table data to Kafka using generic or explicitly typed ABAP serialization and a simple custom ABAP program that reads and sends data through a Kafka channel. This enables straightforward data integration tailored to your requirements.

Can I exchange IDocs through Kafka?

Yes. Send and receive complete IDocs as Kafka records using standard SAP IDoc integration. Outbound IDocs are confirmed after Kafka acknowledges the record.

Which Confluent serializers are supported?

The adapter provides serializers and deserializers compatible with Confluent Avro and Confluent JSON Schema, including Schema Registry integration. Avro supports schema evolution; JSON Schema supports validation within a documented Draft 7 subset.

How does the adapter support reliable delivery?

The adapter commits Kafka consumer offsets only after successful receiver processing. Failed records are retried before processing continues within their partition, helping ensure records are not skipped when processing fails.

Contact

Planning Kafka integration from ABAP?

Tell us about your SAP environment, data formats and processing requirements. We can discuss the integration approach and supported configuration with you.

Categories
SAP Products

Confluent Parallel Consumer Adapter for SAP Integration Suite

Confluent Parallel Consumer Adapter for SAP Integration Suite – KaTe GmbH
SAP Integration Suite (CPI)

Confluent Parallel Consumer Adapter for
SAP Integration Suite

Achieve 10–500x higher throughput than the SAP standard adapter. By leveraging parallelization at the record-key level, Avro, Protocol Buffers, Schema Registry, and automated error handling, you ensure maximum performance in SAP Integration Suite and Edge Integration Cell.

Verified SAP Partner Request a Demo 10–500x Higher Throughput Avro & Protocol Buffers
Architecture Overview
SAP Cloud
Integration
Iflow / EIC
parallel
KaTe Confluent
Parallel Consumer
SAP CPI
Kafka Protocol
Confluent /
Apache Kafka
Cloud · On-Premise · EIC
10–500x Throughput Avro / Protobuf Schema Registry Batch Processing SASL_SSL OAuth2 Error Handling

High-Throughput Kafka Integration –
10–500x More Throughput Than SAP Standard

The KaTe Confluent Parallel Consumer Adapter includes all functions of the Kafka Adapter. Additionally, it supplements these with parallelizable processing capabilities for maximum throughput.

Publish & Subscribe
SAP Cloud Integration Iflow messages are published as Kafka records, while Kafka records are subscribed to as Iflow messages. During this process, Kafka Key, Header, and Data are fully translated into Iflow Messages.
  • Configurable topics, partitions, and ack mode
  • Customizable consumer commits
  • Wildcards and durable subscriptions
  • Processing from beginning, end, or any position
Parallelization & Throughput
Because parallelization occurs at the record-key level instead of the partition level, a 10–500x higher throughput is achieved for single and batched records. Consequently, this happens without losing the functional order of related records.
  • 10–500x Higher Throughput
  • Parallelization at record-key level
  • Configurable thread pool
  • Batch processing (n records per Iflow)
Security
Since enterprise environments demand the highest security standards, all common Kafka security protocols for broker and ServiceRegistry connections are supported. This includes both OAuth2 and Client Certificate Authentication.
  • PLAIN, PLAIN_SASL, SASL_SSL
  • SASL-Plain / SASL-SCRAM
  • OAuth2 Authentication
  • Client Certificate Authentication
Filtering & Flow Control
Gain full control over throughput and processing order. Kafka records can be filtered before triggering an integration, ensuring that only relevant messages are processed.
  • Filtering by headers, keys, and data contents
  • Deduplication via offset or record header
  • Freely definable group names
  • Unordered and ordered scenarios
Automated Error Handling
Errors during deserialization, data transformation, or downstream processing can be handled automatically. Therefore, non-blocking and asynchronous processing is guaranteed even with faulty records.
  • Error forwarding to Error Topic / Dead Letter Topic
  • Deserialization errors automatically handleable
  • Non-blocking and asynchronous processing
  • Fine-grained error strategies configurable
Avro, Protocol Buffers & Schema Registry
Typical binary data formats are supported with Kafka-specific optimizations. As a result, Avro and Protocol Buffers are directly translated into XML – and vice versa.
  • Avro and Protocol Buffers support
  • Schema Registry integration
  • Integrated schema generation tool
  • JSON to XML transformation

Processing Models Compared –
Standard vs. Parallel Consumer

The KaTe Confluent Parallel Consumer Adapter offers three processing models: Partition-Based Processing (SAP Standard), Key-Ordered Processing, and Unordered Processing. Thus, it provides maximum flexibility for different requirements.

Verarbeitungsmodelle: Partition-Based Processing (1 Partition = 1 Thread), Key-Ordered Processing (Ordered per Message Key), Unordered Processing (Concurrent & Unordered)

Why the Confluent Parallel Consumer Adapter
is the Right Choice

High-throughput Kafka integration with SAP Integration Suite – without increasing the partition count, with complete error handling and Avro/Protobuf support.

1
10–500x Higher Throughput
Because parallelization occurs at the record-key level instead of the partition level, a drastically higher throughput is achieved – without increasing the partition count and with only a single consumer.
2
Automated Error Handling
Errors during deserialization, data transformation, or downstream processing are automatically forwarded to an error topic or dead letter topic – non-blocking and without any operational disruption.
3
Avro, Protocol Buffers & Schema Registry
Typical binary data formats are fully supported, so integrated schema generation tools can be used directly without third-party tools.
4
Easy Migration from SAP PO
Existing interfaces can be easily imported because the adapter is fully compatible with the SAP Integration Suite – investments in SAP PO integrations can be easily transferred.

KaTe vs. SAP Standard

Feature KaTe Adapter SAP Standard
Parallelism Model
Record-Key-Level
Partition-Level
Throughput Scaling
10–500x higher
Partition increase required
Automated Error Handling
✓ Yes
✗ No
Avro / Protocol Buffers
✓ Yes
✗ No
Batch Processing
✓ Yes
✗ No
Filtering & Deduplication
✓ Yes
✗ No
Freely Definable Group Names
✓ Yes
✗ No
Advanced Monitoring
Consumer Lag, 1:1 Record-Tracking
✗ Not supported

Summary

FeatureSAP Standard Kafka AdapterConfluent Parallel Consumer Adapter
Parallelism modelLimited by number of Kafka partitionsConfigurable thread pool enables high parallelism with a single consumer
Ordering guaranteesOrdering guaranteed only per partitionOrdering guaranteed at message key level (optional), with parallel execution across different keys
Throughput scalingRequires increasing partition count for scalingScales without increasing partitions; up to 10–500x higher throughput possible.
Iflow executionBlocked calls partially or completely stop message processingNon-blocking and asynchronous processing without message processing being blocked by errors or individual "slow" records.
Automated Error handlingNot supported, some errors are not directly visible (deserialization)Error handling for deserialization / data transformation or iflow execution errors can be set automatically. Errors are then logged (as error messages) and ignored or forwarded to an error topic/dead letter topic.
Offset /Group Name controlNot supportedAny type of start possible (start, end) of the offset or reprocessing of arbitrary offsets. Group names freely definable.
Message acknowledgmentOffset commits at partition levelFine-grained, message-level acknowledgment and processing control
Advanced MonitoringNot supportedRich monitoring user experience (consumer lag, 1:1 recognition of which Kafka record generated which Iflow execution at a glance and out of the box)
FilteringNot supported, nur manuel in jedem einzelnen iflow implementierbarFiltering with headers or data contents to avoid unnecessary message load
Avro/Protobuf and JSON with schema registryNot supportedSupported, including data transformation and schema generation to XML

Getting Started – flexibel skalieren

1
Request Free Trial

Contact us – within 48 hours you will receive access to the adapter for your test environment.

2
Install Adapter

Installation in your SAP Integration Suite is completed in just a few steps. Furthermore, our team will support you during onboarding.

3
Go Live

Initial productive scenarios start quickly. Consequently, you can scale as needed – without upfront investments.

Supported Platforms & Kafka Distributions

Whether Confluent Cloud, Apache Kafka, or Azure Event Hubs – the KaTe Confluent Parallel Consumer Adapter is compatible with all common Kafka distributions and SAP platforms.

SAP Integration Suite
SAP Cloud Integration (CPI) – fully integrated
Supported
SAP Edge Integration Cell
Hybrid on-premise/cloud scenarios (EIC)
Supported
Confluent Platform
Confluent Cloud and on-premise with Schema Registry
Supported
Apache Kafka
Open Source Apache Kafka – all versions
Supported
Azure Event Hubs
Azure Event Hubs with Kafka protocol
Supported
AWS MSK
Amazon Managed Streaming for Apache Kafka
Supported
SAP BTP
SAP Business Technology Platform – Cloud-native
Supported
Avro / Protocol Buffers
Binary data formats with Schema Registry
Supported

KaTe Launchpad – more control during setup and operations

The KaTe Launchpad extends the Confluent Parallel Consumer Adapter with an integrated toolbox for developers and operations teams. As a result, all key functions are accessible directly from the SAP Integration Suite.

Generator

Kafka configurations, topic structures, schema definitions, and SAP Integration Suite Iflow channel definitions are generated automatically. Therefore, the configuration effort for new Kafka integration scenarios is significantly reduced, accelerating deployment.

Service List Display & Tester

The Service List Display provides an overview of all configured Kafka topics, consumer groups, broker connections, and Schema Registry entries. Meanwhile, the integrated tester enables sending and receiving Kafka records directly from the SAP Integration Suite.

SAP Technology and Migration Consulting –
Future-Proof Integration

KaTe GmbH is your competent partner for SAP technology and migration projects. Our focus lies on the secure and smooth transformation of existing SAP PO landscapes to the modern SAP Integration Suite.

  • Migration from SAP PO to SAP Integration Suite – including analysis, architecture design, and technical implementation
  • Implementation and optimization of the SAP Edge Integration Cell – for flexible and secure hybrid integration
  • SAP BTP Consulting – use of modern services and tools for innovation and process automation
  • Technology-based Strategy Consulting – from cloud integration to API management
SAP PO → Integration Suite Migration
Structured Migration Without Operational Disruption

We analyze your existing Kafka interfaces in SAP PO and develop a structured migration plan for the SAP Integration Suite. Consequently, your Kafka integrations are migrated without operational disruption.

Learn more
Kafka Adapter for SAP PO – Request a Demo
All SAP PO Adapters Without Upfront Payment

Test our SAP PO Adapters for Kafka, MQTT, AMQP, AWS and more – 30 days for free.

View all SAP PO Adapters
IntegraMon – SAP Monitoring
Central Dashboard for all SAP CPI Tenants & EIC Instances

IntegraMon unites all SAP CPI tenants and Edge Integration Cells in a single dashboard. It features proactive alerting, AI support, and automated KPI reports.

Discover IntegraMon

Frequently Asked Questions

Answers to the most important questions about the KaTe Confluent Parallel Consumer Adapter for SAP Integration Suite.

What is the Confluent Parallel Consumer Adapter for SAP Integration Suite?

As a specialized high-throughput extension of the KaTe Kafka Adapter, the KaTe Confluent Parallel Consumer Adapter optimizes processing on SAP Integration Suite or Edge Integration Cell (EIC). It contains all functionalities of the KaTe Kafka Adapter and supplements them with parallelizable processing functions for high throughput – up to 10–500x more than the SAP standard adapter.

What core features does the Confluent Parallel Consumer Adapter offer?

The adapter provides the following core features:

  • Parallelization at record-key level (instead of partition level)
  • Batch processing (n records as a single Iflow message)
  • Filtering by headers, keys, and data contents
  • Deduplication via offset or record header
  • Automated Error Handling (Error Topic / Dead Letter Topic)
  • Avro/Protocol Buffers with Schema Registry and automatic schema generation
Which security standards are supported?

Because enterprise environments demand the highest security standards, the adapter supports all common Kafka security protocols: PLAIN, PLAIN_SASL, SASL_SSL, SASL-Plain/SASL-SCRAM, OAuth2, as well as Client Certificate Authentication for broker and ServiceRegistry connections.

How does the adapter support high performance and error handling?

Since parallelization occurs at the record-key level, the adapter enables 10–500x higher throughput for single and batched records without losing the functional order of related records. Furthermore, errors during deserialization, data transformation, or downstream processing can be automatically forwarded to an error destination or documented and ignored – non-blocking and asynchronous.

How does the adapter differ from standard SAP Cloud Integration adapters?

While the SAP standard adapter parallelizes at the partition level, the KaTe Confluent Parallel Consumer Adapter offers compared to the SAP standard:

  • 10–500x higher throughput without increasing the partition count
  • Parallelization without blocking by individual "slow" records
  • Automated error handling even for deserialization errors
  • Avro/Protobuf support with Schema Registry
  • Batch processing in a single Iflow execution
  • Filtering and deduplication at high data throughput
  • Freely definable group names and arbitrary stream positions
On which SAP platforms does the Confluent Parallel Consumer Adapter run?

The adapter is compatible with SAP Cloud Integration (CPI) and SAP Edge Integration Cell (EIC). Thus, both pure cloud scenarios and hybrid on-premise/cloud environments are covered. Furthermore, it is listed in the SAP App Center and therefore officially available as an SAP partner solution.

Which Kafka distributions are supported?

Whether Confluent Cloud, Apache Kafka (all versions), Azure Event Hubs with Kafka protocol, or AWS MSK – the adapter is compatible with all common Kafka distributions, as it uses the standardized Kafka protocol. Additionally, Confluent Platform on-premise with Schema Registry is fully supported.

How does Avro and Protocol Buffers support work?

The adapter supports Avro and Protocol Buffers natively, so binary data formats are directly translated into XML – and vice versa. Thanks to the integrated Schema Registry integration and the included schema generation tool, manual schema maintenance is eliminated. Meanwhile, JSON-to-XML transformation is also available out of the box.

How does installation and deployment work?

After starting the 30-day trial, you will receive access to the adapter within 48 hours. Installation in your SAP Integration Suite is completed in just a few steps, as the KaTe team supports you during onboarding. Initial productive scenarios typically start within a few days – without upfront investments or lengthy project phases.

What licensing models are available for the Confluent Parallel Consumer Adapter?

The KaTe Confluent Parallel Consumer Adapter is available as a flexible license based on your actual usage volume. After the free 30-day trial, you can choose between different licensing options. Contact us for an individual offer so we can find the right model for your requirements together.

Get Started Now

Ready for High-Throughput Kafka Integration?

Contact us for a free 30-day trial or a personal advisory call. We set up your test environment within 48 hours.