Confluent Parallel Consumer Adapter for
SAP Integration Suite
Achieve 10–500x higher throughput than the SAP standard adapter. By leveraging parallelization at the record-key level, Avro, Protocol Buffers, Schema Registry, and automated error handling, you ensure maximum performance in SAP Integration Suite and Edge Integration Cell.
Integration
Iflow / EIC
Parallel Consumer
SAP CPI
Apache Kafka
Cloud · On-Premise · EIC
High-Throughput Kafka Integration –
10–500x More Throughput Than SAP Standard
The KaTe Confluent Parallel Consumer Adapter includes all functions of the Kafka Adapter. Additionally, it supplements these with parallelizable processing capabilities for maximum throughput.
- Configurable topics, partitions, and ack mode
- Customizable consumer commits
- Wildcards and durable subscriptions
- Processing from beginning, end, or any position
- 10–500x Higher Throughput
- Parallelization at record-key level
- Configurable thread pool
- Batch processing (n records per Iflow)
- PLAIN, PLAIN_SASL, SASL_SSL
- SASL-Plain / SASL-SCRAM
- OAuth2 Authentication
- Client Certificate Authentication
- Filtering by headers, keys, and data contents
- Deduplication via offset or record header
- Freely definable group names
- Unordered and ordered scenarios
- Error forwarding to Error Topic / Dead Letter Topic
- Deserialization errors automatically handleable
- Non-blocking and asynchronous processing
- Fine-grained error strategies configurable
- Avro and Protocol Buffers support
- Schema Registry integration
- Integrated schema generation tool
- JSON to XML transformation
Processing Models Compared –
Standard vs. Parallel Consumer
The KaTe Confluent Parallel Consumer Adapter offers three processing models: Partition-Based Processing (SAP Standard), Key-Ordered Processing, and Unordered Processing. Thus, it provides maximum flexibility for different requirements.
Why the Confluent Parallel Consumer Adapter
is the Right Choice
High-throughput Kafka integration with SAP Integration Suite – without increasing the partition count, with complete error handling and Avro/Protobuf support.
KaTe vs. SAP Standard
Summary
| Feature | SAP Standard Kafka Adapter | Confluent Parallel Consumer Adapter |
|---|---|---|
| Parallelism model | Limited by number of Kafka partitions | Configurable thread pool enables high parallelism with a single consumer |
| Ordering guarantees | Ordering guaranteed only per partition | Ordering guaranteed at message key level (optional), with parallel execution across different keys |
| Throughput scaling | Requires increasing partition count for scaling | Scales without increasing partitions; up to 10–500x higher throughput possible. |
| Iflow execution | Blocked calls partially or completely stop message processing | Non-blocking and asynchronous processing without message processing being blocked by errors or individual "slow" records. |
| Automated Error handling | Not supported, some errors are not directly visible (deserialization) | Error handling for deserialization / data transformation or iflow execution errors can be set automatically. Errors are then logged (as error messages) and ignored or forwarded to an error topic/dead letter topic. |
| Offset /Group Name control | Not supported | Any type of start possible (start, end) of the offset or reprocessing of arbitrary offsets. Group names freely definable. |
| Message acknowledgment | Offset commits at partition level | Fine-grained, message-level acknowledgment and processing control |
| Advanced Monitoring | Not supported | Rich monitoring user experience (consumer lag, 1:1 recognition of which Kafka record generated which Iflow execution at a glance and out of the box) |
| Filtering | Not supported, nur manuel in jedem einzelnen iflow implementierbar | Filtering with headers or data contents to avoid unnecessary message load |
| Avro/Protobuf and JSON with schema registry | Not supported | Supported, including data transformation and schema generation to XML |
Getting Started – flexibel skalieren
Contact us – within 48 hours you will receive access to the adapter for your test environment.
Installation in your SAP Integration Suite is completed in just a few steps. Furthermore, our team will support you during onboarding.
Initial productive scenarios start quickly. Consequently, you can scale as needed – without upfront investments.
Supported Platforms & Kafka Distributions
Whether Confluent Cloud, Apache Kafka, or Azure Event Hubs – the KaTe Confluent Parallel Consumer Adapter is compatible with all common Kafka distributions and SAP platforms.
KaTe Launchpad – more control during setup and operations
The KaTe Launchpad extends the Confluent Parallel Consumer Adapter with an integrated toolbox for developers and operations teams. As a result, all key functions are accessible directly from the SAP Integration Suite.
Generator
Kafka configurations, topic structures, schema definitions, and SAP Integration Suite Iflow channel definitions are generated automatically. Therefore, the configuration effort for new Kafka integration scenarios is significantly reduced, accelerating deployment.
Service List Display & Tester
The Service List Display provides an overview of all configured Kafka topics, consumer groups, broker connections, and Schema Registry entries. Meanwhile, the integrated tester enables sending and receiving Kafka records directly from the SAP Integration Suite.
SAP Technology and Migration Consulting –
Future-Proof Integration
KaTe GmbH is your competent partner for SAP technology and migration projects. Our focus lies on the secure and smooth transformation of existing SAP PO landscapes to the modern SAP Integration Suite.
- Migration from SAP PO to SAP Integration Suite – including analysis, architecture design, and technical implementation
- Implementation and optimization of the SAP Edge Integration Cell – for flexible and secure hybrid integration
- SAP BTP Consulting – use of modern services and tools for innovation and process automation
- Technology-based Strategy Consulting – from cloud integration to API management
We analyze your existing Kafka interfaces in SAP PO and develop a structured migration plan for the SAP Integration Suite. Consequently, your Kafka integrations are migrated without operational disruption.
Learn moreTest our SAP PO Adapters for Kafka, MQTT, AMQP, AWS and more – 30 days for free.
View all SAP PO AdaptersIntegraMon unites all SAP CPI tenants and Edge Integration Cells in a single dashboard. It features proactive alerting, AI support, and automated KPI reports.
Discover IntegraMonFrequently Asked Questions
Answers to the most important questions about the KaTe Confluent Parallel Consumer Adapter for SAP Integration Suite.
What is the Confluent Parallel Consumer Adapter for SAP Integration Suite?
As a specialized high-throughput extension of the KaTe Kafka Adapter, the KaTe Confluent Parallel Consumer Adapter optimizes processing on SAP Integration Suite or Edge Integration Cell (EIC). It contains all functionalities of the KaTe Kafka Adapter and supplements them with parallelizable processing functions for high throughput – up to 10–500x more than the SAP standard adapter.
What core features does the Confluent Parallel Consumer Adapter offer?
The adapter provides the following core features:
- Parallelization at record-key level (instead of partition level)
- Batch processing (n records as a single Iflow message)
- Filtering by headers, keys, and data contents
- Deduplication via offset or record header
- Automated Error Handling (Error Topic / Dead Letter Topic)
- Avro/Protocol Buffers with Schema Registry and automatic schema generation
Which security standards are supported?
Because enterprise environments demand the highest security standards, the adapter supports all common Kafka security protocols: PLAIN, PLAIN_SASL, SASL_SSL, SASL-Plain/SASL-SCRAM, OAuth2, as well as Client Certificate Authentication for broker and ServiceRegistry connections.
How does the adapter support high performance and error handling?
Since parallelization occurs at the record-key level, the adapter enables 10–500x higher throughput for single and batched records without losing the functional order of related records. Furthermore, errors during deserialization, data transformation, or downstream processing can be automatically forwarded to an error destination or documented and ignored – non-blocking and asynchronous.
How does the adapter differ from standard SAP Cloud Integration adapters?
While the SAP standard adapter parallelizes at the partition level, the KaTe Confluent Parallel Consumer Adapter offers compared to the SAP standard:
- 10–500x higher throughput without increasing the partition count
- Parallelization without blocking by individual "slow" records
- Automated error handling even for deserialization errors
- Avro/Protobuf support with Schema Registry
- Batch processing in a single Iflow execution
- Filtering and deduplication at high data throughput
- Freely definable group names and arbitrary stream positions
On which SAP platforms does the Confluent Parallel Consumer Adapter run?
The adapter is compatible with SAP Cloud Integration (CPI) and SAP Edge Integration Cell (EIC). Thus, both pure cloud scenarios and hybrid on-premise/cloud environments are covered. Furthermore, it is listed in the SAP App Center and therefore officially available as an SAP partner solution.
Which Kafka distributions are supported?
Whether Confluent Cloud, Apache Kafka (all versions), Azure Event Hubs with Kafka protocol, or AWS MSK – the adapter is compatible with all common Kafka distributions, as it uses the standardized Kafka protocol. Additionally, Confluent Platform on-premise with Schema Registry is fully supported.
How does Avro and Protocol Buffers support work?
The adapter supports Avro and Protocol Buffers natively, so binary data formats are directly translated into XML – and vice versa. Thanks to the integrated Schema Registry integration and the included schema generation tool, manual schema maintenance is eliminated. Meanwhile, JSON-to-XML transformation is also available out of the box.
How does installation and deployment work?
After starting the 30-day trial, you will receive access to the adapter within 48 hours. Installation in your SAP Integration Suite is completed in just a few steps, as the KaTe team supports you during onboarding. Initial productive scenarios typically start within a few days – without upfront investments or lengthy project phases.
What licensing models are available for the Confluent Parallel Consumer Adapter?
The KaTe Confluent Parallel Consumer Adapter is available as a flexible license based on your actual usage volume. After the free 30-day trial, you can choose between different licensing options. Contact us for an individual offer so we can find the right model for your requirements together.
Ready for High-Throughput Kafka Integration?
Contact us for a free 30-day trial or a personal advisory call. We set up your test environment within 48 hours.