Kafka can deliver far more events than a conventional backend can process one request at a time. The bottleneck is often not Kafka itself. It is the repeated integration and network overhead created when every record starts its own SAP Integration Suite message and its own backend call.

The Sender Batch feature of the KaTe Kafka Adapter for SAP Integration Suite changes that equation. Instead of translating every polled Kafka record into a separate Cloud Integration message, the adapter can combine the records returned by a poll into one batch payload. One integration run can then validate, map and deliver many business records together.

What changes in batch mode?

In single-record mode, 100 Kafka records create up to 100 Cloud Integration messages. In batch mode, the adapter can create one message containing up to the configured Max poll records. The backend receives fewer, larger calls, while Kafka offsets are committed only after the complete batch has been transferred successfully into Cloud Integration.

Two payload styles are available:

For continuous processing, the adapter collects available records up to the configured poll limit. If the batch is not full, it can wait for the configurable batch window before submitting the records already available. Interval processing creates one batch from the records returned by that interval’s poll.

An illustrative throughput model

Consider a topic with four partitions and enough parallel consumers to keep all four busy. Assume one single-record backend interaction takes 50 ms end to end: 25 ms of network and protocol latency plus 25 ms of backend work.

These are elapsed times including acknowledgement and commit where applicable. The model assumes four active processing lanes and excludes additional iFlow processing and batch-collection delays.

Batching amortizes the integration and network cost. The exact gain depends on whether the backend has a true bulk operation or still performs most of its work record by record.

The optimistic case assumes that a backend can process the complete batch in roughly the same 50 ms as one record. The conservative case assumes that only the 25-ms network portion is shared while 25 ms of work remains for every record. Its speed-up is:

With these assumptions, eliminating repeated network round trips can approach 2× even when backend work remains linear. A real bulk API, bulk database operation or vectorised transformation can move the result much closer to the batch-size multiplier. These figures are a capacity-planning model, not a measured product benchmark; payload size, mapping cost, worker count, broker latency, retries and backend limits all affect production throughput.

Why a JDBC batch can finish close to a single insert

A 50-ms JDBC insert including network travel and commit is a plausible assumption for a remote database. A modest batch can finish close to that when fixed overhead dominates: JDBC batching reduces round trips, and one transaction shares commit overhead across rows.

For example, 49 ms fixed overhead + 1 ms per row gives 50 ms for one insert and 149 ms for 100 rows. Four concurrent lanes would then deliver approximately 2,685 records/s — 33.6× the single-record baseline. These are illustrative inputs; indexes, triggers and lock contention affect actual results. The earlier near-2× case assumes all backend work repeats per row, so it is not a general JDBC limit.

Fewer metered Integration Suite messages

Batching can also reduce SAP Integration Suite message consumption. At the time of writing, SAP’s public pricing definition counts a Cloud Integration message in 250 KB blocks — not 256 KB. An outbound message of up to 250 KB counts as one message; every additional 250 KB or fraction thereof counts as another. For synchronous receiver calls, SAP considers the request and response together, and special patterns such as splitters have separate counting rules. The exact commercial effect depends on the customer’s edition, included volume, contract and whether a message is chargeable. See the current SAP Integration Suite pricing definition and verify the deployed flow in the tenant’s Message Usage dashboard before using an estimate for budgeting.

For example, 20 individually processed records below 250 KB normally create 20 outbound backend messages and therefore at least 20 metered messages. If the same records form one outbound batch that remains at or below 250 KB — including its envelope and, for a synchronous call, its response — the batch can count as one metered message. If that batch grows to 600 KB, it counts as three rather than 20. Batching does not make payload volume free, but it removes the minimum one-message charge that otherwise applies to every small outbound call.

Treat this as a planning approximation. Headers, envelopes, receiver calls, synchronous responses, retries, splitters and the final deployed flow can change the metered total.

Where batching helps most

Sender Batching is particularly effective when the downstream system already supports bulk requests, database array operations, mass updates or file-oriented interfaces. It can also reduce the number of Cloud Integration message processing logs and repeated connection setups produced for the same business volume.

There is a deliberate trade-off: a batch is one processing unit. Larger batches improve amortisation but can increase per-record latency, memory use and the amount of work retried after a failure. Channel-level single-record deduplication is not available for variable batches, so downstream processing should be idempotent or deduplicate records using stable business identifiers. Batching is also not available for selective support-mode reads.

A practical starting point

Start with a moderate Max poll records value such as 10 or 25, align the number of parallel consumers with the available topic partitions and verify that the backend genuinely handles the batch efficiently. Then increase the batch size while watching end-to-end latency, metered message usage, payload size, memory, retry behaviour and backend saturation. A batch window should be long enough to form useful batches but short enough to meet the scenario’s latency target.

Batching and partition parallelism complement each other: partitions provide concurrent work lanes, while batching makes each lane more efficient. Together they let SAP Integration Suite process Kafka workloads with substantially fewer integration and network round trips.

This product update is published by KaTe GmbH. Throughput and message-consumption figures are illustrative capacity-planning inputs and must be validated with the customer’s payloads, deployed integration flows and commercial terms.