Streaming

API.
Subscribe to real time data streaming from
Mambu.

Owner: Delivery Team

Last updated: Feb 2022
Table of Contents.
1. Why Streaming API

2. Use cases

3. Streaming API - Concepts (optional)

4. Streaming API - Services (aka Endpoints)

5. Streaming API - Hands on

6. Questions?
Simple at first... ...but after a while

Consumer

EVENT Consumer

Consumer

EVENT
NT
EVE

EVENT Consumer

EV
EN
T

EVENT
Consumer

Consumer
Why Streaming API. Abstract implementation via REST API

Distributed messaging system.

● Horizontal scalability
● Fault tolerant

STREAMING API
A high throughput, near real time

High performance (latency of less than 10 ms)

- hence near real time
Use cases. Take load out of Mambu API and webhooks

Streaming APIs allow an external system to listen for Goals:

● reduce the consumption of unnecessary API calls and
changes to individual fields or profile data residing processing time, by reducing the number of requests
within Mambu in real-time, making it useful when trying that return no data;
● improve performance.
to pipeline streaming data.

Extend business logic or integrate ecosystem

Stream data from mambu to data Near real time data movement from
components
lake/data warehouse mambu...
Example: integrations with systems that require extracting
Example: copy of loan account in Arrears > X … to applications that have constant pulling potentially thousands of transactions every day
days to the database and generating action against the Mambu environment and
downstream event notifications for require general notification of data changes or
Collections department. events
Note: since Streaming APIs have limited
triggering events, they cannot fully replace
the entire database backup.
Streaming
API - Concepts.
Optional: what's behind the APIS - theory.
 REST API VS STREAMING API

CLIENT SERVICE API CLIENT SERVICE API

CONNECTION CONNECTION
Request Request
OPENS OPENS
Response

Event
Request
Response Event

Event
Request
Response
Event

Event
Request
Response

CONNECTION CONNECTION
CLOSES CLOSES
Streaming API.
○ Producer - mambu
■ Topic :
● Configure the payload to be sent to a topic as a message and when.
● Unique auto generated entity for each combination of mambu instance and event type

○ Consumer - your application

■ Subscription
● Applications subscribe to topics to consume the messages.
○ Each subscription can have 3 clients per topic
■ One topic in subscription - 3 clients
■ 2 topic in subscription - 6 consuming clients
Producer side of the API.
○ Event types & Events
○ Event types are also called as topics
○ Topics contains logical stream of events
produced by mambu.
○ Events are payload built by mambu for the
consumption of the external application

● Partitions
○ Events of event types are stored in
partitions
○ Offsets
○ By default, current implementation, each
○ Offset has a meaning for only specific partition
event types stores its events in 3
■ For example, offset 3 partition 1 holds an event different to offset 3
partitions
in partition 0
○ Each event in each partition is stored in
○ Ordering is guaranteed only within a partition
ordered fashion, and each event will get its
own incremental id called offset.
Producer side of the API.
○ Event & Partition continued...
○ Events in the partition are stored only for a limited time. Current TTL duration is 96 hours
Consumer side of the API.
Owning Application & Consumer Group

Partition 0 Partition 1 Partition 2

Consumer Consumer Consumer

Consumer Group 1 Consumer Group Consumer Group 2

 Initial setup.
01 03
Events Streaming (Beta) Event Streaming Notification
feature enabled in Mambu template created
tenant
1. Create Events Streaming Notifications

Administration → Events Streaming and click the

"Add Notification" button.

02 2. Event Stream Notification Store Form

The Topic value represents the name of the

API Consumer Keys feature channel to which Mambu notifications are
submitted, is based on the selected notification
enabled in Mambu tenant event type and is auto-generated.

Notifications can be streamed through Event

Streaming API, using the topic value.
 Streaming API in 4 steps.
01 02 03 04
Create Subscription Create Stream Commit Cursor Delete Subscription

A unique subscription to topics Open one or more streams from Commit cursor found in the event Deleting the subscription after event
comprises of: consuming client application to a batches back to Mambu after consumption to reset the cursor
● owning_application , topic through the subscription ID: consumption. offset.
● consumer_group ● Read events sent on the stream The cursor should be committed in
● event_types (Topic Name). as chunk/batch; max 60 seconds, otherwise the POST
● Commit chunk based on the operation should be re-done.
cursor token of the batch.
Headers
Headers Headers Headers - ApiKey
- ApiKey - ApiKey - ApiKey - ContentType
- ContentType - ContentType - ContentType
- X-Mambu-StreamId

/api/v1/subscriptions api/v1/subscriptions/{subscription-i api/v1/subscriptions/{subscriptionId /api/v1/subscriptions/{subscription_

d}/events }/cursors: id}

https://api.mambu.com/CNNmZrTm3GDuWN/index.html#using-the-streaming-api
Structure of Batch.
JSON Structure of a Batch
 Stream configuration.
Empty Batch

● Batch 1
○ Events
■ Event 1 ● batch_limit = 3
■ Event 2 ● batch_flush_timeout = 2 seconds (default - 30 seconds)
■ Event 3

● Batch 2 ● batch_limit = 3
○ Events ● batch_flush_timeout = 2 seconds (default - 30 seconds)
■ Event 4

● max_uncommitted events ● stream_timeout = 10 minutes (default - 70 minutes)

● commit_timeout ● stream_keep_alive_limit = 1 empty batch (default -
unlimited)
● stream_limit = unlimited
Hands on.
 Demo.
● Subscription
○ Different ways to open a cursor

● Stream Configuration
○ Configuring empty batches writing
○ Configuring batch flush timeout
○ Configuring stream timeout

● Stream Reading
○ How many streams can be opened for a
subscription topic
○ Client Rebalancing

https://support.mambu.com/docs/streaming-api
 Demo.

● Stream Reading
○ How an empty batch will look?
○ How a batch with events will look?

● Cursor Commit
○ How to commit the cursor of a received batch?
○ Configuring cursor commit timeout

https://support.mambu.com/docs/streaming-api
Questions.
Useful links/ examples.
● Mambu support pages:
○ https://support.mambu.com/docs/streaming-api

● Mambu api documentation:

○ https://api.mambu.com/CNNmZrTm3GDuWN/index.html#mambu-streaming-api-definition
○ https://api.mambu.com/CNNmZrTm3GDuWN/index.html#using-the-streaming-api
○ https://api.mambu.com/CNNmZrTm3GDuWN/index.html#mambu-streaming-api-definition-subscription-api

● Mambu built sample application (currently, available using Java only)

○ https://github.com/mambu-gmbh/Streaming-API-Java-Sample-Client

● Build your own

○ Implement code per the HTTP Standard - Server Sent Events:
https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events