Reactive (non-busy-wait) service example in iceoryx2

[TE-2]

Hi everyone,

first of all thank you for this awesome project! We’re using iceoryx as middleware in a research project and really enjoy working with it :)

We originally used iceoryx1 for several microservice applications that communicate purely via a request–response pattern. Now we’d like to move to iceoryx2.

In our setup, services should be reactive and efficiently wait for incoming requests without busy waiting. Therefore, our original implementation followed the Client using WaitSet and Server using Listener Example.

Since iceoryx2 refactored the execution control, including the WaitSet and Listener mechanisms, it’s been challenging for me to translate the old examples. The new request–response C++ examples seem to rely on polling, and I could not find an waitset-based variant.

I’ve gone through the iceoryx2 book and the event-based communication example, but I'm still not sure how to implement it correctly.

Is there already an example or a recommended pattern for this in iox2?
Any hints or code snippets would be super helpful.

Thanks a lot!

[elBoberido]

Hi @TE-2

currently there is no example and we also need to extend the Event mechanism to send notifications only to single Listeners. Currently you would have to create multiple Event services and do the mapping of the Event to the Client by yourself. Alternatively, there is the option to also wake up Clients which did not send the request. Here, you could use the event ID to tell if the event was meant for the corresponding client. The event ID could be sent to the server as part of the user header.

It is on our todo list, though and we would like to extend the Event mechanism to support coupling an endpoint to a listener and let the notifier only trigger the listener related to a specific endpoint. There is currently no ETA, though and this feature needs some funding.

[TE-2]

Thanks for the explanation, we’re really looking forward to that feature!

Maybe one additional query (hope I understood things correctly):
Even if all clients are currently woken up by an event, would it still be possible on the server side to attach a callback function to some kind of listener or waitset mechanism, so that the server can efficiently wait for incoming requests and execute the corresponding callback once a request arrives?

In our setup, the server has the longest idle time since it only reacts when a client calls it. So polling on the client side and waiting for the response from the server wouldn’t be an issue, but we’d like to avoid busy waiting on the server.

I tried adapting the waitset-based approach from the event_based_communication example, so that the server thread blocks and wakes up as soon as a request comes in, executing a callback function like waitset.wait_and_process(callback_func).expect("");

However, I couldn’t find a way to attach a server object via waitset.attach...() like the subscriber to trigger the callback when a request/event is received.

Is there a way to achieve this pattern with the current API?

[cerdelen]

Hi, working on something similar the way we approached this problem was using channels from the tokio runtime. In the callback function inserted into the waitset we push incoming "requests" into the channel and then use await on the UnboundedReceiver of the tokio Channel.

This probably should work too with std channels and pushing the server waitset to its own designated thread and then synchronously recv() on unbounded_channels from the std library

[elBoberido]

@TE-2 are you waiting on multiple listener?

The event base communication example creates a custom subscriber, which combines a iceoryx2 subscriber with a listener. It then also implements FileDescriptorBased and SynchronousMultiplexing for the custom subscriber. This is an example that shows how you can create your own abstraction on top of iceoryx2.

With iceoryx2, only the listener is able to block and that is also what you need to use with the server. So there would be a corresponding server to the listener and the listener would be attached to the waitset. But if you do not have multiple listener, you could also just block on that listener, similar to the event example.

[TE-2]

Oh yes of course, thanks for the clarification! I realized I had a misunderstanding about how the Listener and WaitSet works and how they relate to each other.

As I understand it now, the communication would look like this:

• The Client sends its request and additionally triggers an Event via a Notifier.
  Optionally, it can also block on its own Listener until a response event arrives.
• The Server blocks on a Listener for the client’s event, wakes up once the event is triggered,
  processes the request, and then sends a response (and optionally another event to wake the client).
• The WaitSet is an optional higher-level construct for monitoring and managing multiple event sources like a Listener

I have created a small minimal version of my current understanding for testing without a custom server/client wrapper classes to keep it simple as possible. The only remaining detail is that the client still needed a Listener to wait for the response, which I initially solved with a simple blocking wait.

server.cpp

#include "iox2/iceoryx2.hpp"
#include "transmission_data.hpp"

constexpr iox::units::Duration DEADLINE = iox::units::Duration::fromSeconds(100);

using namespace iox2;

int main() {
    set_log_level_from_env_or(LogLevel::Info);

    auto node = NodeBuilder().create<ServiceType::Ipc>().expect("node creation");

    auto rr = node.service_builder(ServiceName::create("My/Funk/ServiceName").expect("valid"))
                  .request_response<uint64_t, TransmissionData>()
                  .open_or_create()
                  .expect("open_or_create");
    auto server = rr.server_builder().create().expect("server");

    // Event used to wake up when a client sends a request
    auto ev = node.service_builder(ServiceName::create("My/Funk/ServiceName/ReqEvent").expect("valid"))
                 .event()
                 .open_or_create()
                 .expect("open_or_create");
    auto listener = ev.listener_builder().create().expect("listener");

    std::cout << "[Server] Waiting for requests..." << std::endl;

    while (true) {
        // Block until an event is triggered
        listener.timed_wait_one(DEADLINE).expect("wait_one");

        // Handle all available requests
        while (true) {
            auto maybe_request = server.receive().expect("receive");
            if (!maybe_request.has_value()) {
                break;
            }

            auto& req = *maybe_request;
            auto value = req.payload();
            std::cout << "[Server] Received request: " << value << std::endl;

            // Zero-copy response
            auto response = req.loan_uninit().expect("loan response");
            auto initialized_response = response.write_payload(
                TransmissionData{value + 10, value * 2, static_cast<double>(value) * 0.5});
            std::cout << "[Server] Sending response: " << initialized_response.payload() << std::endl;
            send(std::move(initialized_response)).expect("send response");
        }
    }
}

client.cpp

#include "iox2/iceoryx2.hpp"
#include "transmission_data.hpp"

using namespace iox2;

constexpr iox::units::Duration PERIOD = iox::units::Duration::fromSeconds(1);
constexpr iox::units::Duration WAIT = iox::units::Duration::fromMilliseconds(10);

int main() {
    set_log_level_from_env_or(LogLevel::Info);

    auto node = NodeBuilder().create<ServiceType::Ipc>().expect("node creation");

    auto rr = node.service_builder(ServiceName::create("My/Funk/ServiceName").expect("valid"))
                  .request_response<uint64_t, TransmissionData>()
                  .open_or_create()
                  .expect("open_or_create");
    auto client = rr.client_builder().create().expect("client");

    // Event used to wake up the server
    auto ev = node.service_builder(ServiceName::create("My/Funk/ServiceName/ReqEvent").expect("valid"))
                 .event()
                 .open_or_create()
                 .expect("open_or_create");
    auto notifier = ev.notifier_builder().create().expect("notifier");

    auto counter = 0U;

    while (true) {
        // Zero-copy request
        auto request = client.loan_uninit().expect("loan request");
        auto initialized_request = request.write_payload(counter);
        auto pending = send(std::move(initialized_request)).expect("send request");

        std::cout << "[Client] Sent request " << counter << std::endl;
        notifier.notify().expect("notify server");

        node.wait(WAIT);        // Optional client listener to wait for response event

        // Receive all responses for this request
        while (true) {
            auto response = pending.receive().expect("receive");
            if (!response.has_value()) {
                break;
            }
            std::cout << "[Client] Received response: " << response->payload() << std::endl;
        }

        counter += 1;
        (void)node.wait(PERIOD);
    }
}

---

Perhaps as a small remark from my perspective of a iox2 beginner I would find a high-level class or API helpful where, similar to Iceoryx1, you can pass the server or client object directly to the Listener or the WaitSet without having to manually define the events/notifiers and their service names e.g.:

listener.attachEvent(
    server,
    iox::popo::ServerEvent::REQUEST_RECEIVED,
    iox::popo::createNotificationCallback(onRequestReceived));

waitset.attachState(
    client,
    iox::popo::ClientState::HAS_RESPONSE);