Channel: A Name Space Based C++ Template Framework For Asynchronous Distributed Message Passing and Event Dispatching
Channel: A Name Space Based C++ Template Framework For Asynchronous Distributed Message Passing and Event Dispatching
Overview | Boost Channel | ACE Channel | Download at Sourceforge | Supported Platforms | Build | Contact / Support
Overview
Channel is a C++ template library to provide name spaces for asynchronous, distributed message
passing and event dispatching. Message senders and receivers bind to names in name space; binding
and matching rules decide which senders will bind to which receivers; then message passing and event
dispatching could happen among bound senders and receivers.
Channel's signature:
template <
typename idtype,
typename platform_type = boost_platform,
typename synchpolicy = mt_synch<platform_type>,
typename executor_type = abstract_executor,
typename name_space = linear_name_space<idtype,executor_type,synchpolicy>,
typename dispatcher = broadcast_dispatcher<name_space,platform_type>
>
class channel;
Various name spaces (linear/hierarchical/associative) can be used for different applications. For
example, we can use integer ids as names to send messages in linear name space, we can use path name
ids (such as "/sports/basketball") to send messages in hierarchical name space and we can use regex
patterns or Linda tuple-space style tuples to send messages in associative name space; User can
configure name space easily by setting a channel template parameter.
Channel's other major components are dispatchers; which dispatch messages/events from senders to
bounded receivers. Dispatcher is also a channel template parameter. Sample dispatchers includes :
synchronous broadcast dispatcher, buffered asynchronous dispatchers,...
Name space and dispatchers are orthogonal; they can mix and match together freely; just as STL
algorithms can be used with any STL containers.
By combining different name space and dispatching policies, we can achieve various models:
synchronous event dispatching
associative name space based on matching/look-up rules similar to Linda tuple space
asynchronous messaging model similar to Microsoft CCR (Concurrency Coordination Runtime)
Similar to distributed files systems, distributed channels can be connected or "mounted" to allow
transparent distributed message passing. Filters and translators are used to control name space changes.
Boost Channel
Boost Channel is the latest implementation of Channel framework for Boost. Boost provides free peer-
reviewed portable C++ source libraries. It is emphasized that libraries work well with the C++ Standard
Library. Boost libraries are intended to be widely useful, and usable across a broad spectrum of
applications. Boost Channel is solely based on standard boost facilities:
Boost::shared_ptr for message/event data life-time management
Boost.Bind, Boost.Function for callback
Boost.Thread for synchronization
Boost.Serialization for message marshaling/demarshaling
Boost.Regex and Boost.Tuple for associative name-matching
Boost.Asio and Boost.Shmem are used to build transports among remote channels.
Detailed Info:
Design Document
Browse CVS Source Code
ACE Channel
ACE Channel is the first implementation of Channel framework on top of ACE (Adaptive
Communication Environment) . ACE is a powerful and portable OO/C++ framework for system
programming. It provides not only wrapper facade classes to abstract the complete OS facilities, but
also frameworks and design patterns for developing multithreaded and distributed applications. ACE
Channel uses several key ACE facilities including Reactor, Service Configurator, Task and Acceptor-
Connector.
Design Docs and more info...
Browse CVS Source Code
Build
checkout boost cvs source code
download latest boost_channel_x_x.tar.gz
tar xvzf boost_channel_x_x.tar.gz
cd <top directory of channel>
copy subdirectory boost/channel/ to <boost root directory>/boost/
copy subdirectory libs/channel/ to <boost root directory>/libs/
cd to <boost root directory>/libs/channel/exmaple/<specific samples such as ping_pong>
bjam
Contact / Support
yigongliu@gmail.com
1. Introduction
2. Build
3. Tutorials
3.1 gui event handling
3.2 gui event handling with 2 local channels
3.3 distributed gui events
3.4 chat with direct connection
3.5 buffered channel with blocking active receiver (synchronous choice, join
synchronization patterns)
3.6 buffered channel with async receivers (asynchronous choice, join synchronization
patterns)
3.7 distributed chat thru a central server
3.8 channel connection thru shared memory
3.9 channel using regex name matching
3.10 channel using Linda-style associative lookup
3.11 channel name space management and security with filters and translators
3.12 port and signal: unnamed point of tightly-coupled local interactions
4. Design
4.0 Overall Design Idea
4.1 Name space
4.1.1 What's in a name?
4.1.2 Types of name space
4.1.3 Name binding set and Name matching algorithm, binding rules
4.1.4 Name spaces merge and connections
4.2 Dispatching
4.2.1 How message data move: push/pull, buffering
4.2.2 How operations are performed: synchronous/asynchronous
4.2.3 Message passing coordination patterns
4.2.4 Messages handling
4.3 Connection related
4.3.1 Connections
4.3.2 Peer
4.4 "Unnamed" binding of output/input or points of tightly-coupled local interactions
4.5 Application architecture and integration
5. Classes
5.1 name space related
5.1.1 name spaces
5.1.2 id_type and id_trait
5.1.3 name and name binding callback
5.1.4 named_out and named_in; publisher and subscriber
5.1.5 unnamed in/out: port and signal/slot
5.1.6 binder, filter and translator
5.2 dispatching related
5.2.1 dispatchers
5.2.2 messages
5.2.3 queues
5.2.4 executors
5.3 connection related
5.3.1 global functions for connecting channels
5.3.2 connection
5.3.3 peer and interface
5.3.4 streams
5.3.5 marshaling registry
5.4 platform abstraction policy and synchronization policy
5.4.1 platform abstraction
5.4.2 synchronization policy
6. Class Concepts and How to extend Channel framework
6.1 id_type and id_trait
6.2 name space
6.3 dispatcher
6.4 executor
6.5 queue
6.6 streams/connectors (or integrate into new architecture)
7. Compare Channel to others (plan9, STL)
7.1 Compare Unix/Plan9/Inferno file-system name space and Channel's name space
7.2 compare STL and Channel
8. Reference Links
1. Introduction
In Unix and most OSes, file systems allow applications to identify, bind to and operate on system
resources and entities (devices, files,...) using a "name" (path name) in a hierarchical name space
(directory system) which is different from variables and pointers in flat address space. Many
interprocess communication facilities (IPC) often depend on some kind of "names" to identify them
too, such as the pathname of FIFO or named-pipe, pathname for unix domain socket, ip-address and
port for tcp/udp socket , and keys for System V shared memory, message queue and semaphores. "The
set of possible names for a given type of IPC is called its name space. The name space is important
because for all forms of IPC other than plain pipes, the name is how the client and server "connect" to
exchange messages." (quote from W. Richard Stevens "Unix Network Programming").
Channel is a C++ template library to provide name spaces for asynchronous, distributed message
passing and event dispatching. Message senders and receivers bind to names in name space; binding
and matching rules decide which senders will bind to which receivers; then message passing and event
dispatching could happen among bound senders and receivers.
Channel's signature:
template <
typename idtype,
typename platform_type = boost_platform,
typename synchpolicy = mt_synch<platform_type>,
typename executor_type = abstract_executor,
typename name_space = linear_name_space<idtype,executor_type,synchpolicy>,
typename dispatcher = broadcast_dispatcher<name_space,platform_type>
>
class channel;
Various name spaces (linear/hierarchical/associative) can be used for different applications. For
example, we can use integer ids as names to send messages in linear name space, we can use string
path name ids (such as "/sports/basketball") to send messages in hierarchical name space and we can
use regex patterns or Linda tuple-space style tuples to send messages in associative name space; User
can configure name space easily by setting a channel template parameter.
Channel's other major components are dispatchers; which dispatch messages/events from senders to
bounded receivers. Dispatcher is also a channel template parameter. The design of dispatchers can vary
in several dimensions:
how msgs move: push or pull;
how callbacks executed: synchronous or asynchronous.
Sample dispatchers includes : synchronous broadcast dispatcher, buffered asynchronous dispatchers,...
Name space and dispatchers are orthogonal; they can mix and match together freely; just as STL
algorithms can be used with any STL containers by means of the iterator range concept, name space
and dispatchers can be used together because of the name binding set concept.
By combining different name space and dispatching policies, we can achieve various models:
synchronous event dispatching
associative name space based on matching/look-up rules similar to Linda tuple space
asynchronous messaging model similar to Microsoft CCR (Concurrency Coordination Runtime)
Similar to distributed files systems, distributed channels can be connected or "mounted" to allow
transparent distributed message passing. Filters and translators are used to control name space changes.
2. Build
Channel is continuously being developed and tested in linux (ubuntu8.04/g++4.2.4 - ubuntu9.04/g+
+4.3.3) and Windows (Visual C++ 2005 - Visual C++ 2008). The implementation is solely based on
standard boost facilities plus Boost.Asio and Boost.Interprocess.
Download: http://channel.sourceforge.net
Build: Channel is a header only library. There is no need to build the library itself to use it. Please
following these steps:
download or checkout boost distribution
download latest boost_channel_x_x.tar.gz
tar xvzf boost_channel_x_x.tar.gz
add boost's directory and Channel's directory to compilers' include path
cd to <channel_top_directory>/libs/channel/exmaple/<specific samples such as ping_pong>
bjam
3. Tutorials
the following are a few samples showing how different name spaces and dispatchers can be used in
various situations:
3.5 buffered channel with blocking active receiver (synchronous choice, join synchronization
patterns)
A sample shows the usage of buffered channels implemented thru a synchronous pull dispatcher. In this
channel configuration, messages are buffered inside channel at sender side. The receiver is active, a
thread blocking waiting for the arrival of messages at synchronous join/choice arbiters and then
processing the messages. details...
3.6 buffered channel with async receivers (asynchronous choice, join synchronization patterns)
This sample shows a buffered channel support asynchronous receivers using asynchronous
coordination patterns: choice and join. The callback actions are dispatched thru a thread pool executor.
details...
3.11 channel name space management and security with filter and translator
This sample demos how we can use filters and translators to achieve name space management and
security. details...
4. Design
The design of Channel is based on the following integration of Plan9/Inferno's name space idea
and Robin Milner's interaction thru names:
Channel provides process local private name_space which is customizable thru
connecting to other channels.
The semantics of names are changed; names do not refer to named resources (which are
relatively static entities), but to named "points of interaction" (which are mostly
dynamic); thus file-oriented api and semantics are dropped and Robin Milner's
operations/interactions on names are adopted as the api: calling/co-calling/matching.
4.1.3 Name binding set and Name matching algorithm, binding rules
No pure name exist; Names are ONLY created into name space when bound for sending/receiving
msgs:
Named_Out: output/send interface bound with name
Named_In: input/receiv interface bound with name
Name binding sets:
for Named_Out, its binding_set is the set of bound Named_Ins to which to send messages
for Named_In, its binding_set is the set of bound Named_Outs from which to receive messages
There are 2 aspects to the name matching algorithms and binding rules to decide binding_sets:
id matching: the id of Named_Out must "match" the id of Named_In based on matching
operation defined in id_trait
scope & membership matching: the membership and scope of both Named_Out and Named_In
must match. This doesn't mean they must be the same. For example, a local sender
(MEMBER_LOCAL) with SCOPE_LOCAL can bind to receivers with <MEMBER_LOCAL,
SCOPE_LOCAL> or <MEMBER_LOCAL, SCOPE_GLOBAL>. There is an internal table
recording all such valid combinations.
Named_Out and Named_In don't bind to each other directly (as in most event dispatching systems).
Instead, they bind to names in name space. Based on binding and matching rules, their binding set will
be resolved which will contain the direct pointers to their counterpart. Actual message passing and
dispatching happen on the binding set; never need to go thru name space again. So the actual message
passing and dispatching behaviour and performance should be the same as we have registered the
Named_In directly with Named_Out ( as we would have done in normal event dispatching systems ).
Based on name-matching, there are possibly the following 4 kinds of binding sets:
1 - 1: one named_out binds with exactly one named_in
1 - N: one named_out binds with a group of named_ins (e.g. when many subscribers subscribe
to the same name)
N - 1: one named_in binds with a group of named_outs (e.g. when a subscriber subscribes using
a wildcard name or regex pattern, it could receive from multiple sources)
N - M: both named_out and named_in bind with a group of counterparts.
4.2 Dispatching
Dispatchers or dispatching policies are operations or algorithms defined over name binding set. They
define the semantics of "interactions thru names". Based on RobinMilner's separation of calling and co-
calling, there are 2 parts defined for each dispatching algorithm:
sending (or sender) algorithm: corresponding to calling
defined over the set of bound Named_In (receiver) objects
may contain message buffering mechanism inside channel (Named_Outs)
receiving (or receiver) algorithm: corresponding to co-calling
defined over the set of bound Named_Out (sender) objects
may support high level messaging coordination patterns (such as Choice and Join)
The following are major design considerations for dispatchers.
4.3.2 Peer
the common interface for connection peers: interfaces and streams
interface:
proxy of peer channel
core of channel connection logic:
. how remote binding/unbinding events will effect local name space
. how messages are propagated from and to remote channels
stream:
Stream is used to wrap a remote transport connection (socket, pipe or message queue inside
shared memory).
In earlier implementation of Channel on ACE[8], a Connector class is included as one of the core
classes; which will connect local and remote channels. the disadvantage of this design is that
Channel is tied to a specific architecture (such as thread per connection); making it difficult to
integrate channel with other existing servers.
In plan9/inferno, when we mount a remote name space to local, the real function is to mount a
descriptor (file, pipe, or socket connection) to a specific point in name space.
Following this style, remote channel connection is to connect/mount a "stream" to a local
channel/name_space; the stream wraps a socket, pipe, or shared_memory message queue
connecting a remote channel in another process or machine. thus avoid interfering with servers'
internal design: such as threading; so that channel can work well with both single-thread async
and multi-thread sync server design.
5. Classes
5.2.1 dispatchers
As we discussed above, dispatchers have 2 parts: sending and receiving algorithms. Dispatcher's API
are not fixed, depending on whether it uses push or pull model and it is synchronous or asynchronous.
The following sample dispatchers are provided:
push dispatchers:
broadcast_dispatcher:
senders/named_out broadcast messages/events to all bound receivers/named_in. This is the
most common event dispatching semantics.
round_robin_dispatcher:
senders/named_out send messages/events to bound receivers/named_in in round-robin
manner. Simple server load balance can be achieved thru this.
always_latest_dispatcher:
senders/named_out always send messages/events to the latest bound receivers/named_in.
This is a dispatcher to simulate plan9's union directory (though most semantics is achieved
thru nam space binding/connection). Suppose we use an id (such as "/dev/printer")
represent a printer resource. To print something, we send a message to that id. On another
machine there is another printer bound to the same id in their local channel. To be able to
use the 2nd printer, we could connect or mount the remote channel to local channel. Then
if always_latest_dispatcher is used, all following printouts (sent to /dev/printer) will come
from the remote printer. The local printer will get print messages again after the channels
disconnect.
pull dispatcher:
In pull dispatcher, messages/events are buffered inside channel at Named_Outs, high level
messaging coordination patterns - "arbiters" are defined at Named_Ins to decide when and how
messages are pulled from Named_Outs and consumed by receiving threads or callbacks.
synchronous arbiters (choice_sync, join_sync):
Both senders and receivers are active threads. Messages are buffered inside channel at
sender/named_out side and sending thread returns right away. Receiving threads block
waiting messages at synchronous arbiters. They unblock and process messages when
messages are available at named_outs and their associated arbiters fired.
asynchronous arbiters (choice_async, join_async):
Callbacks are registered with asynchronous arbiters. Messages are buffered inside channel
at sender/named_out side and sending thread will notify receivers before return.
Depending on arriving messages, asynchronous arbiters will decide which callbacks will
fire; and schedule them to execute in an executor. Join arbiters will quarantee that related
messages are consumed atomically.
5.2.2 messages
Application message/event data can be any data type : primitives, structs and classes. For remote
message passing, proper serialization functions must be defined using Boost.Serialization:
free serialization functions for non-intrusive serialization
message struct and classes can define serialize() methods; and should define default
construtor for serialization, otherwise save_construct/load_construct need to be
overwritten.
Please refer to the tutorials for sample message definitions.
5.2.3 queues
Queues are used for inside channel message buffering. One of pull dispatcher's template parameter is
queue type. Various applications can specify and use different queue types based on applications'
requirements and queues' capability. Queues will support the following common interface:
void put(elem_type & e);
void get(elem_type & e);
5.2.4 executors
Executors allow us avoid explicitly spawning threads for asynchronous operations; thus avoiding
thread life cycle overhead and resource consumption. Executors should support the following
common interface to allow application register asynchronous operations to be executed later and cancel
this registration:
template <typename task_type>
async_task_base * execute(task_type task);
bool cancel(async_task_base *task);
5.3.2 connection
Class connection represent the connection between 2 channels. Deleting a connection object will break
the connection between 2 channels; and deleting any of the member channels will result in connection
object being deleted.
5.3.4 streams
Streams are proxies for remote channels and wrap transport mechanisms. The following streams are
and will be provided:
asio_stream: a stream class using Boost.Asio socket to connect to peer channels.
asio_connector: a helper class providing 2 functions:
publishing local channels at specific ports (so that remote peer channel can
connect)
template <typename sock_conn_handler>
void async_accept(int port, sock_conn_handler hndl) ;
connecting to remote channels at their publication addresses (host, port):
template <typename sock_conn_handler>
void sync_connect(std::string host, std::string port, sock_conn_handler
hndl) ;
template <typename sock_conn_handler>
void async_connect(std::string host, std::string port, sock_conn_handler
hndl) ;
shmem_stream: a stream class using Boost.Interprocess shared memory message queues to
connect to channels in a separate process inside the same node.
soap_stream (coming): use SOAP protocol to connect to remote channels
6.3 dispatcher
Dispatchers are used as policy classes for channel template. As discussed above, each dispatcher
contains 2 algorithms: sending and receiving.
Dispatchers' API are not fixed, depending on whether it uses push or pull model and it is synchronous
or asynchronous. The API of provided dispatchers follow the general convention of providing various
send() and recv().
1. Primary requirements
Each dispatcher class should define 2 nested types:
sender
recver
These nested types are the parent classes of named_in and named_out.
Inside dispatcher nested types (sender and receiver classes), dispatching algorithms retrieve name
binding set from associated "name" object.
2. Secondary requirements
For dispatchers which are used in channel types with possible remote connections, the nested
receiver classes will expect the callback function's signature as :
void callback(id_type id, boost::shared_ptr<void> msg).
This requirement is because of the implementation of "interface" class.
Here is a detailed description of a sample pull dispatcher.
6.4 executor
6.5 queue
7.1 Compare Unix/Plan9/Inferno file-system name space and Channel's name space
In Unix and other OSes, file system provides the machine-wide hierarchical name space for most
system resources. Applications use resources mostly by the standard file system calls:
open/close/read/write. By mounting remote file-systems, remote name space (and resources) can be
imported and accessed transparently by local applications.
Plan9/inferno push this idea further by 3 ideas: 1. all resources are represented as files. 2. each process
has its own private name space which can be customized according to applications' requirements. 3. an
uniform protocol - 9P is used for all remote message passings.[1][2]
Channel provides a process local name space for asynchronous message passing and event dispatching.
Compared to unix/plan9 name space:
Channel's name space is a light-weight user-space data structure; A process can create and use
multiple name spaces for different purpose. Unix(plan9/inferno)'s name space is a more
fundamental kernel feature well integrated with all system facilities (shell, window system,..).
Each process has only one.
file-system name spaces are based on function-call (local or remote procedure call) or (request-
response) semantics. Channel's name space is for asynchronous message passing or one way
request.
file-system name spaces are based on normal client-server model: file servers purely serve; ie.
clients will import names from servers, but servers never import names from clients. Channel is
peer-peer model; connected channels will import names from each other for communication.
In file-system name spaces, names refer to assumely stable/permanent entites (either disk files
or long-running servers); file name spaces are relatively static, ie, a specific name mostly refer
to the same resource either local or from a specific server; operations on names with stale/empty
binding will result in serious errors. Channel name spaces are purely dynamic. It is totally valid
to have a Named_Out object in name space without bound Named_In object (since message
subscribers may join in later). The binding of names (bound senders or receivers) can be
different between this and next invocations. Just as RobinMilner has clarified [3]:
"... is built upon the idea that the respondent to (or referent of) a name
exists no more persistently than a caller of the name. In other words, the notions of
calling and responding are more basic than the notions of caller and respondent; every
activity contains calls and responses, but to have a persistent respondent to x one that
responds similarly to every call on x is a design choice that may be sensible but is
not forced."
File-systems identify entities by string path names in a hierachical directories. Channel use
different naming schemes in different applications: linear name space (such as integer ids),
hierarchical name space (such as string path names), and associative name space (linda style)
in plan9, request dispatching is unicast - only one server get req and serve it. Channel can
support various dispatching policies - broadcast, unicast, buffered,...
file-system api is stream oriented: byte-streams are read from file or write to file. channel's api
is discrete message oriented.
8. Reference Links
[1] Preface to the Second (1995) Edition (Doug McIlroy)
[2] The Use of Name Spaces in Plan 9 (Rob Pike,...)
[3] What's in a name? (Robin Milner)
[4] Turing, Computing and Communication (Robin Milner)
[5] Comega
[6] CCR
[7] Java's executor
[8] http://channel.sourceforge.net