posted on Friday, April 09, 2010 - 07:21 am
I want to know how threads are implemented in DDS ? As I understand, the publishers, Subscribers. data writers, data readers are all created as seperate threads as and when the entities are created, but I am still not very sure if I have understood it correctly. I also want to know if there are any threads implemented to handle the background tasks, which manages the services in the DDS (that is handling of the data, keeping track of the publisher and subscribers etc). Please clarify.
In many DDS implementations (including our OpenSplice DDS implementation), your assumption of DDS-entities being 'represented' by separate threads is not accurate. The processing related to writing and/or reading samples is typically executed in the context of the application. Asynchronous API's such as listeners have their own (middleware) threads and also data-management tasks such as maintaining non-volatile data, providing reliable communication, managing data-lifecycle etc. require services from the middleware that are asynchronous and decoupled from the application processing and can either be implemented as middleware threads of the application-process or require fully separate middleware services as the lifecycle of a specific application-process is not allowed to impact the lifecycle of information in many cases (for example to provide fault-tolerant reliability and/or storage of non-volatile transient/persistent data for late-joining applications)
posted on Friday, April 09, 2010 - 10:17 am
You said "application. Asynchronous API's such as listeners have their own (middleware) threads"
What I understand from the above statement is that, there is a listener for each entity, running on a seperate thread. If so, suppose there are 5 datawriters and 5 data readers, each having their own listners on seperate threads (i.e. 10 threads running in parallel). Will this not affect the performance with number of threads increasing.
Also if there are 10 subscribers, 5 subscribers on 1 topic and 5 subscribers on another topic. How many connection threads are created? is it 1 thread per topic or is it 1 thread per subscriber.
First to clarify a little the 'need' for using listeners: for writing you typically don't need/utilize listeners, for reading data, you can choose to either use a 'waitset' (like a unix 'select') which is executed by the application-thread (so no middleware thread) or use an asynchronous listener which is executed by a middleware thread (we utilize a domainParticipant-specific thread for that). Personally I like waitsets better for 2 reasons: 1) it allows complete application-control as its executed by an application thread and 2) having middleware-threads running unknown application-listener-code is always 'tricky' as we/middleware don't know how well such code behaves.
W.r.t. number of 'connection-threads', thats very dependent on the DDS implementation. In OpenSplice it would be basically 1 Tx-thread (on sending-node) and 1 Rx-thread (on receiving-node) regardless of the number of publishers/subscribers on each node as we've 'offloaded' the realtime networking to what we call a 'network-scheduler' that schedules all network-traffic based upon urgency (LATENCY_BUDGET QoS Policy) and importance (TRANSPORT_PRIORITY QoS Policy) in order to provide an optimal balance between efficiency and determinism.
Finally, to further prevent unbounded priority-inversions, we also support multiple priority-bands that are dynamically choosen by the actual TRANSPORT_PRIORITY of any published data samples. Each of these configurable priority-bands require a Tx/Rx thread-pair at proper OS-priority/scheduling-class and can also do some fancy traffic-shaping w.r.t. allowed bandwidth and related automatic throttling.
Hans has provided a good overview of the threading issues with DDS and a look at the OpenSplice implementation. With OpenDDS we have a similar threading architecture with some variance in the details.
OpenDDS provides "Transport" objects which individual publisher and subscriber entities can bind to. Any given publisher or subscriber can be attached to only a single transport object at a time, but a transport object can be attached to many publisher or subscribers. These objects then provide a receiving thread for inbound traffic and can be configured to provide a separate thread for sending samples. Scaling is managed by limiting the number of transport objects used.
An additional thread is available within a transport object to handle any samples that are queued due to back pressure in the network; these samples are sent using this middleware thread as the network can accept the traffic.
OpenDDS also establishes the middleware transmission threads at a priority/scheduling class determined by the TRANSPORT_PRIORITY QoS policy value.
In addition to these data handling threads, OpenDDS also processes using an additional middleware thread to provide the various internal services such as durability processing and timer management. This is all done within the participant process and not an external process.