Hi, I'm now learning the DDS solution, and have read the available articles and code examples. I would like to run a demo on my computer(s). What is needed in order to begin (enviroment, compiler, computers, etc.)? tnx, Rozy
Probably the first thing you will need to do is to select a DDS compliant product. That product usually comes with some documentation that tells you about supported platforms, compilers, etc. A Getting Started Guide should help you on your way by letting you build and run some example code.
A Domain can be considered a 'physical' partition. There is no communication at all between services or applications in different Domains. One entity can only reside in one single Domain and is only able to communicate with others in that same Domain.
Within a Domain you can introduce any number of partitions to limit your communication to a specific set of Entities. Entities can be associated with multiple Partitions at the same time.
Therefore one data instance can only be available in one single Domain, but it can be in multiple Partitions.
posted on Thursday, January 08, 2009 - 03:58 pm
Hi Niels, thanks for reply.
I have few questions: 1) Can a Topic be written in one partition and read in another partition? 2)Why to use Pub/Sub?Can't we connect Data writers directly to Data readers? 3)what is the difference b/w Listener and Subscriber?
1. The partition(s) of the publisher determines where the data will be available. If a publisher publishes data in partition X, then only subscribers that are 'attached' to X will be able to receive the data. Publishers and subscribers can use multiple partitions by using regular expressions (even with wildcards).
2a. Publishers and subscribers provide facilities that allow influencing how data changes are presented to the subscribing application (ordering, coherency). 2b. The partition QoS policies of Publishers/Subscribers allow using different 'communication planes'. For instance to make a distinction between different combinations of publishers and subscribers that need to communicate with each other or to make a distinction between the semantic meaning of the data instances in different partitions.
3. I am not sure I understand this question, but I'll give it a try. A Subscriber is the Entity that is responsible for the actual reception of the data resulting from its subscriptions. A Listener can be attached to an Entity and allows your application to be asynchronously notified if the 'communication status' of an Entity changes. The possible status changes depend on the type of the Entity. A SubscriberListener for instance allows you to be notified in case new data has been received in one of its attached DataReaders.
Partitions typically specify a 'context' in which published/subscribed data lives and can be 'physical' or 'logical' by nature. Some examples: a. logical-context: maybe in your system you want to distinguish simulated-data from 'real' data whereas the data (topics) itself are the same. In this case you could create 2 partitions to 'hold' (and separate between) the 'real-world' and 'simulated' data. b. physical-context: maybe you have a distributed system where some data needs to 'stay' in the LAN-environment/scope where as some other data has a broader reach and should be available through WAN connections too (probably at a lower rate to accomodate for the physical capabilities of the WAN). In that case you could publish the information that is to stay within the LAN in a separate partition from data that 'spans' the WAN. Some DDS implementations further support this 'physical' oriented usage of data-partitioning by allowing DDS-partitions to be mapped on multicast-groups in order to benefit from the (hardware-)capabilities of multicast enabled networks
2) about 'why pub/sub'
I think your questions relates to why using publish/subscribe rather than what the publiser/subscriber entities represent in the DDS specification, so let me try to answer the question in that context: There is indeed middleware where writers/readers are directly connected to each other (also called 'connection-oriented' middleware for that reason) but the true power of publish/subscribe is that its NOT connection-oriented and with that supporting much better dynamic/asynchronous environmments where information exchange is based upon 'actual' or 'current' interest which can occur spontaneous by emerging publishers and subscribers of the 'same information'. This model also suppports implicitly the 1-to-n communication pattern i.e. when there are multiple interested parties (subscribers) to some kind of information. The other big advantage is that there is no need for a-priori 'global-knowledge' in each (local) component since the publishers and subscribers are principally un-aware of each others. A everyday example of this is when you subscribe to a newspaper: you are not 'connected' to the source of the information in anyway, you don't need to know who/where/when produced the news (the journalist)or how/when/where the newspaper was printed or even how the newspaper is delivered to your home, the only thing you care about is that every morning at 7am, the newspaper is on your doormat.
3) Listener and Subscriber
I understand your confusion since these names are tightly coupled to the DDS-specification. An application subscribes to information by creating a 'subscriber' which is a sort of factory for creating 'readers' for certain types of information (the 'topics'). A listener allows you to be triggered 'asynchronously' when data arrives at your reader (a 'call-back' implementation as it is called in many middleware environments) and alternatively you can use a 'waitset' as a mechanism to 'block' for data to arrive which then can be viewed as a more 'sychronous' way of interaction since it allows an application to 'wait' (a 'blocking call' as it is called in many middleware environments) i.e. to synchronize with the arrival of subscribed information
A DomainParticipant represents the participation of an application in a so-called Domain. A Domain can be considered a virtual network that links applications using the same domainId and isolates them from applications that participate in different Domains. This allows independent distributed applications to coexist in one physical network without interfering with each other.
Best regards, Niels
posted on Monday, January 12, 2009 - 02:21 pm
hi Niels, Thanks for reply.But I am still confused about the role of DomainParticipant?Why do we require this?Why can't an application directly tell the domain about it's participation in that domain?
The most important reason is that a DomainParticipant also provides facilities that allow the application to locally ignore the information from any other given DomainParticipant (ignore_xxx methods). Furthermore, the DomainParticipant provides facilities for the application to be notified of changes in its 'communication status' (using a Listener just like the other entities).
posted on Tuesday, January 13, 2009 - 12:19 pm
I am still not convencied about the use of DomainParticipant.All you have told above can be done by application directly.Why to have DomainParticipant in between?
In order for an application to participate in a DDS-domain, the DomainParticipant class has been created to facilitate this process, i.e. to make the participation 'known' to the DDS infrastructure as well as to provide factories to the application by which subscribers/readers, publishers/writers and topics can be created by the application.
So as the spec says, its (just) an entry-point of the Service.
posted on Tuesday, January 13, 2009 - 04:07 pm
Hello Guys, Iam new into DDS. Can anybody let me know the differences between data modeling, application modeling, information modeling and deployment modeling....
With Regards.. Sandeep.....
posted on Tuesday, January 13, 2009 - 04:14 pm
Hi Hans, Thanks for reply. I have read that DDS uses UDP and it supports priority.I wonder how it can support priority when the underlying protocol(UDP) itself does not support priority?
posted on Tuesday, January 13, 2009 - 04:43 pm
Hi, I am new to DDS.Can anybody please tell me what is 'low footprint'?
In a data-centric architecture, which DDS represents, defining the structure/semantics and required behavior (QoS) of 'information' and its availability is of prime importance. DDS as an architecture and supporting infrastructure provides some unique means of 'annotating' the information with so-called Quality_of_service (QoS) policies w.r.t. the urgency and importance of information, the required durability (persistence) of information etc. Also the fact that this activity of information-modeling can be done prior and outside the (engineering of) the applications is a big plus since application-developers can then concentrate on creating the business-logic rather than worry about how information gets distributed and made available to them.
2) application modeling
Again, in a datacentric (DDS) context, the middleware will make and keep information available for applications that want to 'interact' with it (i.e. producing information as a publisher, or consuming information as a subscriber). To make this interaction as easy as possible, DDS also supports a set of QoS policies to define 'history' for easy access of historical data, filter on frequency and/or content, etc. Be splitting information- and application-modeling, application-developers can fully concentrate on the business-logic that would typically convert subscribed information into published information. The DDS infrastructure then provides a time- and space-decoupling between these applications by maintaining information for late-joining applications as well as do the dynamic routing from information publishing applications to information consuming applications
3) Deployment modeling
When the information is defined and the applications that use it are developed, the applications must of course be deployed on a 'target system'. Such a system will exist of enterprise and/or embeddedd computing nodes, networks, storage etc. Deployment modeling then comprises of configuring the middleware in such way that the available resources are utilized in an optimal way. Examples are the definition of 'resource-limits' (again a DDS QoS-policy), configuration of priority-lanes over the network (where the application-importance expressed by the TRANSPORT_PRIORITY QoS-policy of the information will drive the dynamic/automatic choice of the 'right priority-lane'. Its like in real life, deployment modeling is about defining the housing and transportation infrastructure. DDS is unique in that it provides a number of 'logical' QoS policies that can be mapped on 'physical' features of the underlying system for example transparent/runtime mapping of a logical-grouping of information (callled a 'partition') on a physical multicast group or mapping of a logical transport-priority on a pre-configured priority-lane over the network. The separation between information/application/deployment modeling again supports a 'separation of concern' i.e. definition of the information (data, its relations and required QoS policies w.r.t. distribution and availability) can be done by experts, development of applications that transform information can be done by application/business-logic experts and finally deployment of these de-coupled and autonomous applications can be performance by infrastructure-experts that can assure optimal usage of available resources.
Hope this helps a little,
posted on Tuesday, January 13, 2009 - 05:46 pm
Hi, I am learning DDS.I have read in the context of 'Data Readers and Subscribers' that there is one of the way of notifing application about the data availability which is called polling.I want to know why to have 'poll' for data reader?Will it not be always a preferred choice to receive data that as soon as it is available(as in Listerner Callback Routine)?Does poll and Listener callback routine serve different purposes? Thanks, Sachin
Indeed, DDS implementations typically use UDP/IP as their underlying transport mechanism. However, this is not required nor specified by the OMG DDS specification. One of the QoS settings attached to DDS DataWriters is an integer value TRANSPORT_PRIORITY. Whenever a sample is written by the dataWriter, its TRANSPORT_PRIORITY value is attached to that sample as well. This value is a hint to the middleware and allows the middleware to take decisions on when and how to transport the sample according to the desired priorities. Depending on how the middleware has been designed, priority mechanisms can be implemented on different locations. We are not just talking about 'UDP transport priority' here, but end-to-end (dataWriter to dataReader) transport priority. The OpenSplice implementation, for example, has been designed to use so-called priority lanes. These are responsible for distributing data with the correct priority. If properly configured, different priority lanes will run as threads at different OS threading priorities. Each sample will be picked and distributed via the priority lane corresponding to the attached TRANSPORT_PRIORITY value. This is all done in the context of a separate service (process). That way, lower priority samples can be overaken by higher priority samples, becausee the latter will be transported via the higher priority lane with the higher threading priority. Essentially, this mechanism decouples the data distribution priority from the application threading priority and it assures that higher priority messages will be placed on the wire before lower priority messages. Hope this helps, Reinier
posted on Tuesday, January 13, 2009 - 07:49 pm
Hi Reinier, Thanks a lot for such a wonderful answer. Regards, Bob
DDS provides multiple mechanisms to monitor the availability of new data in your DataReader: 1) The most simple one is to let your application check periodically if there is data available, for instance by calling the 'take' operation on the DataReader and see if it returns any data. This method is often referred to as 'polling' and is very easy, but not very efficient just like you said.
2) The second mechanism is the usage of Listeners. You can attach a Listener to your DataReader to make sure your application is notified of any relevant events (in your case the availability of data). The actual notification is performed by a middleware thread. The thread calls the method of the Listener that matches the event that occurred and you can implement that method yourself. A Listener can be attached directly when you create the DataReader or on later moment in time. The usage of a Listener may require a bit more coding in your application then with 'polling', but it prevents your application from doing unnecessary work when no data is available.
3) The third mechanism is the usage of a so-called Waitset i.c.w. Conditions. You can create relevant Conditions and attach them to a Waitset. If the 'wait' method of the Waitset is called, the calling thread is blocked until one or more Conditions attached to the Waitset become true. By inspecting the Conditions that are returned by the 'wait' method, the application can determine what events have occurred. This implicates that the waiting is done in an application thread, which allows full control over the thread by the application itself. This is the largest difference with the Listener mechanism. The Waitset mechanism mostly requires the most code, but is also the most flexible solution.
All mechanisms can be used at the same time in an application by the way.
Best regards, Niels
posted on Tuesday, January 13, 2009 - 11:02 pm
Hi Sachin and Niels,
In case of high volumes of data where throughput becomes more important than reactivity polling will reduce processing overhead. Polling will process more data samples per read operation and induce less context switches than when using listeners or waitsets. Listeners and waitsets will tend to perform a context switch and execute a read operation per data sample.
Best regards, Robert
posted on Wednesday, January 14, 2009 - 05:12 am
Hi Hans, Thanks for reply.It was well described.Can u please elobrate more on Application modelling?If possible please give an example showing all modellings.
posted on Wednesday, January 14, 2009 - 05:24 am
Hi Niels, Thanks for reply.Does 'read' operation also comes under 'polling'?
Yes, the 'read' operation can also be used for 'polling'. In contradiction to the 'take' operation, the 'read' operation does not remove data from the DataReader when you read it. To prevent your application from reading the same data over and over again you need to use the NOT_READ sample_state as parameter for the read operation.
Best regards, Niels
posted on Wednesday, January 14, 2009 - 08:48 am
Hi Niels, Thanks for reply.Can one DomainParticipant be used in multiple partitions?Ans also can multiple DomainPartcipants be created in one domain?If yes then I want to know the circumtances in which this exercise is required? Regards, Sachin
posted on Wednesday, January 14, 2009 - 08:58 am
Hi, I am new to DDS.Can anybody please tell me what is 'low footprint'?
Look at the post of Hans above in this thread for the explanation of the difference between a Domain and a Partition.
You can create any number of DomainParticipants for one Domain in your application, but typically one application will create only one single DomainParticipant to attach to a specific Domain.
Best regards, Niels
posted on Wednesday, January 14, 2009 - 09:08 am
Hi, suppose we have huge data,and we set QOS policies for all the entities as 'transient'.How long DDS would we able to keep this data in memory?What happens if memory becomes full?How DDS handles this situation?Will then there be no further storing of the data ?
posted on Wednesday, January 14, 2009 - 09:13 am
Hi Niels, I mistakenly used the wrong word in my question.I wanted to ask "Can one DomainParticipant be used in multiple partitions?Ans also can multiple DomainPartcipants be created in one Partition?If yes then I want to know the circumtances in which this exercise is required"?
posted on Wednesday, January 14, 2009 - 09:21 am
Hi Robert, Thanks.As you said Polling will process more data samples per read operation and induce less context switches than when using listeners or waitsets.I want to know if polling provides good performance then Under what conditions listeners will be used? Thanks, Sachin
In the DDS context, the "footprint" can be interpreted as a combination of 'resource-usage' metrics w.r.t.
a) Application size (footprint)
The 'DDS-impact' on both the static size of an aplication (code/data) as well as the 'DDS-induced' dynamic memory-usage expressed in stack and heap memory.
I believe most DDS implementations provide the DDS API's by means of a shared library that is linked to each application (so that it only counts once regardless of the number of applications that run on a single machine). Also here this can be expressed in code/data size.
When DDS-functionality is implemented/supported by pluggable services (in case of OpenSplice DDS for instance for remote SOAP-connectivity, fault-tolerant persistence, priority-pre-emptive networking) also the 'actual' static/dymamic size of the set of configured services should be counted as part of the footprint.
d) shared memory size
Again some DDS implementations such as OpenSplice DDS utilize a ringfenced shared-memory segment to hold, share and manage the DDS-topic data for all 'nodal' applications. This is both very fast as well as very efficient since it prevents multiple copies of the the same data to exist within a single machine yet providing each application with its private 'view' on the data (w.r.t. what data is new/old, read/not-read etc.).
In case of OpenSplice DDS, with low footprint we mean:
1) application sizes
A simple chatroom application has an application size (code+data) of 28Kbyte. With stacksize not exceeding 64 Kbyte
2) shared library size
This depends somewhat on the application language and varies between 1 and 3 Mbyte. Please note that this is a 'shared object' (.so) which means it is present in memory only once, regardless of the number of applications
3) service sizes
Static-sizes are between 100 and 200 Kbyte per pluggable service (networking is around 100Kbyte, durability is around 200Kbyte), total dynamic runtime size is around 400 Kbyte for a configuration with priority-pre-emptive networking and fault-tolerant durability.
4) shared memory size
Since the DDS-specification includes a set of pre-defined so-called 'built-in-topics' that provide information about all available DDS-topics (i.e. is some sort of 'meta-data'), there is a minimal required size to hold this as well as some generic 'management data' of around 1 Mbyte of shared-memory and anything more depends on the amount of information that is shared between applications (i.e. published and or subscribed information expressed in DDS-resources: number of topic-samples, size of topic-samples, required 'history-depth' of readers, etc). With the important remark that by smart-sharing of the actual information there's always only a single copy of each 'sample' when/where-ever used within a single (SMP)machine. This greatly reduces footprint when compared to implementation where each DDS dataWriter and dataReader maintains an its own copies of data (in the so-called writerhistory and reader-history).
A DomainParticipant is not bound to a single Partition nor is a Partition bound to a single DomainParticipant. A Partition can be introduced by means of the Partition QoS policy of Publisher and Subscriber entities. You can create any number of Publishers and/or Subscribers in one single DomainParticipant. A Publisher and Subscriber will only communicate with each other if they share a common Partition.
Using Partitions can be useful in the situation where you want to introduce a different semantical meaning data instances of the same Topic. Consider the example of Hans with a 'real' world and a 'simulation' world in one Domain. If you want to train people in using your applications, you could easily deploy the same applications in the 'simulation' world. People could use 'real' world information and see consequences of their actions in the simulation world without affecting the real world.
Does this answer your question? Best regards, Niels
Perhaps it is an idea to create separate threads for each discussion? This makes it easier for other people to follow each individual discussion and provide input on the subject.
The usage of polling or listeners depend on your usecase and requirements.
In a very short and to the point answer, you would use listeners if you have an event based system in which you want to know as soon as possible when new data is available. The benefit of listeners are (among others): - You know ASAP when new data is available - If you have data that is updated very infrequently, then with polling you would run the chance of doing too many or too little polls, for infrequent data it is hard to determine how many times per second, or even minute to you poll for new data?
The drawback of listeners are (among others): - In scenarios where a lot of data is incoming per second you get far more triggers and thus context switches, thus an impact on performance
Now as for polling, I would use this in scenarios where the application decides when it needs new data (like automated once per second, or a button pressed by a user to update everything). Alternately when you are dealing with an application that needs to process high volumes of data, polling is a good option as well, as polling will perform better due to the less context switches Robert described.
So advantages of polling (among others): - Control of when applying updates is with the application - Better performance when dealing with high throughput
Disadvantages of polling (among others): - you need to figure out when you want to poll for new data, unneeded polls just induce load on the system, which is probably not what you want - You're not reactive, an update might be available just after you polled for data, so you do not see that data until the next time you poll for data.
posted on Wednesday, January 14, 2009 - 01:00 pm
hi, can anybody please tell me what is the difference between 'Topic-instance' and 'sample'?Can there be multiple instance and multiple samples of one topic?
Now say you define the field 'x' as the key field. (in opensplice you'd write the line '#pragma keylist example x' t accomplish this)
Each time that field has a different value, we are talking about a different instance. I.e. an instance is defined by the value of the keyfield of the topic. So multiple instances of a topic can exist.
Now DDS allows for multiple samples to be maintained per instance, so those samples have the same value for the key field (the 'x' field in the example topic), but can have (not necessarily) have different values for the 'someField' field in the example. Now multiple samples per instance are not enabled by default, you need to set specific QoS settings to their correct values, in this case the history QoS. by default DDS will maintain only 1 sample per instance, but if in your use case you find it useful to maintain more, then that's possible.
posted on Wednesday, January 14, 2009 - 03:19 pm
Hi, Thanks Niels.I got my answer.
posted on Wednesday, January 14, 2009 - 03:20 pm
I am trying to write a Java application that functions on 3 domains. However, I am unable to get DomainParticipants on them.
Originally, I took the default ospl.xml file and made the following modifications: 1. Change the name of the domain 2. Change the address of the networking * Best-effort = 3X40 * Reliable = 3X50 * Discovery = 3X60 ... where 'X' above is replaced with the number of the domain (1,2,3)
Can you please give an example of 3 domain configurations that would facilitate this? Perhaps I'm just confused on how to appropriately set up the "networking".
In OpenSplice DDS, a 'Domain' is represented by its URI pointing to the configuration file that defines the 'settings' for that domain i.e. size of the shared-memory to effectively share information within the node, the set of pluggable-services that are to be used on 'this' machine for the domain (i.e. for networking, durability, remote-soap connetivity, RDBMS-connectivity, etc.) and finally for each service the configuration-parameters like in case of networking, the IP-addresses and port-numbers for each network-channel (i.e. a priority-band)and these (IP-address, portnr) should of course match for different nodes that eventually populate a domain) in order for communication to be possible.
Now my question to you is, do you really want to use multiple/static domains or does it make more sense to use DDS-partitions which are much more powerfull to partition your 'global dataspace' in that publishers/subscribers can be attached to multiple partitions even allowing wildcards to for instance allow a subscriber to dynamically 'join' any emerging partition(s) that publishers might start publishing data in.
To use the (logical) DDS-partition concept, no configuration options need to be added/changed in order for them to work as expected. Its just that in our OpenSplice DDS implementation one can optimize the system's efficiency by defining mapping rules (in the networking-service setup within the $OSPL_URI configuration-file) of logical DDS-partition(s) to physical 'network-partitions' (that are characterized by a name and multicast group). In that way the logical partitioning of the global dataspace will be supported by the physical multicast group/join features of modern UDP/IP based networks.
Hi How can we secure data in DDS. We are publishing data over the network. So any one can access it with certain effort. There may be requirement when a publisher wants to publish data for a few subsciber?
For this purpose you will need a secure networking implementation that uses a secure networking protocol. Although this is not part of the DDS standard (yet), there are probably several DDS implementations that already provide it. I know that the commercial edition of OpenSplice DDS provides one for you. (It is not part of the community edition though, you'll have to explicitly request it.)