posted on Friday, September 10, 2010 - 10:26 pm
So I have a certain topic (call it Orders) that contains X number of individual tracks populated by 3 publishers. These are all unique shipments with individual paths. I have another set of subscribers who each monitor all the tracks.
How do i set up the history and key values of the publishers and subscribers so that late joining subscribers receive all the tracks and their history.
Also I need to be able to query Orders to process them based on certain fields contained in the message.
My current solution is to rely on an external SQL database that keeps up to date by also subscribing to Orders and saving the state in a DB table. That way i can use SQL queries to process the data instead of relying on the DDS cache.
I assume all 3 Publishers write 'unique' tracks, i.e. tracks whose identities do not overlap with tracks from other Publishers? In that case you can use 2 keyfields: 1 unique identifier for each Publisher, and 1 sequence number that is maintained on the Publisher level. The identity of a track that way depends on the unique combination of its source publisher AND of the sequence number within that source publisher.
If you want to store all tracks for late joining subscribers, you simply mark the Track topic's Qos as transient, in which case the durability service will maintain a copy for all ALIVE tracks (i.e. tracks hat have not yet been 'disposed') that have been written by TRANSIENT writers and send them as 'initial data' to all late joining datareaders that also have selected the TRANSIENT Qos. At reader creaion time, the reader will directly request the initial data from the durability service, but since this might take some time, you might want to wait for the initial data to be complete using wait_for_historical_data before actually accessing data from this reader.
If you want to query for certain tracks in your readers, you can simply do so by creating a QueryCondition to which you pass an SQL statement (similar to the statement you now pass to your database). Instead of reading/taking data from the reader directly, you now read/take the data through the QueryCondition using read/take_w_condition.
Be aware though that the QueryCondition is part of an optional DDS profile, and that not all DDS vendors might support it yet. If you want to experiment with it though, take a look at the community edition of OpenSplice DDS (www.opensplice.org) which does provide support for this.
Hope that answers your questions.
Regards, Erik Hendriks.
posted on Monday, September 13, 2010 - 05:10 pm
Thank You Erik,
I'm using RTI DDS, but i did see the QueryCondition there too. I was wondering at this point what is the difference between Read and Query?
They both seem to attach to a DataReader and access it's cache. Is the difference just how the request is formulated?
What I've heard from some RTI users that have moved to OpenSplice DDS is that RTI DDS does not support the QueryCondition (not sure if they've added this in their latest release). Thus although there is the method in the API the behaviour might not be implemented as required by the spec.
The difference between a ReadCondition and a QueryCondition is that the latter allows you to specify a SQL WHERE expression to filter the data currently available on the cash.
You should use the QueryCondition in combination with a read_w_condition method.
posted on Wednesday, September 15, 2010 - 07:25 pm
I need more dynamic queries. Right now the conditions for my SQL queries change based on table data. The DDS query is a hard coded where clause. I don't want to have to call update query every second when i receive new information.
Is there any DDS condition i can create based on the value of local variables, that way the condition changes on it's own.