Perform Continuous Query for C++ Clients

Skip to end of metadata
Go to start of metadata

While Coherence provides the ability to obtain a point in time query result from a Coherence cache and the ability to receive events that would change the result of that query, it also provides a feature that combines a query result with a continuous stream of related events in order to maintain an up-to-date query result in a real-time fashion. This capability is called Continuous Query because it has the same effect as if the desired query had zero latency and the query were being executed several times every millisecond!

A continuous query cache is similar to a materialized view in the Oracle database. A materialized view copies data queried from the database tables into the view. If there are any changes to the data in the database, then the data in the view is automatically updated. This allows you to see changes to the result set. In continuous query, a local copy of the cache is created on the client. Filters allow you to limit the size and content of the cache. Combined with an event listener, the cache can be updated in real time.

For example, assume that you want to monitor, in real time, all sales orders for a number of customers. To do this, you can create a continuous query cache and set up an event listener that will listen for any events pertaining to the customers. Coherence will query for all of the data objects on the grid that pertain to a particular customer and copy them to a local cache. The event listener on the query will listen for any inserts, updates, or deletes that take place on the grid for the customer. When an event occurs, the local copy of the customer data is updated.

This section contains the following information:

Uses of Continuous Query Caching

There are several different general use categories for Continuous Query Caching:

  • It is an ideal building block for Complex Event Processing (CEP) systems and event correlation engines.
  • It is ideal for situations in which an application repeats a particular query and would benefit from always having instant access to the up-to-date result of that query.
  • A Continuous Query Cache is analogous to a materialized view and is useful for accessing and manipulating the results of a query using the standard NamedCache API, as well as receiving an ongoing stream of events related to that query.
  • A Continuous Query Cache can be used in a manner similar to a Near Cache because it maintains an up-to-date set of data locally where it is being used, for example, on a particular server node or on a client. Note that while a Near Cache is invalidation-based, a Continuous Query Cache actually maintains its data in an up-to-date manner.

By combining the Coherence*Extend functionality with Continuous Query Caching, an application can support literally tens of thousands of concurrent users.

Continuous Query Caches are useful in almost every type of application, including both client-based and server-based applications, because they provide the ability to very easily and efficiently maintain an up-to-date local copy of a specified sub-set of a much larger and potentially distributed cached data set.

The Coherence Continuous Query Cache

The Coherence implementation of Continuous Query is found in the ${xhtml} class. This class, like all Coherence caches, implements the standard ${xhtml} interface, which includes the following capabilities:

  • Cache access and manipulation using the ${xhtml} interface: NamedCache extends the ${xhtml} interface, which is based on the Map interface from the Java Collections Framework.
  • Events for all object modifications that occur within the cache: NamedCache extends the ${xhtml} interface.
  • Identity-based cluster-wide locking of objects in the cache: NamedCache extends the ${xhtml} interface.
  • Querying the objects in the cache: NamedCache extends the ${xhtml} interface.
  • Distributed Parallel Processing and Aggregation of objects in the cache: NamedCache extends the ${xhtml} interface.

Since the ContinuousQueryCache implements the NamedCache interface, which is the same API provided by all Coherence caches, it is extremely simple to use, and it can be easily substituted for another cache when its functionality is called for.

Defining a Continuous Query Cache

There are two features that define a Continuous Query Cache:

  • The underlying cache that the Continuous Query is based on.
  • A query of the underlying cache that produces the sub-set that the Continuous Query Cache will cache.

The underlying cache can be any Coherence cache, including another Continuous Query Cache. The most straight-forward way of obtaining a cache is by using the ${xhtml} class. This class allows you to create a cache simply by specifing its name. It will be created automatically and its configuration will be based on the application's cache configuration information. For example, the following line of code creates a cache named orders:

The query is the same type of query that would be used to query any other cache. For example, the following code filters for a given trader with a given order status:

Normally, to query a cache, you could use one of the methods from the QueryMap class. For example, to obtain a snap-shot of all open trades for this trader:

In contrast, the Continuous Query Cache is constructed from the ContinuousQueryCache::create method, passing the cache and the filter:

Cleaning up the resources associated with a ContinuousQueryCache
A ContinuousQueryCache places one or more event listeners on its underlying cache. If the ContinuousQueryCache is used for the duration of the application, then the resources will be cleaned up when the node is shut down or otherwise stops. However, if the ContinuousQueryCache is only used for a period of time, then the application must call the release() method on the ContinuousQueryCache when it is done using it.

Caching only keys, or caching both keys and values

When constructing a Continuous Query Cache, you can specify that the cache should only keep track of the keys that result from the query and obtain the values from the underlying cache only when they are asked for. This feature may be useful for creating a Continuous Query Cache that represents a very large query result set or if the values are never or rarely requested. To specify that only the keys should be cached, pass false when creating the ContinuousQueryCache; for example:

Using ReflectionExtractor with Continuous Query Caches
When the ContinuousQueryCache is configured to cache values, the use of the ${xhtml} is not supported. This is because the ReflectionExtractor does not support reflection in C++. In this case, you must provide a custom extractor. When the ContinuousQueryCache is not caching values locally, the ReflectionExtractor can be used since it does not perform the extraction on the client but instead passes the necessary extraction information to the cluster to perform the query.

If necessary, the CacheValues property can be modified after the cache has been instantiated; for example:

CacheValues property and Event Listeners
If the Continuous Query Cache has any standard (non-lite) event listeners, or if any of the event listeners are filtered, then the CacheValues property will automatically be set to true. This is because the Continuous Query Cache uses the locally cached values to filter events and to supply the old and new values for the events that it raises.

Listening to the Continuous Query Cache

Since the Continuous Query Cache is itself observable, it is possible for the client to place one or more event listeners onto it. For example:

If your application has to perform some processing against every item that is already in the cache and every item added to the cache, then provide the listener during construction. The resulting cache will receive one event for each item that is in the Continuous Query Cache, whether it was there to begin with (because it was in the query) or if it got added during or after the construction of the cache. One form of the factory create method of ContinuousQueryCache allows you to specify a cache, a filter, and a listener:

Avoiding unexpected results
There are two alternate approaches to processing the items in the Continuous Query Cache, both of which could yield unexpected and unwanted results. First, if you perform the processing and then add the listener to handle any later additions, then events that occur in the split second after the iteration and before the listener is added will be missed! This is illustrated in the following code sample:

The second approach is to add a listener first, so that no events are missed, and then do the processing. In this case, it is possible that the same entry will show up in both an event and in the Iterator. The events can be asynchronous, so the sequence of operations cannot be guaranteed.

Achieving a stable materialized view
The Continuous Query Cache implementation faced the same challenge: How to assemble an exact point-in-time snapshot of an underlying cache while receiving a stream of modification events from that same cache. The solution has several parts. First, Coherence supports an option for synchronous events, which provides a set of ordering guarantees. Secondly, the Continuous Query Cache has a two-phase implementation of its initial population that allows it to first query the underlying cache and then subsequently resolve all of the events that came in during the first phase. Since achieving these guarantees of data visibility without any missing or repeated events is fairly complex, the ContinuousQueryCache allows a developer to pass a listener during construction, thus avoiding exposing these same complexities to the application developer.
Support for synchronous and asynchronous listeners
By default, listeners to the Continuous Query Cache will have their events delivered asynchronously. However, the ContinuousQueryCache implementation does respect the option for synchronous events as provided by the ${xhtml} interface.

Making the Continuous Query Cache Read-Only

The Continuous Query Cache can be made into a read-only cache by using the boolean setReadOnly method on the ContinuousQueryCache class; for example:

A read-only Continuous Query Cache will not allow objects to be added to, changed in, removed from or locked in the cache.

Once a Continuous Query Cache has been set to read-only, it cannot be changed back to read/write.

Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.