Overview: What Coherence can do for you...
Overview for Implementors
1. Basic Concepts
Audience
This document is targeted at software developers and architects who need an overview of how Coherence can be used. This document outlines product capabilities, usage possibilities, and provides an overview of how one would go about implementing particular features. This document bridges the more abstract Coherence white-papers and the more concrete developer documentation (Tangosol Coherence User Guide and the Coherence JavaDoc).
Clustered Data Management
At the core of Coherence is the concept of clustered data management. This implies the following goals:
- A fully coherent, single system image (SSI)
- Scalability for both read and write access
- Fast, transparent failover and failback
- Linear scalability for storage and processing
- No Single-Points-of-Failure (SPOFs)
- Cluster-wide locking and transactions
Built on top of this foundation are the various services that Coherence provides, including database caching, HTTP session management, grid agent invocation and distributed queries. Before going into detail about these features, some basic aspects of Coherence should be discussed.
A single API for the logical layer, XML configuration for the physical layer
Coherence supports many topologies for clustered data management. Each of these topologies has trade-offs in terms of performance and fault-tolerance. By using a single API, the choice of topology can be deferred until deployment if desired. This allows developers to work with a consistent logical view of Coherence, while providing flexibility during tuning or as application needs change.
Clients are cluster members
A key benefit to Coherence's clustering model is that both server nodes and client nodes are part of the cluster. The benefit of this is that data can be very closely managed by the client without polling or asynchronous events. Locally managed data can be accessed directly without the need to synchronize with other servers. And it also centralizes cluster management, as both server and client membership can be managed. Cluster membership is a very inexpensive resource, so thousands of clients are readily supported. And furthermore, in a typical enterprise application, it is the application tier machines that are client nodes – end users do not directly access Coherence.
Caching Strategies
Coherence provides several cache implementations:
- Local - Local on-heap caching for non-clustered caching.
- Replicated - Perfect for small, read-heavy caches.
- Optimistic - A version of the Replicated cache which gives up locking (but not coherent behavior) for a boost in performance.
- Partitioned - True linear scalability for both read and write access. Data is automatically, dynamically and transparently partitioned across nodes. The distribution algorithm minimizes network traffic and avoids service pauses by incrementally shifting data.
- Near Cache - Provides the performance of local caching with the scalability of distributed caching. Several different near-cache strategies provide varying tradeoffs between performance and synchronization guarantees.
In-process caching provides the highest level of raw performance, since objects are managed within the local JVM. This benefit is most directly realized by the Local, Replicated, Optimistic and Near Cache implementations.
Out-of-process (client-server) caching provides the option of using dedicated cache servers. This can be helpful when you wish to partition workloads (to avoid stressing the application servers). This is accomplished by using the Partitioned cache implementation and simply disabling local storage on client nodes via a single command-line option or a one-line entry in the XML configuration.
Tiered caching (using the NearCache functionality) allows you to couple local caches on the application server with larger, partitioned caches on the cache servers, combining the raw performance of local caching with the scalability of partitioned caching. This is useful for both dedicated cache servers as well as co-located caching (cache partitions stored within the application server JVMs).
Coherence supports heterogeneous configurations – for example, using inexpensive Linux blades as dedicated cache servers, and UNIX machines for the application server instances.
Data Storage Options
While most customers use on-heap storage combined with dedicated cache servers, Coherence has several options for data storage:
- On-heap - The fastest option, though it can affect JVM garbage collection times.
- NIO RAM - No impact on garbage collection, though it does require serialization/deserialization.
- NIO Disk - Similar to NIO RAM, but using memory-mapped files.
- File-based - Uses a special disk-optimized storage system to optimize speed and minimize I/O.
It should be noted that Coherence storage is transient – the disk-based storage options are for managing cached data only. If long-term persistence of data is required, Coherence provides snapshot functionality to persist an image to disk. This is especially useful when the external data sources used to build the cache are extremely expensive. By using the snapshot, the cache can be rebuilt from an image file (rather than reloading from a very slow external datasource).
Serialization Options
Because serialization is often the most expensive part of clustered data management, Coherence provides three options for serializing/deserializing data:
- java.io.Serializable - The simplest, but slowest option.
- java.io.Externalizable - This requires developers to implement serialization manually, but can provide significant performance benefits. Compared to java.io.Serializable, this can cut serialized data size by a factor of two or more (especially helpful with Distributed caches, as they generally cache data in serialized form). Most importantly, CPU usage is dramatically reduced.
- com.tangosol.io.ExternalizableLite - This is very similar to java.io.Externalizable, but offers better performance and less memory usage by using a more efficient I/O stream implementation. The com.tangosol.run.xml.XmlBean class provides a default implementation of this interface (see the section on XmlBean for more details).
Configurability and Extensibility
Coherence's API provides access to all Coherence functionality. The most commonly used subset of this API is exposed via simple XML options to minimize effort for typical use cases. There is no penalty for mixing direct configuration via the API with the easier XML configuration.
Coherence is designed to allow the replacement of its modules as needed. For example, the local "backing maps" (which provide the actual physical data storage on each node) can be easily replaced as needed. The vast majority of the time, this is not required, but it is there for the situations that require it. The general guideline is that 80% of tasks are easy, and the remaining 20% of tasks (the special cases) require a little more effort, but certainly can be done without significant hardship.
Namespace Hierarchy
Coherence is organized as set of services. At the root is the "Cluster" service. A cluster is defined as a set of Coherence instances (one instance per JVM, with one or more JVMs on each physical machine). A cluster is defined by the combination of multicast address and port. A TTL (network packet time-to-live; i.e., the number of network hops) setting can be used to restrict the cluster to a single machine, or the machines attached to a single switch.
Under the cluster service are the various services that comprise the Coherence API. These include the various caching services (Replicated, Distributed, etc.) as well as the Invocation Service (for deploying agents to various nodes of the cluster). Each instance of a service is named, and there is typically a default service instance for each type.
The cache services contain NamedCaches (com.tangosol.net.NamedCache), which are analogous to database tables – that is, they typically contain a set of related objects.
2. Read/Write Caching
NamedCache
The following source code will return a reference to a NamedCache instance. The underlying cache service will be started if necessary.
import com.tangosol.net.*;
...
NamedCache cache = CacheFactory.getCache("MyCache");
Coherence will scan the cache configuration XML file for a name mapping for MyCache. NamedCache name mapping is similar to Servlet name mapping in a web container's web.xml file. Coherence's cache configuration file contains (in the simplest case) a set of mappings (from cache name to cache strategy) and a set of cache strategies.
By default, Coherence will use the coherence-cache-config.xml file found at the root of coherence.jar. This can be overridden on the JVM command-line with -Dtangosol.coherence.cacheconfig=file.xml. This argument can reference either a file system path, or a Java resource path.
The com.tangosol.net.NamedCache interface extends a number of other interfaces:
- java.util.Map - basic Map methods such as get(), put(), remove().
- com.tangosol.util.QueryMap - methods for querying the cache.
- com.tangosol.util.ConcurrentMap - methods for concurrent access such as lock() and unlock().
- com.tangosol.util.ObservableMap - methods for listening to cache events.
- com.tangosol.util.InvocableMap - methods for server-side processing of cache data.
 | Requirements for cached objects
Cache keys and values must be serializable (e.g. java.io.Serializable). Furthermore, cache keys must provide an implementation of the hashCode() and equals() methods, and those methods must return consistent results across cluster nodes. This implies that the implementation of hashCode() and equals() must be based solely on the object's serializable state (i.e. the object's non-transient fields); most built-in Java types, such as String, Integer and Date, meet this requirement. Some cache implementations (specifically the partitioned cache) use the serialized form of the key objects for equality testing, which means that keys for which equals() returns true must serialize identically; most built-in Java types meet this requirement as well. |
NamedCache Usage Patterns
There are two general approaches to using a NamedCache:
- As a clustered implementation of java.util.Map with a number of added features (queries, concurrency), but with no persistent backing (a "side" cache).
- As a means of decoupling access to external data sources (an "inline" cache). In this case, the application uses the NamedCache interface, and the NamedCache takes care of managing the underlying database (or other resource).
Typically, an inline cache is used to cache data from:
- a database - The most intuitive use of a cache – simply caching database tables (in the form of Java objects).
- a service - Mainframe, web service, service bureau – any service that represents an expensive resource to access (either due to computational cost or actual access fees).
- calculations - Financial calculations, aggregations, data transformations. Using an inline cache makes it very easy to avoid duplicating calculations. If the calculation is already complete, the result is simply pulled from the cache. Since any serializable object can be used as a cache key, it's a simple matter to use an object containing calculation parameters as the cache key.
Write-back options:
- write-through - Ensures that the external data source always contains up-to-date information. Used when data must be persisted immediately, or when sharing a data source with other applications.
- write-behind - Provides better performance by caching writes to the external data source. Not only can writes be buffered to even out the load on the data source, but multiple writes can be combined, further reducing I/O. The trade-off is that data is not immediately persisted to disk; however, it is immediately distributed across the cluster, so the data will survive the loss of a server. Furthermore, if the entire data set is cached, this option means that the application can survive a complete failure of the data source temporarily as both cache reads and writes do not require synchronous access the the data source.
To implement a read-only inline cache, you simply implement two methods on the com.tangosol.net.cache.CacheLoader interface, one for singleton reads, the other for bulk reads. Coherence provides an abstract class com.tangosol.net.cache.AbstractCacheLoader which provides a default implementation of the bulk method, which means that you need only implement a single method:public Object load(Object oKey);
This method accepts an arbitrary cache key and returns the appropriate value object.
If you want to implement read-write caching, you need to extend com.tangosol.net.cache.AbstractCacheStore (or implement the interface com.tangosol.net.cache.CacheStore), which adds the following methods:
public void erase(Object oKey);
public void store(Object oKey, Object oValue);
The method erase() should remove the specified key from the external data source. The method store() should update the specified item in the data source if it already exists, or insert it if it does not presently exist.
Once the CacheLoader/CacheStore is implemented, it can be connected easily via the coherence-cache-config.xml file.
3. Querying the Cache
Coherence provides the ability to execute search queries against cached data. With distributed caches, the queries are indexed and parallelized. This means that adding servers to a distributed cache not only increases throughput (total queries per second) but also reduces latency, with queries taking less user time. To query against a NamedCache, all objects should implement a common interface (or base class). Any field of an object can be queried; indexes are optional, and used to increase performance. With a replicated cache, queries are performed locally, and do not use indexes. This approach works well due to the nature of replicated caching (as opposed to distributed caching).
To add an index to a NamedCache, you first need a value extractor (which accepts as input a value object and returns an attribute of that object). Indexes can be added blindly (duplicate indexes are ignored). Indexes can be added at any time, before or after inserting data into the cache.
It should be noted that queries apply only to cached data. For this reason, queries should not be used unless the entire data set has been loaded into the cache, unless additional support is added to manage partially loaded sets.
Developers have the option of implementing additional custom filters for queries, thus taking advantage of query parallelization. For particularly performance-sensitive queries, developers may implement index-aware filters, which can access Coherence's internal indexing structures.
Coherence includes a built-in optimizer, and will apply indexes in the optimal order. Because of the focused nature of the queries, the optimizer is both effective and efficient. No maintenance is required.
Example code to create an index:
NamedCache cache = CacheFactory.getCache("MyCache");
ValueExtractor extractor = new ReflectionExtractor("getAttribute");
cache.addIndex(extractor, true, null);
Example code to query a NamedCache (returns the keys corresponding to all of the value objects with an "Attribute" greater than 5):
NamedCache cache = CacheFactory.getCache("MyCache");
Filter filter = new GreaterFilter("getAttribute", 5);
Set keySet = cache.keySet(filter);
4. Transactions
Coherence supports local transactions against the cache through both a direct API, as well as through J2CA adapters for J2EE containers. Transactions support either pessimistic or optimistic concurrency strategies, as well as the Read Committed, Repeatable Read, Serializable isolation levels.
5. HTTP Session Management
Coherence*Web is an HTTP session-management module (shipped with Coherence) with support for a wide range of application servers. Using Coherence session management does not require any changes to the application.
Coherence*Web uses the NearCache technology to provide fully fault-tolerant caching, with almost unlimited scalability (to several hundred cluster nodes without issue).
Heterogeneous applications running on mixed hardware/OS/application servers can share common user session data. This dramatically simplifies supporting Single-Sign-On across applications.
6. Invocation Service
The Coherence Invocation service can be used to deploy computational agents to various nodes within the cluster. These agents can be either execute-style (deploy and asynchronously listen) or query-style (deploy and wait for results).
The Invocation service is accessed through the interface com.tangosol.net.InvocationService through the two following methods:
public void execute(Invocable task, Set setMembers, InvocationObserver observer);
public Map query(Invocable task, Set setMembers);
An instance of the service can be retrieved from the com.tangosol.net.CacheFactory class.
7. Listeners
All NamedCache instances in Coherence implement the com.tangosol.util.ObservableMap interface, which allows the option of attaching a cache listener implementation (of com.tangosol.util.MapListener). It should be noted that applications can observe events as logical concepts regardless of which physical machine caused the event. Customizable server-side filters and lightweight events can be used to minimize network traffic and processing. Cache listeners follow the JavaBean paradigm, and can distinguish between system cache events (e.g., eviction) and application cache events (e.g., get/put operations).
Similarly, any service can be watched for members joining and leaving, including the cluster service as well as the cache and invocation services.
8. JDO/JDBC Integration
The following products support Coherence as a clustered caching plug-in:
- BEA SolarMetric Kodo (JDO)
- Hemtech JDO Genie (JDO)
- Riflexo JCredo (JDO)
- Hibernate (O/R Mapping)
- Isocra Livestore (transparent JDBC caching)
9. C++/.Net Integration
Coherence can be accessed by C++ applications (via CodeMesh JunC++ion). and .Net applications (via JNBridge). The integration takes the form of local proxy objects that control remote Coherence objects within the JVM.
10. WAN Support
Coherence's TCMP clustering protocol is specifically designed to handle the unreliable, high-latency, low-bandwidth conditions typically found in WAN links. Distributed locking provides better performance by avoiding single-server bottlenecks. Tiered caching minimizes network traffic. Transactions and deterministic split-brain behavior ensure proper application function. Coherence supports wire compression for WAN environments.
Coherence 3.0 has expanded support for WAN environments, including WKA (well-known-address) support and improvements to the TCMP protocol specifically for geographically-distributed applications.
11. XmlBean
Coherence includes a helper class for managing XML-to-Object mapping. XmlBean (com.tangosol.run.xml.XmlBean) supports most common Java types including primitives (both intrinsic and object forms) as well as collections. An added benefit is that XmlBean provides an implementation of com.tangosol.io.ExternalizableLite for very fast serialization/deserialization (as well as reducing both memory and network usage). XmlBean supports does not support cyclical object graphs (which generally do not appear in value objects anyway).
12. Manageability
Coherence offers almost transparent manageability. Coherence automatically maintains cluster membership and handles damaged nodes transparently. New machines can be added dynamically to increase cluster capacity.
Coherence is specifically designed to require minimal maintenance, as this is an implied part of the Reliability and Availability goals. Evidence of this can be found in the number of OEMs who embed Coherence within their products (including some vendors of high-volume packaged software).
Coherence offers extensive JMX instrumentation. For more details, please see
com.tangosol.net.management.Registry.
Cluster your objects and your data
Overview
Coherence is an essential ingredient for building reliable, high-scale clustered applications. The term clustering refers to the use of more than one server to run an application, usually for reliability and scalability purposes. Coherence provides all of the necessary capabilities for applications to achieve the maximum possible availability, reliability, scalability and performance. Virtually any clustered application will benefit from using Coherence.
One of primary uses of Coherence is to cluster an application's objects and data. In the simplest sense, this means that all of the objects and data that an application delegates to Coherence is automatically available to and accessible by all servers in the application cluster, and none of those objects and none of that data will be lost in the event of server failure.
By clustering the application's objects and data, Coherence solves many of the difficult problems related to achieving availability, reliability, scalability, performance, serviceability and manageability of clustered applications.
Availability
Availability refers to the percentage of time that an application is operating. High Availability refers to achieving availability close to 100%. Coherence is used to achieve High Availability in several different ways:
Supporting redundancy in Java applications
Coherence makes it possible for an application to run on more than one server, which means that the servers are redundant. Using a load balancer, for example, an application running on redundant servers will be available as long as one server is still operating. Coherence enables redundancy by allowing an application to share, coordinate access to, update and receive modification events for critical runtime information across all of the redundant servers. Most applications cannot operate in a redundant server environment unless they are architected to run in such an environment; Coherence is a key enabler of such an architecture.
Enabling dynamic cluster membership
Coherence tracks exactly what servers are available at any given moment. When the application is started on an additional server, Coherence is instantly aware of that server coming online, and automatically joins it into the cluster. This allows redundancy (and thus availability) to be dynamically increased by adding servers.
Exposing knowledge of server failure
Coherence reliably detects most types of server failure in less than a second, and immediately fails over all of the responsibilities of the failed server without losing any data. As a result, server failure does not impact availability.
Part of an availability management is Mean Time To Recovery (MTTR), which is a measurement of how much time it takes for an unavailable application to become available. Since server failure is detected and handled in less than a second, and since redundancy means that the application is available even when that server goes down, the MTTR due to server failure is zero from the point of view of application availability, and typically sub-second from the point of view of a load-balancer re-routing an incoming request.
Eliminating other Single Points Of Failure (SPOFs)
Coherence provides insulation against failures in other infrastructure tiers. For example, Coherence write-behind caching and Coherence distributed parallel queries can insulate an application from a database failure; in fact, using these capabilities, two different Coherence customers have had database failure during operational hours, yet their production Coherence-based applications maintained their availability and their operational status.
Providing support for Disaster Recovery (DR) and Continuancy Planning
Coherence can even insulate against failure of an entire data center, by clustering across multiple data centers and failing over the responsibilities of an entire data center. Again, this capability has been proven in production, with a Coherence customer running a mission-critical real-time financial system surviving a complete data center outage.
Reliability
Reliability refers to the percentage of time that an application is able to process correctly. In other words, an application may be available, yet unreliable if it cannot correctly handle the application processing. An example that we use to illustrate high availability but low reliability is a mobile phone network: While most mobile phone networks have very high uptimes (referring to availability), dropped calls tend to be relatively common (referring to reliability).
Coherence is explicitly architected to achieve very high levels of reliability. For example, server failure does not impact "in flight" operations, since each operation is atomically protected from server failure, and will internally re-route to a secondary node based on a dynamic pre-planned recovery strategy. In other words, every operation has a backup plan ready to go!
Coherence is architected based on the assumption that failures are always about to occur. As a result, the algorithms employed by Coherence are carefully designed to assume that each step within a operation could fail due to a network, server, operating system, JVM or other resource outage. An example of how Coherence plans for these failures is the synchronous manner in which it maintains redundant copies of data; in other words, Coherence does not gamble with the application's data, and that ensures that the application will continue to work correctly, even during periods of server failure.
Scalability
Scalability refers to the ability of an application to predictably handle more load. An application exhibits linear scalability if the maximum amount of load that an application can sustain is directly proportional to the hardware resources that the application is running on. For example, if an application running on 2 servers can handle 2000 requests per second, then linear scalability would imply that 10 servers would handle 10000 requests per second.
Linear scalability is the goal of a scalable architecture, but it is difficult to achieve. The measurement of how well an application scales is called the scaling factor (SF). A scaling factor of 1.0 represents linear scalability, while a scaling factor of 0.0 represents no scalability. Coherence provides a number of capabilities designed to help applications achieve linear scalability.
When planning for extreme scale, the first thing to understand is that application scalability is limited any necessary shared resource that does not exhibit linear scalability. The limiting element is referred to as a bottleneck, and in most applications, the bottleneck is the data source, such as a database or an EIS.
Coherence helps to solve the scalability problem by targeting obvious bottlenecks, and by completely eliminating bottlenecks whenever possible. It accomplishes this through a variety of capabilities, including:
Distributed Caching
Coherence uses a combination of replication, distribution, partitioning and invalidation to reliably maintain data in a cluster in such a way that regardless of which server is processing, the data that it obtains from Coherence is the same. In other words, Coherence provides a distributed shared memory implementation, also referred to as Single System Image (SSI) and Coherent Clustered Caching.
Any time that an application can obtain the data it needs from the application tier, it is eliminating the data source as the Single Point Of Bottleneck (SPOB).
Partitioning
Partitioning refers to the ability for Coherence to load-balance data storage, access and management across all of the servers in the cluster. For example, when using Coherence data partitioning, if there are four servers in a cluster then each will manage 25% of the data, and if another server is added, each server will dynamically adjust so that each of the five servers will manage 20% of the data, and this data load balancing will occur without any application interruption and without any lost data or operations. Similarly, if one of those five servers were to die, each of the remaining four servers would be managing 25% of the data, and this data load balancing will occur without any application interruption and without any lost data or operations – including the 20% of the data that was being managed on the failed server.
Coherence accomplishes failover without data loss by synchronously maintaining a configurable number of copies of the data within the cluster. Just as the data management responsibility is spread out over the cluster, so is the responsibility for backing up data, so in the previous example, each of the remaining four servers would have roughly 25% of the failed server's data backed up on it. This mesh architecture guarantees that on server failure, no particular remaining server is inundated with a massive amount of additional responsibility.
Coherence prevents loss of data even when multiple instances of the application are running on a single physical server within the cluster. It does so by ensuring that backup copies of data are being managed on different physical servers, so that if a physical server fails or is disconnected, all of the the data being managed by the failed server has backups ready to go on a different server.
Lastly, partitioning supports linear scalability of both data capacity and throughput. It accomplishes the scalability of data capacity by evenly balancing the data across all servers, so four servers can naturally manage two times as much data as two servers. Scalability of throughput is also a direct result of load-balancing the data across all servers, since as servers are added, each server is able to utilize its full processing power to manage a smaller and smaller percentage of the overall data set. For example, in a ten-server cluster each server has to manage 10% of the data operations, and – since Coherence uses a peer-to-peer architecture – 10% of those operations are coming from each server. With ten times that many servers (i.e. 100 servers), each server is managing only 1% of the data operations, and only 1% of those operations are coming from each server – but there are ten times as many servers, so the cluster is accomplishing ten times the total number of operations! In the 10-server example, if each of the ten servers was issuing 100 operations per second, they would each be sending 10 of those operations to each of the other servers, and the result would be that each server was receiving 100 operations (10x10) that it was responsible for processing. In the 100-server example, each would still be issuing 100 operations per second, but each would be sending only one operation to each of the other servers, so the result would be that each server was receiving 100 operations (100x1) that it was responsible for processing. This linear scalability is made possible by modern switched network architectures that provide backplanes that scale linearly to the number of ports on the switch, providing each port with dedicated fully-duplexed (upstream and downstream) bandwidth. Since each server is only sending and receiving 100 operations (in both the 10-server and 100-server examples), the network bandwidth utilization is roughly constant per port regardless of the number of servers in the cluster.
Session Management
One common use case for Coherence clustering is to manage user sessions (conversational state) in the cluster. This capability is provided by the Coherence*Web module, which is a built-in feature of Coherence. Coherence*Web provides linear scalability for HTTP Session Management in clusters of hundreds of production servers. It can achieve this linear scalability because at its core it is built on Coherence dynamic partitioning.
Session management highlights the scalability problem that typifies shared data sources: If an application could not share data across the servers, it would have to delegate that data management entirely to the shared store, which is typically the application's database. If the HTTP session were stored in the database, each HTTP request (in the absence of sticky load-balancing) would require a read from the database, causing the desired reads-per-second from the database to increase linearly with the size of the server cluster. Further, each HTTP request causes an update of its corresponding HTTP session, so regardless of sticky load balancing, to ensure that HTTP session data is not lost when a server fails the desired writes-per-second to the database will also increase linearly with the size of the server cluster. In both cases, the actual reads and writes per second that a database is capable of does not scale in relation to the number of servers requesting those reads and writes, and the database quickly becomes a bottleneck, forcing availability, reliability (e.g. asynchronous writes) and performance compromises. Additionally, related to performance, each read from a database has an associated latency, and that latency increases dramatically as the database experiences increasing load.
Coherence*Web, on the other hand, has the same latency in a 2-server cluster as it has in a 200-server cluster, since all HTTP session read operations that cannot be handled locally (e.g. locality as the result of to sticky load balancing) are spread out evenly across the rest of the cluster, and all update operations (which must be handled remotely to ensure survival of the HTTP sessions) are likewise spread out evently across the rest of the cluster. The result is linear scalability with constant latency, regardless of the size of the cluster.
Performance
Performance is the inverse of latency, and latency is the measurement of how long something takes to complete. If increasing performance is the goal, then getting rid of anything that has any latency is the solution. Obviously, it is impossible to get rid of all latencies, since the High Availability and reliability aspects of an application are counting on the underlying infrastructure, such as Coherence, to maintain reliable up-to-date back-ups of important information, which means that some operations (such as data modifications and pessimistic transactions) have unavoidable latencies. On the other hand, every remaining operation that could possibly have any latency needs to be targeted for elimination, and Coherence provides a large number of capabilities designed to do just that:
Replication
Just like partitioning dynamically load-balances data evenly across the entire server cluster, replication ensures that a desired set of data is up-to-date on every single server in the cluster at all times. Replication allows operations running on any server to obtain the data that they need locally, at basically no cost, because that data has already been replicated to that server. In other words, replication is a tool to guarantee locality of reference, and the end result is zero-latency access to replicated data.
Near Caching
Since replication works best for data that should be on all servers, it follows that replication is inefficient for data that an application would want to avoid copying to all servers. For example, data that changes all of the time and very large data sets are both poorly suited to replication, but both are excellently suited to partitioning, since it exhibits linear scale of data capacity and throughput.
The only downside of partitioning is that it introduces latency for data access, and in most applications the data access rate far out-weighs the data modification rate. To eliminate the latency associated with partitioned data access, near caching maintains frequently- and recently-used data from the partitioned cache on the specific servers that are accessing that data, and it keeps that data coherent by means of event-based invalidation. In other words, near caching keeps the most-likely-to-be-needed data near to where it will be used, thus providing good locality of access, yet backed up by the linear scalability of partitioning.
Write-Behind, Write-Coalescing and Write-Batching
Since the transactional throughput in the cluster is linearly scalable, the cost associated with data changes can be a fixed latency, typically in the range of a few milliseconds, and the total number of transactions per second is limited only by the size of the cluster. In one application, Coherence was able to achieve transaction rates close to a half-million transactions per second – and that on a cluster of commodity two-CPU servers.
Often, the data being managed by Coherence is actually a temporary copy of data that exists in an official System Of Record (SOR), such as a database. To avoid having the database become a transaction bottleneck, and to eliminate the latency of database updates, Coherence provides a Write-Behind capability, which allows the application to change data in the cluster, and those changes are asynchronously replayed to the application's database (or EIS). By managing the changes in a clustered cache (which has all of the High Availability, reliability and scalability attributes described previously,) the pending changes are immune to server failure and the total rate of changes scales linearly with the size of the cluster.
The Write-Behind functionality is implemented by queueing each data change; the queue contains a list of what changes needs to be written to the System Of Record. The duration of an item within the queue is configurable, and is referred to as the Write-Behind Delay. When data changes, it is added to the write-behind queue (if it is not already in the queue), and the queue entry is set to ripen after the configured Write-Behind Delay has passed. When the queue entry has ripened, the latest copy of the corresponding data is written to the System Of Record.
To avoid overwhelming the System Of Record, Coherence will replay only the latest copies of data to the database, thus coalescing many updates that occur to the same piece data into a single database operation. The longer the Write-Behind Delay, the more coalescing may occur. Additionally, if many different pieces of data have changed, all of those updates can be batched (e.g. using JDBC statement batching) into a single database operation. In this way, a massive breadth of changes (number of pieces of data changed) and depth of changes (number of times each was changed) can be bundled into a single database operation, which results in dramatically reduced load on the database. The batching is also fully configurable; one option, called the Write Batch Factor, even allows some of the queue entries that have not yet ripened to be included in the batched update.
Serviceability
Serviceability refers to the ease and extent of changes that can be affected without affecting availability. Coherence helps to increase an application's serviceability by allowing servers to be taken off-line without impacting the application availability. Those servers can be serviced and brought back online without any end-user or processing interruptions. Many configuration changes related to Coherence can also be made on a node-by-node basis in the same manner. With careful planning, even major application changes can be rolled into production – again, one node at a time – without interrupting the application.
Manageability
Manageability refers to the level of information that a running system provides, and the capability to tweak settings related to that information. For example, Coherence provides a cluster-wide view of management information via the standard JMX API, so that the entire cluster can be managed from a single server. The information provided includes hit and miss rates, cace sizes, read-, write- and write-behind statistics, and detailed information all the way down to the network packet level.
Additionally, Coherence allows applications to place their own management information – and expose their own tweakable settings – through the same clustered JMX implementation. The result is an application infrastructure that makes managing and monitoring a clustered application as simple as managing and monitoring a single server, and all through Java's standard management API.
Summary
There are a lot of challenges in building a highly available application that exhibits scalable performance and is both serviceable and manageable. While there are many ways to build distributed applications, only Coherence reliably clusters objects and data. Once objects and data are clustered by Coherence, all the servers in the cluster can access and modify those objects and that data, and the objects and data managed by Coherence will not be effected if and when servers fail. By providing a variety of advanced capabilities, each of which is configurable, and application can achieve the optimal balance of redundancy, scalability and performance, and do so within a manageable and serviceable environment.
Deliver events for changes as they occur
Overview
Coherence provides cache events using the JavaBean Event model. It is extremely simple to receive the events that you need, where you need them, regardless of where the changes are actually occurring in the cluster. Developers with any experience with the JavaBean model will have no difficulties working with events, even in a complex cluster.
Listener interface and Event object
In the JavaBeans Event model, there is an EventListener interface that all listeners must extend. Coherence provides a MapListener interface, which allows application logic to receive events when data in a Coherence cache is added, modified or removed:
public interface MapListener
extends EventListener
{
/**
* Invoked when a map entry has been inserted.
*
* @param evt the MapEvent carrying the insert information
*/
public void entryInserted(MapEvent evt);
/**
* Invoked when a map entry has been updated.
*
* @param evt the MapEvent carrying the update information
*/
public void entryUpdated(MapEvent evt);
/**
* Invoked when a map entry has been removed.
*
* @param evt the MapEvent carrying the delete information
*/
public void entryDeleted(MapEvent evt);
}
An application object that implements the MapListener interface can sign up for events from any Coherence cache or class that implements the ObservableMap interface, simply by passing an instance of the application's MapListener implementation to one of the addMapListener() methods.
The MapEvent object that is passed to the MapListener carries all of the necessary information about the event that has occurred, including the source (ObservableMap) that raised the event, the identity (key) that the event is related to, what the action was against that identity (insert, update or delete), what the old value was and what the new value is:
public class MapEvent
extends EventObject
{
/**
* Return an ObservableMap object on which this event has actually
* occured.
*
* @return an ObservableMap object
*/
public ObservableMap getMap()
/**
* Return this event's id. The event id is one of the ENTRY_*
* enumerated constants.
*
* @return an id
*/
public int getId()
/**
* Return a key assosiated with this event.
*
* @return a key
*/
public Object getKey()
/**
* Return an old value assosiated with this event.
* <p>
* The old value represents a value deleted from or updated in a map.
* It is always null for "insert" notifications.
*
* @return an old value
*/
public Object getOldValue()
/**
* Return a new value assosiated with this event.
* <p>
* The new value represents a new value inserted into or updated in
* a map. It is always null for "delete" notifications.
*
* @return a new value
*/
public Object getNewValue()
/**
* Return a String representation of this MapEvent object.
*
* @return a String representation of this MapEvent object
*/
public String toString()
/**
* This event indicates that an entry has been added to the map.
*/
public static final int ENTRY_INSERTED = 1;
/**
* This event indicates that an entry has been updated in the map.
*/
public static final int ENTRY_UPDATED = 2;
/**
* This event indicates that an entry has been removed from the map.
*/
public static final int ENTRY_DELETED = 3;
}
Caches and classes that support events
All Coherence caches implement ObservableMap; in fact, the NamedCache interface that is implemented by all Coherence caches extends the ObservableMap interface. That means that an application can sign up to receive events from any cache, regardless of whether that cache is local, partitioned, near, replicated, using read-through, write-through, write-behind, overflow, disk storage, etc.
|
Regardless of the cache topology and the number of servers, and even if the modifications are being made by other servers, the events will be delivered to the application's listeners. |
In addition to the Coherence caches (those objects obtained through a Coherence cache factory), several other supporting classes in Coherence also implement the ObservableMap interface:
- ObservableHashMap
- LocalCache
- OverflowMap
- NearCache
- ReadWriteBackingMap
- AbstractSerializationCache, SerializationCache and SerializationPagedCache
- WrapperObservableMap, WrapperConcurrentMap and WrapperNamedCache
For a full list of published implementing classes, see the Coherence JavaDoc for ObservableMap.
Signing up for all events
To sign up for events, simply pass an object that implements the MapListener interface to one of the addMapListener methods on ObservableMap:
public void addMapListener(MapListener listener);
public void addMapListener(MapListener listener, Object oKey, boolean fLite);
public void addMapListener(MapListener listener, Filter filter, boolean fLite);
Let's create an example MapListener implementation:
/**
* A MapListener implementation that prints each event as it receives
* them.
*/
public static class EventPrinter
extends Base
implements MapListener
{
public void entryInserted(MapEvent evt)
{
out(evt);
}
public void entryUpdated(MapEvent evt)
{
out(evt);
}
public void entryDeleted(MapEvent evt)
{
out(evt);
}
}
Using this implementation, it is extremely simple to print out all events from any given cache (since all caches implement the ObservableMap interface):
cache.addMapListener(new EventPrinter());
Of course, to be able to later remove the listener, it is necessary to hold on to a reference to the listener:
Listener listener = new EventPrinter();
cache.addMapListener(listener);
m_listener = listener;
Later, to remove the listener:
Listener listener = m_listener;
if (listener != null)
{
cache.removeMapListener(listener);
m_listener = null; }
Each addMapListener method on the ObservableMap interface has a corresponding removeMapListener method. To remove a listener, use the removeMapListener method that corresponds to the addMapListener method that was used to add the listener.
Using an inner class as a MapListener
When creating an an inner class to use as a MapListener, or when implementing a MapListener that only listens to one or two types of events (inserts, updates or deletes), you can use the AbstractMapListener base class. For example, the following anonymous inner class prints out only the insert events for the cache:
cache.addMapListener(new AbstractMapListener()
{
public void entryInserted(MapEvent evt)
{
out(evt);
}
});
Another helpful base class for creating a MapListener is the MultiplexingMapListener, which routes all events to a single method for handling. For example, the EventPrinter example could be simplified to:
public static class EventPrinter
extends MultiplexingMapListener
{
public void onMapEvent(MapEvent evt)
{
out(evt);
}
}
Since only one method needs to be implemented to capture all events, the MultiplexingMapListener can also be very useful when creating an an inner class to use as a MapListener.
Configuring a MapListener for a Cache
If the listener should always be on a particular cache, then place it into the cache configuration using the listener element and Coherence will automatically add the listener when it configures the cache.
Signing up for Events on specific identities
Signing up for events that occur against specific identities (keys) is just as simple. For example, to print all events that occur against the Integer key "5":
cache.addMapListener(new EventPrinter(), new Integer(5), false);
So the following code would only trigger an event when the Integer key "5" is inserted or updated:
for (int i = 0; i < 10; ++i)
{
Integer key = new Integer(i);
String value = "test value for key " + i;
cache.put(key, value);
}
Filtering Events
Similar to listening to a particular key, it is possible to listen to particular events. Consider the following example:
public class DeletedFilter
implements Filter, Serializable
{
public boolean evaluate(Object o)
{
MapEvent evt = (MapEvent) o;
return evt.getId() == MapEvent.ENTRY_DELETED;
}
}
cache.addMapListener(new EventPrinter(), new DeletedFilter(), false);
 | Filtering events versus filtering cached data
When building a Filter for querying, the object that will be passed to the evaluate method of the Filter will be a value from the cache, or – if the Filter implements the EntryFilter interface – the entire Map.Entry from the cache. When building a Filter for filtering events for a MapListener, the object that will be passed to the evaluate method of the Filter will always be of type MapEvent.
For more information on how to use a query filter to listen to cache events, see the section below titled Advanced: Listening to Queries. |
The listener is added to the cache with a filter that allows the listener to only receive delete events. For example, if the following sequence of calls were made:
cache.put("hello", "world");
cache.put("hello", "again");
cache.remove("hello");
The result would be:
For more information, see the Advanced: Listening to Queries section below.
"Lite" Events
By default, Coherence provides both the old and the new value as part of an event. Consider the following example:
MapListener listener = new MultiplexingMapListener()
{
public void onMapEvent(MapEvent evt)
{
out("event has occurred: " + evt);
out("(the wire-size of the event would have been "
+ ExternalizableHelper.toBinary(evt).length()
+ " bytes.)");
}
};
cache.addMapListener(listener);
cache.put("test", new byte[1024]);
cache.put("test", new byte[2048]);
cache.remove("test");
The output from running the test shows that the first event carries the 1KB inserted value, the second event carries both the replaced 1KB value and the new 2KB value, and the third event carries the removed 2KB value:
When an application does not require the old and the new value to be included in the event, it can indicate that by requesting only "lite" events. When adding a listener, you can request lite events by using one of the two addMapListener methods that takes an additional boolean fLite parameter. In the above example, the only change would be:
cache.addMapListener(listener, (Filter) null, true);
|
Obviously, a lite event's old value and new value may be null. However, even if you request lite events, the old and the new value may be included if there is no additional cost to generate and deliver the event. In other words, requesting that a MapListener receive lite events is simply a hint to the system that the MapListener does not need to know the old and new values for the event. |
Advanced: Listening to Queries
All Coherence caches support querying by any criteria. When an application queries for data from a cache, the result is a point-in-time snapshot, either as a set of identities ("keySet") or a set of identity/value pairs ("entrySet"). The mechanism for determining the contents of the resulting set is referred to as filtering, and it allows an application developer to construct queries of arbitrary complexity using a rich set of out-of-the-box filters (e.g. equals, less-than, like, between, etc.), or to provide their own custom filters (e.g. XPath).
The same filters that are used to query a cache can be used to listen to events from a cache. For example, in a trading system it is possible to query for all open "Order" objects for a particular trader:
NamedCache mapTrades = ...
Filter filter = new AndFilter(new EqualsFilter("getTrader", traderid),
new EqualsFilter("getStatus", Status.OPEN));
Set setOpenTrades = mapTrades.entrySet(filter);
To receive notifications of new trades being opened for that trader, closed by that trader or reassigned to or from another trader, the application can use the same filter:
trades.addMapListener(listener, new MapEventFilter(filter), true);
The MapEventFilter converts a query filter into an event filter.
| Filtering events versus filtering cached data
When building a Filter for querying, the object that will be passed to the evaluate method of the Filter will be a value from the cache, or – if the Filter implements the EntryFilter interface – the entire Map.Entry from the cache. When building a Filter for filtering events for a MapListener, the object that will be passed to the evaluate method of the Filter will always be of type MapEvent.
The MapEventFilter converts a Filter that is used to do a query into a Filter that is used to filter events for a MapListener. In other words, the MapEventFilter is constructed from a Filter that queries a cache, and the resulting MapEventFilter is a filter that evaluates MapEvent objects by converting them into the objects that a query Filter would expect. |
The MapEventFilter has a number of very powerful options, allowing an application listener to receive only the events that it is specifically interested in. More importantly for scalability and performance, only the desired events have to be communicated over the network, and they are communicated only to the servers and clients that have expressed interest in those specific events! For example:
trades.addMapListener(listener, new MapEventFilter(filter,
MapEventFilter.E_ALL), true);
trades.addMapListener(listener, new MapEventFilter(filter,
MapEventFilter.E_UPDATED_LEFT | MapEventFilter.E_DELETED), true);
trades.addMapListener(listener, new MapEventFilter(filter,
MapEventFilter.E_INSERTED | MapEventFilter.E_UPDATED_ENTERED), true);
trades.addMapListener(listener, new MapEventFilter(filter,
MapEventFilter.E_INSERTED), true);
For more information on the various options supported, see the API documentation for MapEventFilter.
Advanced: Synthetic Events
Events usually reflect the changes being made to a cache. For example, one server is modifying one entry in a cache while another server is adding several items to a cache while a third server is removing an item from the same cache, all while fifty threads on each and every server in the cluster is accessing data from the same cache! All the modifying actions will produce events that any server within the cluster can choose to receive. We refer to these actions as client actions, and the events as being dispatched to clients, even though the "clients" in this case are actually servers. This is a natural concept in a true peer-to-peer architecture, such as a Coherence cluster: Each and every peer is both a client and a server, both consuming services from its peers and providing services to its peers. In a typical Java Enterprise application, a "peer" is an application server instance that is acting as a container for the application, and the "client" is that part of the application that is directly accessing and modifying the caches and listening to events from the caches.
Some events originate from within a cache itself. There are many examples, but the most common cases are:
- When entries automatically expire from a cache;
- When entries are evicted from a cache because the maximum size of the cache has been reached;
- When entries are transparently added to a cache as the result of a Read-Through operation;
- When entries in a cache are transparently updated as the result of a Read-Ahead or Refresh-Ahead operation.
Each of these represents a modification, but the modifications represent natural (and typically automatic) operations from within a cache. These events are referred to as synthetic events.
When necessary, an application can differentiate between client-induced and synthetic events simply by asking the event if it is synthetic. This information is carried on a sub-class of the MapEvent, called CacheEvent. Using the previous EventPrinter example, it is possible to print only the synthetic events:
public static class EventPrinter
extends MultiplexingMapListener
{
public void onMapEvent(MapEvent evt)
{
if (evt instanceof CacheEvent && ((CacheEvent) evt).isSynthetic())
{
out(evt);
)
}
}
For more information on this feature, see the API documentation for CacheEvent.
Advanced: Backing Map Events
While it is possible to listen to events from Coherence caches, each of which presents a local view of distributed, partitioned, replicated, near-cached, continuously-queried, read-through/write-through and/or write-behind data, it is also possible to peek behind the curtains, so to speak. Normally, the advice from the Wizard of Oz is sufficient:
 | "Pay no attention to the man behind the curtain!"
|
For some advanced use cases, it may be necessary to pay attention the man behind the curtain – or more correctly, to "listen to" the "map" behind the "service". Replication, partitioning and other approaches to managing data in a distributed environment are all distribution services. The service still has to have something in which to actually manage the data, and that something is called a "backing map".
Backing maps are configurable. If all the data for a particular cache should be kept in object form on the heap, then use an unlimited and non-expiring LocalCache (or a SafeHashMap if statistics are not required). If only a small number of items should be kept in memory, use a LocalCache. If data are to be read on demand from a database, then use a ReadWriteBackingMap (which knows how to read and write through an application's DAO implementation), and in turn give the ReadWriteBackingMap a backing map such as a SafeHashMap or a LocalCache to store its data in.
Some backing maps are observable. The events coming from these backing maps are not usually of direct interest to the appication. Instead, Coherence translates them into actions that must be taken (by Coherence) to keep data in sync and properly backed up, and it also translates them when appropriate into clustered events that are delivered throughout the cluster as requested by application listeners. For example, if a partitioned cache has a LocalCache as its backing map, and the local cache expires an entry, that event causes Coherence to expire all of the backup copies of that entry. Furthermore, if any listeners have been registered on the partitioned cache, and if the event matches their event filter(s), then that event will be delivered to those listeners on the servers where those listeners were registered.
In some advanced use cases, an application needs to process events on the server where the data are being maintained, and it needs to do so on the structure (backing map) that is actually managing the data. In these cases, if the backing map is an observable map, a listener can be configured on the backing map or one can be programmatically added to the backing map. (If the backing map is not observable, it can be made observable by wrapping it in an WrapperObservableMap.)
For more information on this feature, see the API documentation for BackingMapManager.
Advanced: Synchronous Event Listeners
Some events are delivered asynchronously, so that application listeners do not disrupt the cache services that are generating the events. In some rare scenarios, asynchronous delivery can cause ambiguity of the ordering of events compared to the results of ongoing operations. To guarantee that the cache API operations and the events are ordered as if the local view of the clustered system were single-threaded, a MapListener must implement the SynchronousListener marker interface.
One example in Coherence itself that uses synchronous listeners is the Near Cache, which can use events to invalidate locally cached data ("Seppuku").
For more information on this feature, see the API documentation for SynchronousListener.
Summary
Coherence provides an extremely rich event model for caches, providing the means for an application to request the specific events it requires, and the means to have those events delivered only to those parts of the application that require them.
Automatically manage dynamic cluster membership
Overview
Coherence manages cluster membership, automatically adding new servers to the cluster when they start up and automatically detecting their departure when they are shut down or fail. Applications have full access to this information, and can sign up to receive event notifications when members join and leave the cluster. Coherence also tracks all the services that each member is providing and consuming, and uses this information to plan for service resiliency in case of server failure, and to load-balance data management and other responsibilities across all members of the cluster.
Cluster and Service objects
From any cache, the application can obtain a reference to the local representation of a cache's service. From any service, the application can obtain a reference to the local representation of the cluster.
CacheService service = cache.getCacheService();
Cluster cluster = service.getCluster();
From the Cluster object, the application can determine the set of services that are running in the cluster:
for (Enumeration enum = cluster.getServiceNames(); enum.hasMoreElements(); )
{
String sName = (String) enum.nextElement();
ServiceInfo info = cluster.getServiceInfo(sName);
}
The ServiceInfo object provides information about the service, including its name, type, version and membership.
For more information on this feature, see the API documentation for NamedCache, CacheService, Service, ServiceInfo and Cluster.
Member object
The primary information that an application can determine about each member in the cluster is:
- The Member's IP address
- What date/time the Member joined the cluster
As an example, if there are four servers in the cluster with each server running one copy ("instance") of the application and all four instances of the application are clustered together, then the cluster is composed of four Members. From the Cluster object, the application can determine what the local Member is:
Member memberThis = cluster.getLocalMember();
From the Cluster object, the application can also determine the entire set of cluster members:
Set setMembers = cluster.getMemberSet();
From the ServiceInfo object, the application can determine the set of cluster members that are participating in that service:
ServiceInfo info = cluster.getServiceInfo(sName);
Set setMembers = info.getMemberSet();
For more information on this feature, see the [API documentation for Member.
Listener interface and Event object
To listen to cluster and/or service membership changes, the application places a listener on the desired Service. As discussed before, the Service can come from a cache:
Service service = cache.getCacheService();
The Service can also be looked up by its name:
Service service = cluster.getService(sName);
To receive membership events, the application implements a MemberListener. For example, the following listener example prints out all the membership events that it receives:
public class MemberEventPrinter
extends Base
implements MemberListener
{
public void memberJoined(MemberEvent evt)
{
out(evt);
}
public void memberLeaving(MemberEvent evt)
{
out(evt);
}
public void memberLeft(MemberEvent evt)
{
out(evt);
}
}
The MemberEvent object carries information about the event type (joined / leaving / left), the member that generated the event, and the service that acts as the source of the event. Additionally, the event provides a method, isLocal(), that indicates to the application that it is this member that is joining or leaving the cluster. This is useful for recognizing soft restarts in which an application automatically rejoins a cluster after a failure occurs. For example:
public class RejoinEventPrinter
extends Base
implements MemberListener
{
public void memberJoined(MemberEvent evt)
{
if (evt.isLocal())
{
out("this member just rejoined the cluster: " + evt);
}
}
public void memberLeaving(MemberEvent evt)
{
}
public void memberLeft(MemberEvent evt)
{
}
}
For more information on these feature, see the [API documentation for Service, MemberListener and MemberEvent.
Provide a Queryable Data Fabric
Overview
Tangosol invented the concept of a data fabric with the introduction of the Coherence partitioned data management service in 2002. Since then, Forrester Research has labeled the combination of data virtualization, transparent and distributed EIS integration, queryability and uniform accessibility found in Coherence as an information fabric. The term fabric comes from a 2-dimensional illustration of interconnects, as in a switched fabric. The purpose of a fabric architecture is that all points within a fabric have a direct interconnect with all other points.
Data Fabric
An information fabric, or the more simple form called a data fabric or data grid, uses a switched fabric concept as the basis for managing data in a distributed environment. Also referred to as a dynamic mesh architecture, Coherence automatically and dynamically forms a reliable, increasingly resilient switched fabric composed of any number of servers within a grid environment. Consider the attributes and benefits of this architecture:
- The aggregate data throughput of the fabric is linearly proportional to the number of servers;
- The in-memory data capacity and data-indexing capacity of the fabric is linearly proportional to the number of servers;
- The aggregate I/O throughput for disk-based overflow and disk-based storage of data is linearly proportional to the number of servers;
- The resiliency of the fabric increases with the extent of the fabric, resulting in each server being responsible for only 1/n of the failover responsibility for a fabric with an extent of n servers;
- If the fabric is servicing clients, such as trading systems, the aggregage maximum number of clients that can be served is linearly proportional to the number of servers.
Coherence accomplishes these technical feats through a variety of algorithms:
- Coherence dynamically partitions data across all data fabric nodes;
- Since each data fabric node has a configurable maximum amount of data that it will manage, the capacity of the data fabric is linearly proportional to the number of data fabric nodes;
- Since the partitioning is automatic and load-balancing, each data fabric node ends up with its fair share of the data management responsibilities, allowing the throughput (in terms of network throughput, disk I/O throughput, query throughput, etc.) to scale linearly with the number of data fabric nodes;
- Coherence maintains a configurable level of redundancy of data, automatically eliminating single points of failure (SPOFs) by ensuring that data is kept synchronously up-to-date in multiple data fabric nodes;
- Coherence spreads out the responsibility for data redundancy in a dynamically load-balanced manner so that each server backs up a small amount of data from many other servers, instead of backing up all of the data from one particular server, thus amortizing the impact of a server failure across the entire data fabric;
- Each data fabric node can handle a large number of client connections, which can be load-balanced by a hardware load balancer.
EIS and Database Integration
The Coherence information fabric can automatically load data on demand from an underlying database or EIS using automatic read-through functionality. If data in the fabric are modified, the same functionality allows that data to be synchronously updated in the database, or queued for asynchronous write-behind.
Coherence automatically partitions data access across the data fabric, resulting in load-balanced data accesses and efficient use of database and EIS connectivity. Furthermore, the read-ahead and write-behind capabilities can cut data access latencies to near-zero levels and insulate the application from temporary database and EIS failures.
| Coherence solves the data bottleneck for large-scale compute grids
In large-scale compute grids, such as in DataSynapse financial grids and biotech grids, the bottleneck for most compute processes is in loading a data set and making it available to the compute engines that require it. By layering a Coherence data fabric onto (or beside) a compute grid, these data sets can be maintained in memory at all times, and Coherence can feed the data in parallel at close to wire speed to all of the compute nodes. In a large-scale deployment, Coherence can provide several thousand times the aggregate data throughput of the underlying data source. |
Queryable
The Coherence information fabric supports querying from any server in the fabric or any client of the fabric. The queries can be performed using any criteria, including custom criteria such as XPath queries and full text searches. When Coherence partitioning is used to manage the data, the query is processed in parallel across the entire fabric (i.e. the query is also partitioned), resulting in an data query engine that can scale its throughput up to fabrics of thousands of servers. For example, in a trading system it is possible to query for all open "Order" objects for a particular trader:
NamedCache mapTrades = ...
Filter filter = new AndFilter(new EqualsFilter("getTrader", traderid),
new EqualsFilter("getStatus", Status.OPEN));
Set setOpenTrades = mapTrades.entrySet(filter);
When an application queries for data from the fabric, the result is a point-in-time snapshot. Additionally, the query results can be kept up-to-date by placing a listener on the query itself or by using the Coherence Continuous Query feature.
Continuous Query
While it is possible to obtain a point in time query result from a Coherence data fabric, and it is possible to receive events that would change the result of that query, Coherence provides a feature that combines a query result with a continuous stream of related events that maintain the query result in a real-time fashion. This capability is called Continuous Query, because it has the same effect as if the desired query had zero latency and the query were repeated several times every millisecond!
Coherence implements Continuous Query using a combination of its data fabric parallel query capability and its real-time event-filtering and streaming. The result is support for thousands of client application instances, such as trading desktops. Using the previous trading system example, it can be converted to a Continuous Query with only one a single line of code changed:
NamedCache mapTrades = ...
Filter filter = new AndFilter(new EqualsFilter("getTrader", traderid),
new EqualsFilter("getStatus", Status.OPEN));
NamedCache mapOpenTrades = new ContinuousQueryCache(mapTrades, filter);
The result of the Continuous Query is maintained locally, and optionally all of corresponding data can be cached locally as well.
Summary
Coherence is successfully deployed as a large-scale data fabric for many of the world's largest financial, telecommunications, logistics, travel and media organizations. With unlimited scalability, the highest levels of availability, close to zero latency, an incredibly rich set of capabilities and a sterling reputation for quality, Coherence is the Information Fabric of choice.
Provide a Data Grid
Overview
Coherence provides the ideal infrastructure for building Data Grid services, as well as the client and server-based applications that utilize a Data Grid. At a basic level, Coherence can manage an immense amount of data across a large number of servers in a grid; it can provide close to zero latency access for that data; it supports parallel queries across that data; and it supports integration with database and EIS systems that act as the system of record for that data. For more information on the infrastructure for the Data Grid features in Coherence, refer to the discussion on Data Fabric capabilities. Additionally, Coherence provides a number of services that are ideal for building effective data grids.
| All of the Data Grid capabilities described below are features of the Coherence Enterprise Edition.
|
Targeted Execution
Coherence provides for the ability to execute an agent against an entry in any map of data managed by the Data Grid:
In the case of partitioned data, the agent executes on the grid node that owns the data to execute against. This means that the queueing, concurrency management, agent execution, data access by the agent and data modification by the agent all occur on that grid node. (Only the synchronous backup of the resultant data modification, if any, requires additional network traffic.) For many processing purposes, it is much more efficient to move the serialized form of the agent (usually only a few hundred bytes, at most) than to handle distributed concurrency control, coherency and data updates.
For request/response processing, the agent returns a result:
Object oResult = map.invoke(key, agent);
In other words, Coherence as a Data Grid will determine the location to execute the agent based on the configuration for the data topology, move the agent there, execute the agent (automatically handling concurrency control for the item while executing the agent), back up the modifications if any, and return a result.
Parallel Execution
Coherence additionally provides for the ability to execute an agent against an entire collection of entries. In a partitioned Data Grid, the execution occurs in parallel, meaning that the more nodes that are in the grid, the broader the work is load-balanced across the Data Grid:
map.invokeAll(collectionKeys, agent);
For request/response processing, the agent returns one result for each key processed:
Map mapResults = map.invokeAll(collectionKeys, agent);
In other words, Coherence determines the optimal location(s) to execute the agent based on the configuration for the data topology, moves the agent there, executes the agent (automatically handling concurrency control for the item(s) while executing the agent), backing up the modifications if any, and returning the coalesced results.
Query-Based Execution
As discussed in the queryable data fabric topic, Coherence supports the ability to query across the entire data grid. For example, in a trading system it is possible to query for all open "Order" objects for a particular trader:
NamedCache map = CacheFactory.getCache("trades");
Filter filter = new AndFilter(new EqualsFilter("getTrader", traderid),
new EqualsFilter("getStatus", Status.OPEN));
Set setOpenTradeIds = mapTrades.keySet(filter);
By combining this feature with Parallel Execution in the data grid, Coherence provides for the ability to execute an agent against a query. As in the previous section, the execution occurs in parallel, and instead of returning the identities or entries that match the query, Coherence executes the agents against the entries:
map.invokeAll(filter, agent);
For request/response processing, the agent returns one result for each key processed:
Map mapResults = map.invokeAll(filter, agent);
In other words, Coherence combines its Parallel Query and its Parallel Execution together to achieve query-based agent invocation against a Data Grid.
Data-Grid-Wide Execution
Passing an instance of AlwaysFilter (or a null) to the invokeAll method will cause the passed agent to be executed against all entries in the InvocableMap:
map.invokeAll((Filter) null, agent);
As with the other types of agent invocation, request/response processing is supported:
Map mapResults = map.invokeAll((Filter) null, agent);
In other words, with a single line of code, an application can process all the data spread across a particular map in the Data Grid.
Agents for Targeted, Parallel and Query-Based Execution
An agent implements the EntryProcessor interface, typically by extending the AbstractProcessor class.
A number of agents are included with Coherence, including:
- AbstractProcessor - an abstract base class for building an EntryProcessor
- ExtractorProcessor - extracts and returns a specific value (such as a property value) from an object stored in an InvocableMap
- CompositeProcessor - bundles together a collection of EntryProcessor objects that are invoked sequentially against the same Entry
- ConditionalProcessor - conditionally invokes an EntryProcessor if a Filter against the Entry-to-process evaluates to true
- PropertyProcessor - an abstract base class for EntryProcessor implementations that depend on a PropertyManipulator
- NumberIncrementor - pre- or post-increments any property of a primitive integral type, as well as Byte, Short, Integer, Long, Float, Double, BigInteger, BigDecimal
- NumberMultiplier - multiplies any property of a primitive integral type, as well as Byte, Short, Integer, Long, Float, Double, BigInteger, BigDecimal, and returns either the previous or new value
The EntryProcessor interface (contained within the InvocableMap interface) contains only two methods:
/**
* An invocable agent that operates against the Entry objects within a
* Map.
*/
public interface EntryProcessor
extends Serializable
{
/**
* Process a Map Entry.
*
* @param entry the Entry to process
*
* @return the result of the processing, if any
*/
public Object process(Entry entry);
/**
* Process a Set of InvocableMap Entry objects. This method is
* semantically equivalent to:
* <pre>
* Map mapResults = new ListMap();
* for (Iterator iter = setEntries.iterator(); iter.hasNext(); )
* {
* Entry entry = (Entry) iter.next();
* mapResults.put(entry.getKey(), process(entry));
* }
* return mapResults;
* </pre>
*
* @param setEntries a read-only Set of InvocableMap Entry objects to
* process
*
* @return a Map containing the results of the processing, up to one
* entry for each InvocableMap Entry that was processed, keyed
* by the keys of the Map that were processed, with a
* corresponding value being the result of the processing for
* each key
*/
public Map processAll(Set setEntries);
}
(The AbstractProcessor implements the processAll method as described in the JavaDoc above.)
The InvocableMap.Entry that is passed to an EntryProcessor is an extension of the Map.Entry interface that allows an EntryProcessor implementation to obtain the necessary information about the entry and to make the necessary modifications in the most efficient manner possible:
/**
* An InvocableMap Entry contains additional information and exposes
* additional operations that the basic Map Entry does not. It allows
* non-existent entries to be represented, thus allowing their optional
* creation. It allows existent entries to be removed from the Map. It
* supports a number of optimizations that can ultimately be mapped
* through to indexes and other data structures of the underlying Map.
*/
public interface Entry
extends Map.Entry
{
/**
* Return the key corresponding to this entry. The resultant key does
* not necessarily exist within the containing Map, which is to say
* that <tt>InvocableMap.this.containsKey(getKey)</tt> could return
* false. To test for the presence of this key within the Map, use
* {@link #isPresent}, and to create the entry for the key, use
* {@link #setValue}.
*
* @return the key corresponding to this entry; may be null if the
* underlying Map supports null keys
*/
public Object getKey();
/**
* Return the value corresponding to this entry. If the entry does
* not exist, then the value will be null. To differentiate between
* a null value and a non-existent entry, use {@link #isPresent}.
* <p/>
* <b>Note:</b> any modifications to the value retrieved using this
* method are not guaranteed to persist unless followed by a
* {@link #setValue} or {@link #update} call.
*
* @return the value corresponding to this entry; may be null if the
* value is null or if the Entry does not exist in the Map
*/
public Object getValue();
/**
* Store the value corresponding to this entry. If the entry does
* not exist, then the entry will be created by invoking this method,
* even with a null value (assuming the Map supports null values).
*
* @param oValue the new value for this Entry
*
* @return the previous value of this Entry, or null if the Entry did
* not exist
*/
public Object setValue(Object oValue);
/**
* Store the value corresponding to this entry. If the entry does
* not exist, then the entry will be created by invoking this method,
* even with a null value (assuming the Map supports null values).
* <p/>
* Unlike the other form of {@link #setValue(Object) setValue}, this
* form does not return the previous value, and as a result may be
* significantly less expensive (in terms of cost of execution) for
* certain Map implementations.
*
* @param oValue the new value for this Entry
* @param fSynthetic pass true only if the insertion into or
* modification of the Map should be treated as a
* synthetic event
*/
public void setValue(Object oValue, boolean fSynthetic);
/**
* Extract a value out of the Entry's value. Calling this method is
* semantically equivalent to
* <tt>extractor.extract(entry.getValue())</tt>, but this method may
* be significantly less expensive because the resultant value may be
* obtained from a forward index, for example.
*
* @param extractor a ValueExtractor to apply to the Entry's value
*
* @return the extracted value
*/
public Object extract(ValueExtractor extractor);
/**
* Update the Entry's value. Calling this method is semantically
* equivalent to:
* <pre>
* Object oTarget = entry.getValue();
* updater.update(oTarget, oValue);
* entry.setValue(oTarget, false);
* </pre>
* The benefit of using this method is that it may allow the Entry
* implementation to significantly optimize the operation, such as
* for purposes of delta updates and backup maintenance.
*
* @param updater a ValueUpdater used to modify the Entry's value
*/
public void update(ValueUpdater updater, Object oValue);
/**
* Determine if this Entry exists in the Map. If the Entry is not
* present, it can be created by calling {@link #setValue} or
* {@link #setValue}. If the Entry is present, it can be destroyed by
* calling {@link #remove}.
*
* @return true iff this Entry is existent in the containing Map
*/
public boolean isPresent();
/**
* Remove this Entry from the Map if it is present in the Map.
* <p/>
* This method supports both the operation corresponding to
* {@link Map#remove} as well as synthetic operations such as
* eviction. If the containing Map does not differentiate between
* the two, then this method will always be identical to
* <tt>InvocableMap.this.remove(getKey())</tt>.
*
* @param fSynthetic pass true only if the removal from the Map
* should be treated as a synthetic event
*/
public void remove(boolean fSynthetic);
}
Data Grid Aggregation
While the above agent discussion correspond to scalar agents, the InvocableMap interface also supports aggregation:
/**
* Perform an aggregating operation against the entries specified by the
* passed keys.
*
* @param collKeys the Collection of keys that specify the entries within
* this Map to aggregate across
* @param agent the EntryAggregator that is used to aggregate across
* the specified entries of this Map
*
* @return the result of the aggregation
*/
public Object aggregate(Collection collKeys, EntryAggregator agent);
/**
* Perform an aggregating operation against the set of entries that are
* selected by the given Filter.
* <p/>
* <b>Note:</b> calling this method on partitioned caches requires a
* Coherence Enterprise Edition license.
*
* @param filter the Filter that is used to select entries within this
* Map to aggregate across
* @param agent the EntryAggregator that is used to aggregate across
* the selected entries of this Map
*
* @return the result of the aggregation
*/
public Object aggregate(Filter filter, EntryAggregator agent);
A simple EntryAggregator processes a set of InvocableMap.Entry objects to achieve a result:
/**
* An EntryAggregator represents processing that can be directed to occur
* against some subset of the entries in an InvocableMap, resulting in a
* aggregated result. Common examples of aggregation include functions
* such as min(), max() and avg(). However, the concept of aggregation
* applies to any process that needs to evaluate a group of entries to
* come up with a single answer.
*/
public interface EntryAggregator
extends Serializable
{
/**
* Process a set of InvocableMap Entry objects in order to produce an
* aggregated result.
*
* @param setEntries a Set of read-only InvocableMap Entry objects to
* aggregate
*
* @return the aggregated result from processing the entries
*/
public Object aggregate(Set setEntries);
}
For efficient execution in a Data Grid, an aggregation process must be designed to operate in a parallel manner.
/**
* A ParallelAwareAggregator is an advanced extension to EntryAggregator
* that is explicitly capable of being run in parallel, for example in a
* distributed environment.
*/
public interface ParallelAwareAggregator
extends EntryAggregator
{
/**
* Get an aggregator that can take the place of this aggregator in
* situations in which the InvocableMap can aggregate in parallel.
*
* @return the aggregator that will be run in parallel
*/
public EntryAggregator getParallelAggregator();
/**
* Aggregate the results of the parallel aggregations.
*
* @return the aggregation of the parallel aggregation results
*/
public Object aggregateResults(Collection collResults);
}
Coherence comes with all of the natural aggregation functions, including:
| All aggregators that come with Coherence are parallel-aware.
|
See the com.tangosol.util.aggregator package for a list of Coherence aggregators. To implement your own aggregator, see the AbstractAggregator abstract base class.
Node-Based Execution
Coherence provides an Invocation Service which allows execution of single-pass agents (called Invocable objects) anywhere within the grid. The agents can be executed on any particular node of the grid, in parallel on any particular set of nodes in the grid, or in parallel on all nodes of the grid.
An invocation service is configured using the invocation-scheme element in the cache configuration file. Using the name of the service, the application can easily obtain a reference to the service:
InvocationService service = CacheFactory.getInvocationService("agents");
Agents are simply runnable classes that are part of the application. The simplest example is a simple agent that is designed to request a GC from the JVM:
/**
* Agent that issues a garbage collection.
*/
public class GCAgent
extends AbstractInvocable
{
public void run()
{
System.gc();
}
}
To execute that agent across the entire cluster, it takes one line of code:
service.execute(new GCAgent(), null, null);
Here is an example of an agent that supports a grid-wide request/response model:
/**
* Agent that determines how much free memory a grid node has.
*/
public class FreeMemAgent
extends AbstractInvocable
{
public void run()
{
Runtime runtime = Runtime.getRuntime();
int cbFree = runtime.freeMemory();
int cbTotal = runtime.totalMemory();
setResult(new int[] {cbFree, cbTotal});
}
}
To execute that agent across the entire grid and retrieve all the results from it, it still takes only one line of code:
Map map = service.query(new FreeMemAgent(), null);
While it is easy to do a grid-wide request/response, it takes a bit more code to print out the results:
Iterator iter = map.entrySet().iterator();
while (iter.hasNext())
{
Map.Entry entry = (Map.Entry) iter.next();
Member member = (Member) entry.getKey();
int[] anInfo = (int[]) entry.getValue();
if (anInfo != null) System.out.println("Member " + member + " has "
+ anInfo[0] + " bytes free out of "
+ anInfo[1] + " bytes total");
}
The agent operations can be stateful, which means that their invocation state is serialized and transmitted to the grid nodes on which the agent is to be run.
/**
* Agent that carries some state with it.
*/
public class StatefulAgent
extends AbstractInvocable
{
public StatefulAgent(String sKey)
{
m_sKey = sKey;
}
public void run()
{
String sKey = m_sKey;
}
private String m_sKey;
}
Work Manager
Coherence provides a grid-enabled implementation of the IBM and BEA CommonJ Work Manager, which is the basis for JSR-237. Once JSR-237 is complete, Tangosol has committed to support the standardized J2EE API for Work Manager as well.
Using a Work Manager, an application can submit a collection of work that needs to be executed. The Work Manager distributes that work in such a way that it is executed in parallel, typically across the grid. In other words, if there are ten work items submitted and ten servers in the grid, then each server will likely process one work item. Further, the distribution of work items across the grid can be tailored, so that certain servers (e.g. one that acts as a gateway to a particular mainframe service) will be the first choice to run certain work items, for sake of efficiency and locality of data.
The application can then wait for the work to be completed, and can provide a timeout for how long it is willing to wait. The API for this purpose is quite powerful, allowing an application to wait for the first work item to complete, or for a specified set of the work items to complete. By combining methods from this API, it is possible to do things like "Here are 10 items to execute; for these 7 unimportant items, wait no more than 5 seconds, and for these 3 important items, wait no more than 30 seconds":
Work[] aWork = ...
Collection collBigItems = new ArrayList();
Collection collAllItems = new ArrayList();
for (int i = 0, c = aWork.length; i < c; ++i)
{
WorkItem item = manager.schedule(aWork[i]);
if (i < 3)
{
collBigItems.add(item);
}
collAllItems.add(item);
}
Collection collDone = manager.waitForAll(collAllItems, 5000L);
if (!collDone.containsAll(collBigItems))
{
manager.waitForAll(collBigItems, 25000L);
}
Of course, the best descriptions come from real-world production usage:
Our primary use case for the Work Manager is to allow our application to serve coarse-grained service requests using our blade infrastructure in a standards-based way. We often have what appears to be a simple request, like "give me this family's information." In reality, however, this request expands into a large number of requests to several diverse back-end data sources consisting of web services, RDMBS calls, etc. This use case expands into two different but related problems that we are looking to the distributed version of the work manager to solve.
1. How do we take a coarse-grained request that expands into several fine-grained requests and execute them in parallel to avoid blocking the caller for an unreasonable time? In the above example, we may have to make upwards of 100 calls to various places to retrieve the information. Since J2EE has no legal threading model, and since the threading we observed when trying a message-based approach to this was unacceptable, we decided to use the Coherence Work Manager implementation.
2. Given that we want to make many external system calls in parallel while still leveraging low-cost blades, we are hoping that fanning the required work across many dual processor (logically 4-processor because of hyperthreading) machines allows us to scale an inherently vertical scalability problem with horizontal scalability at the hardware level. We think this is reasonable because the cost to marshall the request to a remote Work Manager instance is small compared to the cost to execute the service, which usually involves dozens or hundreds of milliseconds.
For more information on the Work Manager Specification and API, see Timer and Work Manager for Application Servers on the BEA dev2dev web site and JSR 237.
Summary
Coherence provides an extensive set of capabilities that make Data Grid services simple, seamless and seriously scalable. While the data fabric provides an entire unified view of the complete data domain, the Data Grid features enable applications to take advantage of the partitioning of data that Coherence provides in a scale-out environment.
Using Coherence and BEA WebLogic Portal
Introduction
Coherence integrates closely with BEATM
WebLogicTM
Portal to provide WAN-capable clustered session management and caching for portal applications. Specifically, Coherence includes the following integration points:
- Coherence*Web support for WebLogic Portal
- P13N CacheProvider SPI implementation
- A blueprint for efficiently sharing data between WSRP-federated portals that leverages Coherence and the WebLogic Portal Custom Data Transfer mechanism
Coherence*Web for WebLogic Portal
When Coherence*Web is installed into a WebLogic Portal web application, everything that the portal framework and portlets place into the HttpSession will be managed by Coherence. This has several benefits as described in the following article:
http://dev2dev.bea.com/pub/a/2005/05/session_management.html
Additionally, combining Coherence*Web and WebLogic Portal gives you extreme flexibility in your choice of a session management cache topology. For example, if you find that your Portal servers are bumping into the 4GB heap limit (on 32-bit JVMs) or are experiencing slow GC times, you can leverage a cache client/server topology to move all HttpSession state out of your Portal JVMs and into one or more dedicated Coherence cache servers, thus reducing your Portal JVM heap size and GC times. Also, you can leverage the Coherence Management Framework to closely monitor HttpSession-specific statistics to better tune your Coherence*Web and session management cache settings.
For details on installing Coherence*Web into a WebLogic Portal web application, please see Installing Coherence*Web Session Management Module.
P13N CacheProvider SPI Implementation
| Requires WebLogic Portal 8.1.6
Please note that the following feature requires WebLogic Portal 8.1.6. |
Internally, WebLogic Portal uses its own caching service to cache portal, personalization, and commerce data as described here:
http://e-docs.bea.com/wlp/docs81/javadoc/com/bea/p13n/cache/package-summary.html
WebLogic Portal 8.1.6 includes an SPI for the P13N caching service that can be implemented by third party cache vendors. Coherence includes a P13N CacheProvider SPI implementation that - when installed into a WebLogic Portal application - has the same benefits for serializable WebLogic Portal data as Coherence*Web has for HttpSession state, all without requiring code changes. Additionally, the Coherence CacheProvider allows your portlets to leverage Coherence caching services simply by using the standard P13N Cache API.
To install the Coherence P13N CacheProvider, simply copy the coherence-wlp.jar, coherence.jar and tangosol.jar libraries included in the lib directory of the Coherence installation to the APP-INF/lib directory of your WebLogic Portal application. On startup, WebLogic Portal will automatically discover the Coherence CacheProvider and transparently use it to cache data.
Please see the JavaDoc for the
PortalCacheProvider class for details on configuring the Coherence CacheProvider and Coherence caches used by the provider. Additionally, please see the following document for a list of some of the caches used by WebLogic Portal:
http://e-docs.bea.com/wlp/docs81/perftune/apenB.html
Sharing Data Between WSRP-Federated Portals Using Coherence
The Web Services for Remote Portlets (WSRP) protocol was designed to support the federation of portals hosted by arbitrary portal servers and server clusters. Developers use WSRP to aggregate content and the user interface (UI) from various portlets hosted by other remote portals. By itself, though, WSRP does not address the challenge of implementing scalable, reliable, and high-performance federated portals that create, access, and manage the lifecycle of data shared by distributed portlets. Fortunately, BEA WebLogic Portal provides an extension to the WSRP specification that — when coupled with Tangosol Coherence — allows WSRP Consumers and Producers to create, view, modify, and control concurrent access to shared, scoped data in a scalable, reliable, and highly performant manner.
See the following document for complete details:
http://dev2dev.bea.com/pub/a/2005/11/federated-portal-cache.html
Using Coherence and Hibernate
Introduction
Hibernate and Coherence can be used together in several combinations. This document discusses the various options, including when and each one is appropriate, along with usage instructions. These options including using Coherence as a Hibernate plug-in, using Hibernate as a Coherence plug-in via the CacheStore interface and bulk-loading Coherence caches from a Hibernate query. Most applications that use Coherence and Hibernate use a mixture of these approaches. The Hibernate API features powerful management of entities and relationships, and the Coherence API delivers maximum performance and scalability.
Conventions
This document refers to the following Java classes and interfaces:
com.tangosol.coherence.hibernate.CoherenceCache
com.tangosol.coherence.hibernate.CoherenceCacheProvider
com.tangosol.coherence.hibernate.HibernateCacheLoader
com.tangosol.coherence.hibernate.HibernateCacheStore
com.tangosol.net.NamedCache (extends java.util.Map)
com.tangosol.net.cache.CacheLoader
com.tangosol.net.cache.CacheStore
org.hibernate.Query
org.hibernate.Session
org.hibernate.SessionFactory
As the CacheStore interface extends CacheLoader, the term "CacheStore" will be used generically to refer to both interfaces (the appropriate interface being determined by whether read-only or read-write support is required). Similarly, "HibernateCacheStore" will refer to both implementations.
The Coherence cache configuration file is referred to as coherence-cache-config.xml (the default name) and the Hibernate root configuration file is referred to as hibernate.cfg.xml (the default name).
Selecting a Caching Strategy
Overview
Generally, the Hibernate API is the optimal choice for accessing data held in a relational database where performance is not the dominant factor. For application state (or any type of data that fits naturally into the Map interface) use the Coherence API. For performance-sensitive operations, specifically those that may benefit from Coherence-specific features like write-behind caching or cache queries, use the Coherence API.
Hibernate API
The Hibernate API provides flexible queries and relational management features including referential integrity, cascading deletes and child object fetching. While these features may be implemented using Coherence, this involves development effort which may not be worthwhile in cases where performance is not an issue.
Coherence NamedCache API
There are many Coherence features that require direct access to the Coherence
NamedCache API, including:
- Write-Behind Caching (low-latency, high-throughput database updates)
- Distributed Queries (low-latency, high-throughput search queries)
- Cache Transactions (application-tier transactions)
- InvocableMap (stored procedures, aggregations)
- Invocation Service (messaging and remote invocation)
- Cache Listeners (event-based processing)
Direct access to these features may be critical for achieving the highest levels of scalable performance.
Coherence CacheStore Integration
CacheStore modules are useful for transparently keeping cache and database synchronized. They are also more efficient than independently updating the cache and database as updates are routed through Coherence's partitioning facilities, minimizing locking.
CacheStore modules give very high performance for caching that can be expressed via a Map interface, that is a key-value pair. The NamedCache interface is a much simpler and by extension much lower-overhead API than the Hibernate query API. Additionally, in some cases (where complex queries can be mapped into a key-based pattern), very complex queries can be answered by a simple cache retrieval.
One final reason for using CacheStore is that it provides a means of coordinating all database (or other backend) access through a single API (NamedCache) and through a controlled set of JVMs (server machines). This is because the nodes which are responsible for managing cache partitions are the same machines responsible for synchronizing with the database server.
Using Coherence as the Hibernate L2 Cache
Introduction
Hibernate supports three primary forms of caching:
- Session cache
- L2 cache
- Query cache
The Session cache is responsible for caching records within a Session (a Hibernate transaction, potentially spanning multiple database transactions, and typically scoped on a per-thread basis). As a non-clustered cache (by definition), the Session cache is managed entirely by Hibernate. The L2 and Query caches span multiple transactions, and support the use of Coherence as a cache provider. The L2 cache is responsible for caching records across multiple sessions (for primary key lookups). The query cache caches the result sets generated by Hibernate queries. Hibernate manages data in an internal representation in the L2 and Query caches, meaning that these caches are usable only by Hibernate. For more details, see the Hibernate Reference Documentation (shipped with Hibernate), specifically the section on the Second Level Cache.
Configuration and Tuning
To use the Coherence Caching Provider for Hibernate, specify the Coherence provider class in the "hibernate.cache.provider_class" property. Typically this is configured in the default Hibernate configuration file, hibernate.cfg.xml.
<property name="hibernate.cache.provider_class">com.tangosol.coherence.hibernate.CoherenceCacheProvider</property>
The file coherence-hibernate.jar (found in the lib/ subdirectory) must be added to the application classpath.
Hibernate provides the configuration property hibernate.cache.use_minimal_puts, which optimizes cache access for clustered caches by increasing cache reads and decreasing cache updates. This is enabled by default by the Coherence Cache Provider. Setting this property to false may increase overhead for cache management and also increase the number of transaction rollbacks.
The Coherence Caching Provider includes a setting for how long a lock acquisition should be attempted before timing out. This may be specified by the Java property tangosol.coherence.hibernate.lockattemptmillis. The default is one minute.
Specifying a Coherence Cache Topology
By default, the Coherence Caching Provider uses the default Coherence cache configuration file (coherence-cache-config.xml) to define cache mappings for Hibernate L2 caches. If desired, a dedicated cache configuration resource (e.g. hibernate-cache-config.xml) may be specified for Hibernate L2 caches via the tangosol.coherence.hibernate.cacheconfig Java property. It is possible to use the existing coherence-cache-config.xml file if mappings are properly configured. It may be beneficial to use dedicated cache service(s) to manage Hibernate-specific caches to ensure that any CacheStore modules don't cause re-entrant calls back into Coherence-managed Hibernate L2 caches.
In conjunction with the scheme mapping section of the Coherence cache configuration file, the hibernate.cache.region_prefix property may be used to specify a cache topology. For example, if the cache configuration file includes a wildcard mapping for "near-*", and the Hibernate region prefix property is set to "near-", then all Hibernate caches will be named using the "near-" prefix, and will use the cache scheme mapping specified for the "near-*" cache name pattern.
It is possible to specify a cache topology per entity by creating a cache mapping based on the combined prefix and qualified entity name (e.g. "near-com.company.EntityName"); or equivalently, by providing an empty prefix and specifying a cache mapping for each qualified entity name.
Also, L2 caches should be size-limited to avoid excessive memory usage. Query caches in particular must be size-limited as the Hibernate API does not provide any means of controlling the query cache other than a complete eviction.
Cache Concurrency Strategies
Hibernate generally emphasizes the use of optimistic concurrency for both cache and database. With optimistic concurrency in particular, transaction processing depends on having accurate data available to the application at the beginning of the transaction. If the data is inaccurate, the commit processing will detect that the transaction was dependent on incorrect data, and the transaction will fail to commit. While most optimistic transactions must cope with changes to underlying data by other processes, the use of caching adds the possibility of the cache itself being stale. Hibernate provides a number of cache concurrency strategies to control updates to the L2 cache. While this is less of an issue for Coherence due to support for cluster-wide coherent caches, appropriate selection of cache concurrency strategy will aid application efficiency.
Note that cache configuration strategies may be specified at the table level. Generally, the strategy should be specified in the mapping file for the class.
For mixed read-write activity, the read-write strategy is recommended. The transactional strategy is implemented similarly to the nonstrict-read-write strategy, and relies on the optimistic concurrency features of Hibernate. Note that nonstrict-read-write may deliver better performance if its impact on optimistic concurrency is acceptable.
For read-only caching, use the nonstrict-read-write strategy if the underlying database data may change, but slightly stale data is acceptable. If the underlying database data never changes, use the read-only strategy.
Query Cache
To cache query results, set the hibernate.cache.use_query_cache property to "true". Then whenever issuing a cacheable query, use Query.setCacheable(true) to enable caching of query results. As org.hibernate.cache.QueryKey instances in Hibernate may not be binary-comparable (due to non-deterministic serialization of unordered data members), use a size-limited Local or Replicated cache to store query results (which will force the use of hashcode()/equals() to compare keys). The default query cache name is "org.hibernate.cache.StandardQueryCache" (unless a default region prefix is provided, in which case "[prefix]." will be prepended to the cache name). Use the cache configuration file to map this cache name to a Local/Replicated topology, or explicitly provide an appropriately-mapped region name when querying.
Fault-Tolerance
The Hibernate L2 cache protocol supports full fault-tolerance during client or server failure. With the read-write cache concurrency strategy, Hibernate will lock items out of the cache at the start of an update transaction, meaning that client-side failures will simply result in uncached entities and an uncommitted transaction. Server-side failures are handled transparently by Coherence (dependent on the specified data backup count).
Deployment
When used with application servers that do not have a unified class loader, the Coherence Cache Provider must be deployed as part of the application so that it can use the application-specific class loader (required to serialize-deserialize objects).
Using the Coherence HibernateCacheStore
Overview
Coherence includes a default entity-based CacheStore implementation, HibernateCacheStore (and a corresponding CacheLoader implementation, HibernateCacheLoader). More detailed technical information may be found in the JavaDoc for the implementing classes.
Configuration
The examples below show a simple HibernateCacheStore constructor, accepting only an entity name. This will configure Hibernate using the default configuration path, which looks for a hibernate.cfg.xml file in the classpath. There is also the ability to pass in a resource name or file specification for the hibernate.cfg.xml file as the second <init-param> (set the <param-type> element to java.lang.String for a resource name and java.io.File for a file specification). See the class JavaDoc for more details.
The following is a simple coherence-cache-config.xml file used to define a NamedCache called "TableA" which caches instances of a Hibernate entity (com.company.TableA). To add additional entity caches, add additional <cache-mapping> elements.
<?xml version="1.0"?>
<!DOCTYPE cache-config SYSTEM "cache-config.dtd">
<cache-config>
<caching-scheme-mapping>
<cache-mapping>
<cache-name>TableA</cache-name>
<scheme-name>distributed-hibernate</scheme-name>
<init-params>
<init-param>
<param-name>entityname</param-name>
<param-value>com.company.TableA</param-value>
</init-param>
</init-params>
</cache-mapping>
</caching-scheme-mapping>
<caching-schemes>
<distributed-scheme>
<scheme-name>distributed-hibernate</scheme-name>
<backing-map-scheme>
<read-write-backing-map-scheme>
<internal-cache-scheme>
<local-scheme></local-scheme>
</internal-cache-scheme>
<cachestore-scheme>
<class-scheme>
<class-name>
com.tangosol.coherence.hibernate.HibernateCacheStore
</class-name>
<init-params>
<init-param>
<param-type>java.lang.String</param-type>
<param-value>{entityname}</param-value>
</init-param>
</init-params>
</class-scheme>
</cachestore-scheme>
</read-write-backing-map-scheme>
</backing-map-scheme>
</distributed-scheme>
</caching-schemes>
</cache-config>
It is also possible to use the pre-defined {cache-name} macro to eliminate the need for the <init-params> portion of the cache mapping:
<?xml version="1.0"?>
<!DOCTYPE cache-config SYSTEM "cache-config.dtd">
<cache-config>
<caching-scheme-mapping>
<cache-mapping>
<cache-name>TableA</cache-name>
<scheme-name>distributed-hibernate</scheme-name>
</cache-mapping>
</caching-scheme-mapping>
<caching-schemes>
<distributed-scheme>
<scheme-name>distributed-hibernate</scheme-name>
<backing-map-scheme>
<read-write-backing-map-scheme>
<internal-cache-scheme>
<local-scheme></local-scheme>
</internal-cache-scheme>
<cachestore-scheme>
<class-scheme>
<class-name>
com.tangosol.coherence.hibernate.HibernateCacheStore
</class-name>
<init-params>
<init-param>
<param-type>java.lang.String</param-type>
<param-value>com.company.{cache-name}</param-value>
</init-param>
</init-params>
</class-scheme>
</cachestore-scheme>
</read-write-backing-map-scheme>
</backing-map-scheme>
</distributed-scheme>
</caching-schemes>
</cache-config>
And, if naming conventions allow, the mapping may be completely generalized to allow a cache mapping for any qualified class name (entity name):
<?xml version="1.0"?>
<!DOCTYPE cache-config SYSTEM "cache-config.dtd">
<cache-config>
<caching-scheme-mapping>
<cache-mapping>
<cache-name>com.company.*</cache-name>
<scheme-name>distributed-hibernate</scheme-name>
</cache-mapping>
</caching-scheme-mapping>
<caching-schemes>
<distributed-scheme>
<scheme-name>distributed-hibernate</scheme-name>
<backing-map-scheme>
<read-write-backing-map-scheme>
<internal-cache-scheme>
<local-scheme></local-scheme>
</internal-cache-scheme>
<cachestore-scheme>
<class-scheme>
<class-name>
com.tangosol.coherence.hibernate.HibernateCacheStore
</class-name>
<init-params>
<init-param>
<param-type>java.lang.String</param-type>
<param-value>{cache-name}</param-value>
</init-param>
</init-params>
</class-scheme>
</cachestore-scheme>
</read-write-backing-map-scheme>
</backing-map-scheme>
</distributed-scheme>
</caching-schemes>
</cache-config>
Configuration Requirements
Hibernate entities accessed via the HibernateCacheStore module must use the "assigned" ID generator and also have a defined ID property.
Be sure to disable the "hibernate.hbm2ddl.auto" property in the hibernate.cfg.xml used by the HibernateCacheStore, as this may cause excessive schema updates (and possible lockups).
JDBC Isolation Level
In cases where all access to a database is through Coherence, CacheStore modules will naturally enforce ANSI-style Repeatable Read isolation as reads and writes are executed serially on a per-key basis (via the Partitioned Cache Service). Increasing database isolation above Repeatable Read will not yield increased isolation as CacheStore operations may span multiple Partitioned Cache nodes (and thus multiple database transactions). Using database isolation levels below Repeatable Read will not result in unexpected anomalies, and may reduce processing load on the database server.
Fault-Tolerance
For single-cache-entry updates, CacheStore operations are fully fault-tolerant in that the cache and database are guaranteed to be consistent during any server failure (including failures during partial updates). While the mechanisms for fault-tolerance vary, this is true for both write-through and write-behind caches.
Coherence does not support two-phase CacheStore operations across multiple CacheStore instances. In other words, if two cache entries are updated, triggering calls to CacheStore modules sitting on separate servers, it is possible for one database update to succeed and for the other to fail. In this case, it may be preferable to use a cache-aside architecture (updating the cache and database as two separate components of a single transaction) in conjunction with the application server transaction manager. In many cases it is possible to design the database schema to prevent logical commit failures (but obviously not server failures). Write-behind caching avoids this issue as "puts" are not affected by database behavior (and the underlying issues will have been addressed earlier in the design process).
Extending HibernateCacheStore
In some cases, it may be desired to extend the HibernateCacheStore with application-specific functionality. The most obvious reason for this is to leverage a pre-existing programmatically-configured SessionFactory instance.
Creating a Hibernate CacheStore
Introduction
While the provided HibernateCacheStore module provides a solution for most entity-based caches, there may be cases where an application-specific CacheStore module is necessary. For example, providing parameterized queries or including or post-processing of query results.
Re-entrant Calls
In a CacheStore-backed cache implementation, when the application thread accesses cached data, the cache operations may trigger a call to the associated CacheStore implementation via the managing CacheService. The CacheStore must not call back into the CacheService API. This implies, indirectly, that Hibernate should not attempt to access cache data. Therefore, all methods in CacheLoader/CacheStore should be careful to call Session.setCacheMode(CacheMode.IGNORE) to disable cache access. Alternatively, the Hibernate configuration may be cloned (either programmatically or via hibernate.cfg.xml), with CacheStore implementations using the version with the cache disabled.
It is important that a CacheStore implementation does not call back into the hosting cache service. Therefore, in addition to avoiding calls to NamedCache methods, you should also ensure that Hibernate itself does not use any cache services. To do this, call Session.setCacheMode(CacheMode.IGNORE) each time a session is used. Alternatively, the Hibernate configuration may be cloned (either programmatically or via hibernate.cfg.xml), with CacheStore implementations using the version with the cache disabled.
Fully Cached DataSets
Distributed Queries
Distributed queries offer the potential for lower latency, higher throughput and less database server load compared to executing queries on the database server. For set-oriented queries, the dataset must be entirely cached to produce correct query results. More precisely, for a query issued against the cache to produce correct results, the query must not depend on any uncached data.
This means that you can create hybrid caches. For example, it is possible to combine two uses of a NamedCache: a fully cached size-limited dataset for querying (e.g. the data for the most recent week), and a partially cached historical dataset used for singleton reads. This is a good approach to avoid data duplication and minimize memory usage.
While fully cached datasets are usually bulk-loaded during application startup (or on a periodic basis), CacheStore integration may be used to ensure that both cache and database are kept fully synchronized.
Detached Processing
Another reason for using fully-cached datasets is to provide the ability to continue application processing even if the underlying database goes down. Using write-behind caching extends this mode of operation to support full read-write applications. With write-behind, the cache becomes (in effect) the temporary system of record. Should the database fail, updates will be queued in Coherence until the connection is restored, at which point all cache changes will be sent to the database.
Getting Started
Introduction
Overview
This document is targeted at software developers and architects. This document provides detailed technical information for installing, configuring, developing with, and finally deploying Tangosol Coherence.
For ease-of-reading, this document uses only the most basic formatting conventions. Code elements and file contents are printed with a fixed-width font. Multi-line code segments are also color-coded for easier reading.
Tangosol Coherence is a JCache-compliant in-memory caching and data management solution for clustered J2EE applications and application servers. Coherence makes sharing and managing data in a cluster as simple as on a single server. It accomplishes this by coordinating updates to the data using cluster-wide concurrency control, replicating and distributing data modifications across the cluster using the highest performing clustered protocol available, and delivering notifications of data modifications to any servers that request them. Developers can easily take advantage of Coherence features using the standard Java collections API to access and modify data, and use the standard JavaBean event model to receive data change notifications. Functionality such as HTTP Session Management is available out-of-the-box for applications deployed to WebLogic, WebSphere, Tomcat, Jetty and other Servlet 2.2, 2.3 and 2.3 compliant application servers.
Terms
Overview
There are several terms which are used to describe the ability of multiple servers to work together to handle additional load or to survive the failure of a particular server:
- Failback
Failback is an extension to failover that allows a server to reclaim its responsibilities once it restarts. For example, "When the server came back up, the processes that it was running previously were failed back to it."
- Failover
Failover refers to the ability of a server to assume the responsibilities of a failed server. For example, "When the server died, its processes failed over to the backup server."
- Federated Server Model
A federated server model allows multiple servers to cooperate in a distributed manner as if they were a single server, while delegating responsibilities to certain specific servers within the federation. For example, in a federated database, servers from different database vendors can appear to an application to be a single server.
- Load Balancer
A load balancer is a hardware device or software program that delegates network requests to a number of servers, such as in a server farm or server cluster. Load balancers typically can detect server failure and optionally retry operations that were being processed by that server at the time of its failure. Load balancers typically attempt to keep the servers to which they delegate equally busy, hence the use of the term "balancer". Load balancer devices often have a high-availability option that uses a second load balancer, allowing one of the load balancer devices to die without affecting availability.
- Server Cluster
A server cluster is composed of multiple servers that are each aware of the other servers in the cluster, can directly communicate with each other, share responsibilities (load-balance), and are able to assume the responsibilities failed servers. Generally speaking, clustering usually implies a concept of shared resources or shared state.
- Server Farm
A server farm utilizes multiple servers to handle increased load and provide increased availability. It is common for a load-balancer to be used to assign work to the various servers in the server farm, and server farms often share back-end resources, such as database servers, but each server is typically unaware of other servers in the farm, and usually the load-balancer is responsible for failover.
- JCache
JCache (also known as JSR-107), is a caching API specification that is currently in progress. While the final version of this API has not been released yet, Tangosol and other companies with caching products have been tracking the current status of the API. The API has been largely solidified at this point. Few significant changes are expected going forward.
It is worth noting that the terms federated server model, server clustering and server farming are often used loosely and interchangeably.
Tangosol Coherence supports both homogenous server clusters and the federated server model. Any application or server process that is running the Coherence software is called a cluster node. All cluster nodes on the same network will automatically cluster together . Cluster nodes use a peer-to-peer protocol, which means that any cluster node can talk directly to any other cluster node.
Coherence is logically sub-divided into clusters, services and caches. A Coherence cluster is a group of cluster nodes that share a group address, which allows the cluster nodes to communicate. Generally, a cluster node will only join one cluster, but it is possible for a cluster node to join (be a member of) several different clusters, by using a different group address for each cluster.
Within a cluster, there exists any number of named services. A cluster node can participate in (join) any number of these services; when a cluster node joins a service, it automatically has all of the information from that service available to it; for example, if the service is a replicated cache service, then joining the service includes replicating the data of all the caches in the service. These services are all peer-to-peer, which means that a cluster node typically plays both the client and the server role through the service; furthermore, all of these services will failover in the event of cluster node failure without any data loss.
Installing Tangosol Coherence
Downloading and Extracting Coherence
For Windows and any other OS supporting the .zip format, Coherence is downloadable as a .zip file; for Unix and Linux, Coherence is also downloadable as a .tar.gz file. If prompted by your browser, choose to save the downloaded file. Once it has completed downloading, expand the .zip file (using WinZip or the unzip command-line utility) or the .tar.gz file (using gunzip and tar) to the location of your choice. On Windows, you can expand it to your c:\ directory; on Unix, it is suggested that you expand it to the /opt directory. Expanding the .zip or .tar.gz will create a tangosol directory with several sub-directories.
To expand the .tar.gz file on Unix, go to the directory into which you want to install, such as /opt, and issue the following command, supplying the path to the .tar.gz file if necessary:
If you expand the .zip file on Unix, you must mark the .sh file(s) in the tangosol/bin directory as executable using the chmod command. (If you expand the .tar.gz file on Unix, the .sh file(s) will automatically be marked as executable when they are untarred.)
Installing Coherence
If you are adding Coherence to an application server, you will need to make sure that tangosol.jar and coherence.jar libraries (found in tangosol/lib/) are in the CLASSPATH (or the equivalent mechanism that your application server uses).
Alternatively, if your application server supports it, you can package the tangosol.jar and coherence.jar libraries into your application's .ear, .jar or .war file.
For purposes of compilation, you will need to make sure that tangosol.jar and coherence.jar libraries are in the CLASSPATH (or the equivalent mechanism that your compiler or IDE uses).
Verifying that multiple nodes and servers are able to form a cluster
Coherence includes a self-contained console application that can be used to verify that installation is successful and that all the servers that are meant to participate in the cluster are indeed capable of joining the cluster. We recommend that you perform this quick test when you first start using Coherence in a particular network and server environment, to verify that the nodes do indeed connect as expected. You can do that by repeating the following set of steps to start Coherence on each server (you can start multiple instances of Coherence on the same server as well):
- Change the current directory to the Tangosol library directory (%TANGOSOL_HOME%\lib on Windows and $TANGOSOL_HOME/lib on Unix).
- Make sure that the paths are configured so that the Java command will run.
- Run the following command to start Coherence command line:
This is what you should see after you start the first member:
c:\tangosol\lib>java -jar coherence.jar
******************************************************************************
*
* Tangosol Coherence(tm): Enterprise Edition is licensed by Tangosol, Inc.
* License details are available at: http://www.tangosol.com/license.jsp
*
* Licensed for evaluation use with the following restrictions:
*
* Effective Date : 1 Jun 2005 00:00:00 GMT
* Termination Date : 1 Sep 2005 00:00:00 GMT
*
* A production license is required for production use.
*
* Copyright (c) 2000-2005 Tangosol, Inc.
*
******************************************************************************
Tangosol Coherence Version 3.0/315
SafeCluster: Group{Address=224.3.0.0, Port=30315, TTL=4}
MasterMemberSet
(
ThisMember=Member(Id=1, Timestamp=Mon Jun 27 09:28:08 EDT 2005, Address=192.168.0.247, Port=8088, MachineId=26871)
OldestMember=Member(Id=1, Timestamp=Mon Jun 27 09:28:08 EDT 2005, Address=192.168.0.247, Port=8088, MachineId=26871)
ActualMemberSet=MemberSet(Size=1, BitSetCount=2
Member(Id=1, Timestamp=Mon Jun 27 09:28:08 EDT 2005, Address=192.168.0.247, Port=8088, MachineId=26871)
)
RecycleMillis=240000
RecycleSet=MemberSet(Size=0, BitSetCount=0
)
)
TcpRing{TcpSocketAccepter{State=STATE_OPEN, ServerSocket=192.168.0.247:8088}, Connections=}
ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_JOINED), Id=0, Version=3.0, OldestMemberId=1}
Map (?):
As you can see there is only one member listed in the ActualMemberSet. When the second member is started, you should see something similar to the following at its start up:
c:\tangosol\lib>java -jar coherence.jar
******************************************************************************
*
* Tangosol Coherence(tm): Enterprise Edition is licensed by Tangosol, Inc.
* License details are available at: http://www.tangosol.com/license.jsp
*
* Licensed for evaluation use with the following restrictions:
*
* Effective Date : 1 Jun 2005 00:00:00 GMT
* Termination Date : 1 Sep 2005 00:00:00 GMT
*
* A production license is required for production use.
*
* Copyright (c) 2000-2005 Tangosol, Inc.
*
******************************************************************************
Tangosol Coherence Version 3.0/315
SafeCluster: Group{Address=224.3.0.0, Port=30315, TTL=4}
MasterMemberSet
(
ThisMember=Member(Id=2, Timestamp=Mon Jun 27 09:35:51 EDT 2005, Address=192.168.0.247, Port=8089, MachineId=26871)
OldestMember=Member(Id=1, Timestamp=Mon Jun 27 09:29:10 EDT 2005, Address=192.168.0.247, Port=8088, MachineId=26871)
ActualMemberSet=MemberSet(Size=2, BitSetCount=2
Member(Id=1, Timestamp=Mon Jun 27 09:29:10 EDT 2005, Address=192.168.0.247, Port=8088, MachineId=26871)
Member(Id=2, Timestamp=Mon Jun 27 09:35:51 EDT 2005, Address=192.168.0.247, Port=8089, MachineId=26871)
)
RecycleMillis=240000
RecycleSet=MemberSet(Size=0, BitSetCount=0
)
)
TcpRing{TcpSocketAccepter{State=STATE_OPEN, ServerSocket=192.168.0.247:8089}, Connections=}
ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_JOINED), Id=0, Version=3.0, OldestMemberId=1}
Map (?):
If you execute the who command at the Map(?): prompt of the first member after the second member is started, you should see the same two members:
Map (?): who
SafeCluster: Group{Address=224.3.0.0, Port=30315, TTL=4}
MasterMemberSet
(
ThisMember=Member(Id=2, Timestamp=Mon Jun 27 09:35:51 EDT 2005, Address=192.168.0.247, Port=8089, MachineId=26871)
OldestMember=Member(Id=1, Timestamp=Mon Jun 27 09:29:10 EDT 2005, Address=192.168.0.247, Port=8088, MachineId=26871)
ActualMemberSet=MemberSet(Size=2, BitSetCount=2
Member(Id=1, Timestamp=Mon Jun 27 09:29:10 EDT 2005, Address=192.168.0.247, Port=8088, MachineId=26871)
Member(Id=2, Timestamp=Mon Jun 27 09:35:51 EDT 2005, Address=192.168.0.247, Port=8089, MachineId=26871)
)
RecycleMillis=240000
RecycleSet=MemberSet(Size=0, BitSetCount=0
)
)
TcpRing{TcpSocketAccepter{State=STATE_OPEN, ServerSocket=192.168.0.247:8089}, Connections=[1]}
ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_JOINED), Id=0, Version=3.0, OldestMemberId=1}
Map (?):
As more new members are started, you should see their addition reflected in the ActualMemberSet list. If you do not see new members being added, your network may not be properly configured for multicast traffic, you may used the Multicast Test to validate this.
Installing Coherence*Web Session Management Module
| Applies to Coherence 3.0 or later
Please note that the following installation documentation applies to the Coherence*Web module in Coherence release 3.0 or later. This Session Management Module is very different and much more comprehensive than the module provided with Coherence releases prior to Release 2.3. Since we chose to not include the previous version's module jars and documentation in release 2.3 to avoid confusion, if you are looking for information on how to install pre-Release 2.3 module, please refer to the documentation and the User Guide included with the doc directory of the software distribution for the release level you are installing. |
Coherence*Web Session Management Module: Supported Web Containers
The following table summarizes the web containers that are currently supported by the Coherence*Web Session Management Module and the installation information specific to each supported web container. For detailed installation instructions for a particular web container, click on its name.
Notes:
1 The name of the Java utility class used to patch web container libraries so that Coherence*Web can tightly integrate with the target web container.
2 The server type alias passed to the Coherence*Web installer via the -server command line option.
General Instructions for Installing Coherence*Web Session Management Module
To enable Coherence*Web in your J2EE application, you need to run a ready to deploy application (recommended) through the automated installer prior to deploying it. The automated installer prepares the application for deployment.
To install Coherence*Web for the J2EE application you are deploying:
- Make sure that the application directory, .ear file or .war file are not being used or accessed by another process.
- Change the current directory to the Tangosol library directory (%TANGOSOL_HOME%\lib on Windows and $TANGOSOL_HOME/lib on Unix).
- Make sure that the paths are configured so that the Java command will run.
- Go through the application inspection step by running the following command and specifying the full path to your application and the name of your server found in the chart above (replacing the <app-path> and <server-type> with them in the command line below):
A successful result of this step is the creation (or an update, if it already exists) of the coherence-web.xml configuration descriptor file for your J2EE application in the directory where the application is located. This configuration descriptor contains the default Coherence*Web settings for your application that the installer suggests be used in the following install step. You may at this point proceed to the install step, or review and modify the settings to fit them to your requirements prior to running the install step (which would make the install step use your modified settings). For example, you can enable certain features by setting the "context-param" options in the coherence-web.xml configuration descriptor:
- Go through the Coherence*Web application installation step by running the following command and specifying the full path to your application (replacing the <app-path> with it in the command line below):
Please note that the installer expects to find the valid coherence-web.xml configuration descriptor for its use in the same directory the application is located.
- Deploy the updated application and verify that everything functions as expected, using the load balancer if necessary. Please remember that the load balancer is only intended for testing and should not be used in a production environment.
Installing Coherence*Web Session Management Module on BEATM
WebLogicTM
8.x
The following are additional steps to take when installing the Coherence*Web Session Management Module into a BEA WebLogic 8.x server:
Installing Coherence*Web Session Management Module on BEATM
WebLogicTM
Portal 8.x
The following are additional steps to take when installing the Coherence*Web Session Management Module into a BEA WebLogic Portal 8.x server:
Installing Coherence*Web Session Management Module on Caucho Resin®
3.0.x
The following are additional steps to take when installing the Coherence*Web Session Management Module into a Caucho Resin server:
Installing Coherence*Web Session Management Module on Oracle®
OC4J 10.1.x
The following are additional steps to take when installing the Coherence*Web Session Management Module into a Oracle OC4J server:
How the Coherence*Web Installer instruments a J2EE application
During the inspect step, the Coherence*Web Installer performs the following tasks:
- Generate a template coherence-web.xml configuration file that contains basic information about the application and target web container along with a set of default Coherence*Web configuration context parameters appropriate for the target web container. If an existing coherence-web.xml configuration file exists (for example, from a previous run of the Coherence*Web Installer), the context parameters in the existing file are merged with those in the generated template.
- Enumerate the JSPs from each web application in the target J2EE application and add information about each JSP to the coherence-web.xml configuration file.
- Enumerate the TLDs from each web application in the target J2EE application and add information about each TLD to the coherence-web.xml configuration file.
During the install step, the Coherence*Web Installer performs the following tasks:
- Create a backup of the original J2EE application so that it can be restored during the uninstall step.
- Add the Coherence*Web configuration context parameters generated in step (1) of the inspect step to the web.xml descriptor of each web application contained in the target J2EE application.
- Unregister any application-specific ServletContextListener, ServletContextAttributeListener, ServletRequestListener, ServletRequestAttributeListener, HttpSessionListener, and HttpSessionAttributeListener classes (including those registered by TLDs) from each web application.
- Register a Coherence*Web ServletContextListener in each web.xml descriptor. At runtime, the Coherence*Web ServletContextListener will propagate each ServletContextEvent to each application-specific ServletContextListener.
- Register a Coherence*Web ServletContextAttributeListener in each web.xml descriptor. At runtime, the Coherence*Web ServletContextAttributeListener will propagate each ServletContextAttributeEvent to each application-specific ServletContextAttributeListener.
- Wrap each application-specific Servlet declared in each web.xml descriptor with a Coherence*Web SessionServlet. At runtime, each Coherence*Web SessionServlet will delegate to the wrapped Servlet.
- Add the following directive to each JSP enumerated in step (2) of the inspect step: <%@ page extends="com.tangosol.coherence.servlet.api22.JspServlet" %>
During the uninstall step, the Coherence*Web Installer replaces the instrumented J2EE application with the backup of the original version created in step (1) of the install process.
Testing HTTP session management (without a dedicated loadbalancer)
Coherence comes with a light-weight software load balancer; it is only intended for testing purposes. The load balancer is very useful when testing functionality such as Session Management and is very easy to use.
- Start multiple application server processes, on one or more server machines, each running your application on a unique IP address and port combination.
- Open a command (or shell) window.
- Change the current directory to the Tangosol library directory (%TANGOSOL_HOME%\lib on Windows and $TANGOSOL_HOME/lib on Unix).
- Make sure that the paths are configured so that the Java command will run.
- Start the software load balancer with the following command lines (each of these command lines makes the application available on the default HTTP port, which is port 80):
To test load-balancing locally on one machine with two application server instances on ports 7001 and 7002:
To run the load-balancer locally on a machine named server1 that load balances to port 7001 on server1, server2 and server3:
Assuming the above command line, an application that previously was accessed with the URL http://server1:7001/my.jsp would now be accessed with the URL http://server1:80/my.jsp or just http://server1/my.jsp.
The following command line options are supported:
| -backlog |
Sets the TCP/ IP accept backlog option to the specified value, for example:
-backlog=64 |
| -threads |
Uses the specified number of request/ response thread pairs (so the total number of additional daemon threads will be two times the specified value), for example:
-threads=64 |
| -roundrobin |
Specifies the use of a round-robin load-balancing algorithm |
| -random |
Specifies the use of a random load-balancing algorithm (default) |
Make sure that your application uses only relative re-directs or the address or the load-balancer.
Using the Coherence*Web Installer Ant Task
Description
The Coherence*Web Installer Ant task allows you to run the Coherence*Web Installer from within your existing Ant build files. To use the Coherence*Web Installer Ant task, add the following task import statement to your Ant build file:
<taskdef name="cwi" classname="com.tangosol.coherence.misc.CoherenceWebAntTask">
<classpath>
<pathelement location="${tangosol.home}/lib/webInstaller.jar"/>
</classpath>
</taskdef>
where ${tangosol.home} refers to the root directory of your Coherence installation.
The basic process of installing Coherence*Web into a J2EE application from an Ant build is as follows:
- Build your J2EE application as you normally would
- Run the Coherence*Web Ant task with the operations attribute set to inspect
- Make any necessary changes to the generated Coherence*Web XML descriptor
- Run the Coherence*Web Ant task with the operations attribute set to install
If you are performing iterative development on your application (modifying JSPs, Servlets, static resources, etc.), the installation process would consist of the following steps:
- Run the Coherence*Web Ant task with the operations attribute set to uninstall, the failonerror attribute set to false, and the descriptor attribute set to the location of the previously generated Coherence*Web XML descriptor (from step 2 above)
- Build your J2EE application as you normally would
- Run the Coherence*Web Ant task with the operations attribute set to inspect, install and the descriptor attribute set to the location of the previously generated Coherence*Web XML descriptor (from step 2 above)
If you want to change the Coherence*Web configuration settings of a J2EE application that already has Coherence*Web installed:
- Run the Coherence*Web Ant task with the operations attribute set to uninstall and the descriptor attribute set to the location of the Coherence*Web XML descriptor for the J2EE application.
- Change the necessary configuration parameters in the Coherence*Web XML descriptor.
- Run the Coherence*Web Ant task with the operations attribute set to install and the descriptor attribute set to the location of the modified Coherence*Web XML descriptor (from step 2).
Parameters
| Attribute |
Description |
Required |
| app |
Path to the target J2EE application. This can be a path to a WAR file, an EAR file, an exploded WAR directory, or an exploded EAR directory. |
true, if the operations attribute is set to any value other than version |
| backup |
Path to a directory that will hold a backup of the original target J2EE application. This attribute defaults to the directory that contains the J2EE application. |
false |
| descriptor |
Path to the Coherence*Web XML descriptor. This attribute defaults to coherence-web.xml in the directory that contains the target J2EE application. |
false |
| failonerror |
Stop the Ant build if the Coherence*Web installer exits with a status other than 0. The default is true. |
false |
| nowarn |
Suppress warning messages. This attribute can be either true or false. The default is false. |
false |
| operations |
comma- or space-separated list of operations to perform; each operation must be one of inspect, install, uninstall, or version. |
true |
| server |
The alias of the target J2EE application server. |
false |
| touch |
Touch JSPs and TLDs that are modified by the Coherence*Web installer. This attribute can be either true, false, or 'M/d/y h:mm a'. The default is false. |
false |
| verbose |
Show verbose output. This attribute can be either true or false. The default is false. |
false |
Examples
Inspect the myWebApp.war web application and generate a Coherence*Web XML descriptor called my-coherence-web.xml in the current working directory:
<cwi app="myWebApp.war" operations="inspect" descriptor="my-coherence-web.xml"/>
Install Coherence*Web into the myWebApp.war web application using the Coherence*Web XML descriptor called my-coherence-web.xml found in the current working directory:
<cwi app="myWebApp.war" operations="install" descriptor="my-coherence-web.xml"/>
Uninstall Coherence*Web from the myWebApp.war web application:
<cwi app="myWebApp.war" operations="uninstall">
Install Coherence*Web into the myWebApp.war web application located in the /dev/myWebApp/build directory using the Coherence*Web XML descriptor called my-coherence-web.xml found in the /dev/myWebApp/src directory, and place a backup of the original web application in the /dev/myWebApp/work directory:
<cwi app="/dev/myWebApp/build/myWebApp.war" operations="install" descriptor="/dev/myWebApp/src/my-coherence-web.xml" backup="/dev/myWebApp/work"/>
Install Coherence*Web into the myWebApp.war web application located in the /dev/myWebApp/build directory using the Coherence*Web XML descriptor called coherence-web.xml found in the /dev/myWebApp/build directory. If the web application has not already been inspected (i.e. /dev/myWebApp/build/coherence-web.xml does not exists), inspect the web application prior to installing Coherence*Web:
<cwi app="/dev/myWebApp/build/myWebApp.war" operations="inspect,install"/>
Reinstall Coherence*Web into the myWebApp.war web application located in the /dev/myWebApp/build directory using the Coherence*Web XML descriptor called my-coherence-web.xml found in the /dev/myWebApp/src directory:
<cwi app="/dev/myWebApp/build/myWebApp.war" operations="uninstall,install" descriptor="/dev/myWebApp/src/my-coherence-web.xml"/>
Types of Caches in Coherence
Overview
The following is an overview of the types of caches offered by Coherence. More detail is provided in later sections.
Replicated
Data is fully replicated to every member in the cluster. Offers the fastest read performance. Clustered, fault-tolerant cache with linear performance scalability for reads, but poor scalability for writes (as writes must be processed by every member in the cluster). Because data is replicated to all machines, adding servers does not increase aggregate cache capacity.
Optimistic
OptimisticCache is a clustered cache implementation similar to the ReplicatedCache implementation, but without any concurrency control. This implementation has the highest possible throughput. It also allows to use an alternative underlying store for the cached data (for example, a MRU/MFU-based cache). However, if two cluster members are independently pruning or purging the underlying local stores, it is possible that a cluster member may have a different store content than that held by another cluster member.
Distributed (Partitioned)
Clustered, fault-tolerant cache with linear scalability. Data is partitioned among all the machines of the cluster. For fault-tolerance, partitioned caches can be configured to keep each piece of data on one, two or more unique machines within a cluster.
Near
A hybrid cache; fronts a fault-tolerant, scalable partitioned cache with a local cache. Near cache invalidates front cache entries, using configurable invalidation strategy, and provides excellent performance and synchronization. Near cache backed by a partitioned cache offers zero-millisecond local access for repeat data access, while enabling concurrency and ensuring coherency and fail-over, effectively combining the best attributes of replicated and partitioned caches.
Though rarely needed, NearCache can be configured to work with any type of back-end cache, not just Partitioned.
VersionedNearCache
An extended version of NearCache, that provides for the ability to verify the version of the object in the cache. With improvemenets in NearCache capabilities introduced in Coherence release 2.3, we suggest that you use NearCache instead of VersionedNearCache, but legacy Coherence applications use VersionedNearCache to ensure coherence through object versioning instead of the reliable and efficient front cache invalidation (which was not available prior to release 2.3).
Summary of Cache Types
Numerical Terms:
JVMs = number of JVMs
DataSize = total size of cached data (measured without redundancy)
Redundancy = number of copies of data maintained
LocalCache = size of local cache (for near caches)
| |
Replicated Cache |
Optimistic Cache |
Partitioned Cache |
Near Cache backed by partitioned cache |
VersionedNearCache backed by partitioned cache |
LocalCache not clustered |
| Topology |
Replicated |
Replicated |
Partitioned Cache |
Local Caches + Partitioned Cache |
Local Caches + Partitioned Cache |
Local Cache |
| Fault Tolerance |
Extremely High |
Extremely High |
Configurable 4
Zero to Extremely High |
Configurable 4
Zero to Extremely High |
Configurable 4
Zero to Extremely High |
Zero |
| Read Performance |
Instant 5 |
Instant 5 |
Locally cached: instant 5
Remote: network speed 1 |
Locally cached: instant 5
Remote: network speed 1 |
Locally cached: instant 5
Remote: network speed 1 |
Instant 5 |
| Write Performance |
Fast 2 |
Fast 2 |
Extremely fast 3 |
Extremely fast 3 |
Extremely fast 3 |
Instant 5 |
| Memory Usage (Per JVM) |
DataSize |
DataSize |
DataSize/JVMs x Redundancy |
LocalCache + [DataSize / JVMs] |
LocalCache +
[DataSize/JVMs] |
DataSize |
| Memory Usage (Total) |
JVMs x DataSize |
JVMs x DataSize |
Redundancy x DataSize |
[Redundancy x DataSize] +
[JVMs x LocalCache] |
[Redundancy x DataSize] + [JVMs x LocalCache] |
n/a |
| Coherency |
fully coherent |
fully coherent |
fully coherent |
fully coherent 6 |
fully coherent |
n/a |
| Locking |
fully transactional |
none |
fully transactional |
fully transactional |
fully transactional |
fully transactional |
| Typical Uses |
Metadata |
n/a (see Near Cache) |
Read-write caches |
Read-heavy caches w/ access affinity |
n/a (see Near Cache) |
Local data |
Notes:
1 As a rough estimate, with 100mbit ethernet, network reads typically require ~20ms for a 100KB object. With gigabit ethernet, network reads for 1KB objects are typically sub-millisecond.
2 Requires UDP multicast or a few UDP unicast operations, depending on JVM count.
3 Requires a few UDP unicast operations, depending on level of redundancy.
4 Partitioned caches can be configured with as many levels of backup as desired, or zero if desired. Most installations use one backup copy (two copies total).
5 Limited by local CPU/memory performance, with negligible processing required (typically sub-millisecond performance).
6 Listener-based Near caches are coherent; expiry-based near caches are partially coherent for non-transactional reads and coherent for transactional access.
Cache Semantics
Overview
Coherence caches are used to cache value objects. These objects may represent data from any source, either internal (session data, transient data, etc...) or external (database, mainframe, etc...).
Objects placed in the cache must be capable of being serialized. The simplest approach to doing this is to implement java.io.Serializable. For higher performance, Coherence also supports the java.io.Externalizable and (even faster) com.tangosol.io.ExternalizableLite interfaces. The primary difference between Externalizable and {{ExternalizableLite }}is the I/O object used. In most cases, porting from one to the other is a trivial exercise.
Any objects that implement com.tangosol.run.xml.XmlBean will automatically support ExternalizableLite. For more details, see the API JavaDoc for com.tangosol.run.xml.XmlBean.
As a reminder, when serializing an object, Java serialization automatically crawls every object visible (via object references, including collections like Map and List). As a result, cached objects should not refer to their parent objects directly (holding onto an identifying value like an integer is okay). Of course, objects that implement their own serialization routines do not need to worry about this.
Creating and Using Coherence Caches
Overview
The simplest and most flexible way to create caches in Coherence is to use the cache configuration descriptor to define attributes and names for your application's or cluster's caches, and to instantiate the caches in your application code referring to them by name that matches the names or patterns as defined in the descriptor.
This approach to configuring and using Coherence caches has a number of very important benefits. It separates the cache initialization and access logic for the cache in your application from its attributes and characteristics. This way your code is written in a way that is independent of the cache type that will be utilized in your application deployment and changing the characteristics of each cache (such as cache type, cache eviction policy, and cache type-specific attributes, etc.) can be done without making any changes to the code whatsoever. It allows you to create multiple configurations for the same set of named caches and to instruct your application to use the appropriate configuration at deployment time by specifying the descriptor to use in the java command line when the node JVM is started.
Creating a cache in your application.
To instantiate a cache in your application code, you need to:
- Make sure that coherence.jar and tangosol.jar are in your classpath.
- Use CacheFactory.getCache() to access the cache in your code.
Your code will look similar to the following:
import com.tangosol.net.CacheFactory;
import com.tangosol.net.NamedCache;
...
NamedCache cache = CacheFactory.getCache("VirtualCache");
Now you can retrieve and store objects in the cache, using the NamedCache API, which extends the standard java.util.Map interface, adding a number of additional capabilities that provide concurrency control (ConcurrentMap interface), ability to listen for cache changes (ObservableMap interface) and ability to query the cache (QueryMap interface).
The following is an example of typical cache operations:
String key = "key";
MyValue value = (MyValue) cache.get(key);
cache.put(key, value);
Configuring the caches
The cache attributes and settings are defined in the cache configuration descriptor. Cache attributes determine the cache type (what means and resources the cache will use for storing, distributing and synchronizing the cached data) and cache policies (what happens to the objects in the cache based on cache size, object longevity and other parameters).
The structure of the cache configuration descriptor (described in detail by the cache-config.dtd included in the coherence.jar) consists of two primary sections: caching-schemes section and caching-scheme-mapping section.
The caching-schemes section is where the attributes of a cache or a set of caches get defined. The caching schemes can be of a number of types, each with its own set of attributes. The caching schemes can be defined completely from scratch, or can incorporate attributes of other existing caching schemes, referring to them by their scheme-names (using a scheme-ref element) and optionally overriding some of their attributes to create new caching schemes. This flexibility enables you to create caching scheme structures that are easy to maintain, foster reuse and are very flexible.
The caching-scheme-mapping section is where the specific cache name or a naming pattern is attached to the cache scheme that defines the cache configuration to use for the cache that matches the name or the naming pattern.
So if we would like to define the cache descriptor for the cache we mentioned in the previous section (VirtualCache), it may look something like the following:
<?xml version="1.0"?>
<!DOCTYPE cache-config SYSTEM "cache-config.dtd">
<cache-config>
<caching-scheme-mapping>
<!--
Caches with any name will be created as default replicated.
-->
<cache-mapping>
<cache-name>*</cache-name>
<scheme-name>default-replicated</scheme-name>
</cache-mapping>
</caching-scheme-mapping>
<caching-schemes>
<!--
Default Replicated caching scheme.
-->
<replicated-scheme>
<scheme-name>default-replicated</scheme-name>
<service-name>ReplicatedCache</service-name>
<backing-map-scheme>
<class-scheme>
<scheme-ref>default-backing-map</scheme-ref>
</class-scheme>
</backing-map-scheme>
</replicated-scheme>
<!--
Default backing map scheme definition used by all
The caches that do not require any eviction policies
-->
<class-scheme>
<scheme-name>default-backing-map</scheme-name>
<class-name>com.tangosol.util.SafeHashMap</class-name>
</class-scheme>
</caching-schemes>
</cache-config>
The above cache configuration descriptor specifies that all caches will be created (including our VirtualCache cache) utilizing the default-replicated caching scheme. It defines the default-replicated caching scheme as a replicated-scheme, utilizing a service named ReplicatedCache and utilizing the backing map named default-backing-map, which is defined as a class com.tangosol.util.SafeHashMap (the default backing map storage that Coherence uses when no eviction policies are required).
Then, at a later point, let's say we decide that, since the number of entries that our cache is holding is too large and updates to the objects too frequent to use a replicated cache, we want our VirtualCache cache to become a distributed cache instead (while keeping all other caches replicated). To accommodate these new circumstances, we can change the cache configuration by adding the following cache-scheme definition for the distributed cache to the caching-schemes section:
<!--
Default Distributed caching scheme.
-->
<distributed-scheme>
<scheme-name>default-distributed</scheme-name>
<service-name>DistributedCache</service-name>
<backing-map-scheme>
<class-scheme>
<scheme-ref>default-backing-map</scheme-ref>
</class-scheme>
</backing-map-scheme>
</distributed-scheme>
and then mapping the VirtualCache cache to it in the caching-schemes-mapping section:
<cache-mapping>
<cache-name>VirtualCache</cache-name>
<scheme-name>default-distributed</scheme-name>
</cache-mapping>
The resulting cache definition descriptor will look as follows:
<?xml version="1.0"?>
<!DOCTYPE cache-config SYSTEM "cache-config.dtd">
<cache-config>
<caching-scheme-mapping>
<!--
Caches with any name will be created as default replicated.
-->
<cache-mapping>
<cache-name>*</cache-name>
<scheme-name>default-replicated</scheme-name>
</cache-mapping>
<cache-mapping>
<cache-name>VirtualCache</cache-name>
<scheme-name>default-distributed</scheme-name>
</cache-mapping>
</caching-scheme-mapping>
<caching-schemes>
<!--
Default Replicated caching scheme.
-->
<replicated-scheme>
<scheme-name>default-replicated</scheme-name>
<service-name>ReplicatedCache</service-name>
<backing-map-scheme>
<class-scheme>
<scheme-ref>default-backing-map</scheme-ref>
</class-scheme>
</backing-map-scheme>
</replicated-scheme>
<!--
Default Distributed caching scheme.
-->
<distributed-scheme>
<scheme-name>default-distributed</scheme-name>
<service-name>DistributedCache</service-name>
<backing-map-scheme>
<class-scheme>
<scheme-ref>default-backing-map</scheme-ref>
</class-scheme>
</backing-map-scheme>
</distributed-scheme>
<!--
Default backing map scheme definition used by all
The caches that do not require any eviction policies
-->
<class-scheme>
<scheme-name>default-backing-map</scheme-name>
<class-name>com.tangosol.util.SafeHashMap</class-name>
</class-scheme>
</caching-schemes>
</cache-config>
Once we revise and deploy the descriptor and restart the cluster, the VirtualCache cache will be a distributed cache instead of replicated, all without any changes to the code we wrote.
Cache Configuration Descriptor location
A few words about how to instruct Coherence where to find the cache configuration descriptor. Without specifying anything in the command java command line, Coherence will attempt to use the cache configuration descriptor named coherence-cache-config.xml that it finds in the classpath. Since Coherence ships with this file packaged into the coherence.jar, unless you place another file with the same name in the classpath location preceding coherence.jar, that is the one that Coherence will use. You can tell Coherence to use a different default descriptor by using the -Dtangosol.coherence.cacheconfig java command line property as follows:
The above command instructs Coherence to use my-config.xml file in /cfg directory as the default cache configuration descriptor. As you can see, this capability can give you the flexibility to modify the cache configurations of your applications without making any changes to the application code and by simply specifying different cache configuration descriptors at application deployment or start-up.
Putting it all together: your first Coherence cache example
Let's try walking through creating a working example cache using the caches and the cache configuration descriptor we described in the previous section. The easiest way to initially do that is to use the Coherence command line application. A couple of general comments regarding this example before we get started:
- In the examples we refer to the 'nodes' or 'JVMs'. We make no assumption as to where they will run - you can run all of them on the same machine multiple machines or a combination of multiple nodes per machine and multiple machines. To see the clustered cache in action you will need at least 2 nodes to see the JVMs sharing data (all the following examples were captured with 2 JVMs on a single machine).
- This example uses Windows conventions and commands but it will work equally well in any of the Unix environments (with the appropriate adjustments for the Unix commands and conventions) and we encourage you to try it on multiple machines with different operating systems, as this is the way Coherence is designed to function: on multiple platforms simultaneously.
Setting up your test environment
To set up the test environment, you will need install Coherence by unzipping the software distribution in the desired location on one or more machines.
The tangosol/examples directory of the software contains the following examples that we will be making use of in this exercise:
- examples/config/explore-config.xml is the configuration descriptor we will use.
- examples/java/com/tangosol/examples/explore/SimpleCacheExplorer.java is the java class that demonstrates how you can access the cache from a command line.
To deploy and run it, you need to execute the following java command line (from the tangosol directory):
- In Windows:
- In Unix:
You should see something like the following when you bring it up:
T:\tangosol>java -cp ./lib/coherence.jar;./lib/tangosol.jar;./examples/java
-Dtangosol.coherence.cacheconfig=./examples/config/explore-config.xml
com.tangosol.examples.explore.SimpleCacheExplorer
******************************************************************************
*
* Tangosol Coherence(tm): Enterprise Edition is licensed by Tangosol, Inc.
* License details are available at: http://www.tangosol.com/license.jsp
*
* Licensed for evaluation use with the following restrictions:
*
* Effective Date : 1 Jun 2005 00:00:00 GMT
* Termination Date : 1 Sep 2005 00:00:00 GMT
*
* A production license is required for production use.
*
* Copyright (c) 2000-2005 Tangosol, Inc.
*
******************************************************************************
Tangosol Coherence Version 3.0/315
Command:
Typing in 'help' at the command prompt will show you the commands you can try:
Command: help
clear
get
keys
info
put
quit
remove
Command:
Typing in 'info' will show you the configuration and the other member information (please note that in the following example there are 2 cluster members active):
Command: info
>> VirtualCache cache is using a cache-scheme named 'default-replicated' defined as:
default-replicated
ReplicatedCache
default-backing-map
>> The following member nodes are currently active:
Member(Id=1, Timestamp=Mon Jun 27 09:49:00 EDT 2005, Address=192.168.0.247, Port=8088, MachineId=26871)
Member(Id=2, Timestamp=Mon Jun 27 09:49:07 EDT 2005, Address=192.168.0.247, Port=8089, MachineId=26871) <-- this node
Command:
You can also put a value into the cache:
Command: put 1 One
>> Put Complete
Command:
And retrieve a value from the cache:
Command: get 1
>> Value is One
Command:
Try these commands from multiple sessions and see the results.
The examples/jsp/explore/SimpleCacheExplorer.jsp is the JSP file that can be used with your favorite application server:
- To deploy and run it, you will need to deploy the JSP to the default web applications directory of your application server (along with the contents of the examples/jsp/images directory), modify the server start-up script to make sure that the classpath includes tangosol.jar and coherence.jar, and specify the location of the cache configuration file on the Java command line using the -Dtangosol.coherence.cacheconfig option (e.g. -Dtangosol.coherence.cacheconfig=$TANGOSOL_HOME/examples/config/explore-config.xml).
- You can then start one or more instances of the application server (on different machines or different ports) and access the SimpleCacheExplorer.jsp from the browser. You should see something like the following when you bring it up:
As with the command line application please try adding, updating and removing entries from multiple instances of the application server. Also please notice the information about the cache configuration and cluster membership at the bottom of the page. As cluster members are added and removed, this information will change.
Modifying the cache configuration
Once you are comfortable with the test setup, let's change the cache configuration and test our changes, using this simple test harness. Please remember that after each cache configuration change all the cluster members need to be shut down and then restarted (whether you are using application server instances or just plain java JVMs). All our test are configured to use tangosol/examples/config/explore-config.xml, so this the file that needs to be edited to make cache configuration changes.
Let's make the first change we described previously, changing the VirtualCache to be a distributed cache by adding the following (bolded) sections:
<?xml version="1.0"?>
<!DOCTYPE cache-config SYSTEM "cache-config.dtd">
<cache-config>
<caching-scheme-mapping>
<!--
Caches with any name will be created as default replicated.
-->
<cache-mapping>
<cache-name>*</cache-name>
<scheme-name>default-replicated</scheme-name>
</cache-mapping>
<cache-mapping>
<cache-name>VirtualCache</cache-name>
<scheme-name>default-distributed</scheme-name>
</cache-mapping>
</caching-scheme-mapping>
<caching-schemes>
<!--
Default Replicated caching scheme.
-->
<replicated-scheme>
<scheme-name>default-replicated</scheme-name>
<service-name>ReplicatedCache</service-name>
<backing-map-scheme>
<class-scheme>
<scheme-ref>default-backing-map</scheme-ref>
</class-scheme>
</backing-map-scheme>
</replicated-scheme>
<!--
Default Distributed caching scheme.
-->
<distributed-scheme>
<scheme-name>default-distributed</scheme-name>
<service-name>DistributedCache</service-name>
<backing-map-scheme>
<class-scheme>
<scheme-ref>default-backing-map</scheme-ref>
</class-scheme>
</backing-map-scheme>
</distributed-scheme>
<!--
Default backing map scheme definition used by all
The caches that do not require any eviction policies
-->
<class-scheme>
<scheme-name>default-backing-map</scheme-name>
<class-name>com.tangosol.util.SafeHashMap</class-name>
</class-scheme>
</caching-schemes>
</cache-config>
After the changes are saved, the test intances are restarted and you have had a chance to do some test data entry to see how the cache behaves, you should see the following in the cache configuration section of the tests:
- SimpleCacheExplorer.java:
Command: info
>> VirtualCache cache is using a cache-scheme named 'default-distributed' defined as:
default-distributed
DistributedCache
default-backing-map
>> The following member nodes are currently active:
Member(Id=1, Timestamp=Mon Jun 27 09:49:37 EDT 2005, Address=192.168.0.247, Port=8088, MachineId=26871)
Member(Id=2, Timestamp=Mon Jun 27 09:49:43 EDT 2005, Address=192.168.0.247, Port=8089, MachineId=26871) <-- this node
Command:
- SimpleCacheExplorer.jsp:
As you can see, our VirtualCache cache is now distributed according to the cache configuration descriptor.
Now let's add an eviction policy for our default distributed cache, limiting it's size to 5 entries (per node) and setting the entry expiry to 60 seconds with an LRU eviction policy. To do that we need to make the following (bolded) changes to our descriptor:
<?xml version="1.0"?>
<!DOCTYPE cache-config SYSTEM "cache-config.dtd">
<cache-config>
<caching-scheme-mapping>
<!--
Caches with any name will be created as default replicated.
-->
<cache-mapping>
<cache-name>*</cache-name>
<scheme-name>default-replicated</scheme-name>
</cache-mapping>
<cache-mapping>
<cache-name>VirtualCache</cache-name>
<scheme-name>default-distributed</scheme-name>
</cache-mapping>
</caching-scheme-mapping>
<caching-schemes>
<!--
Default Replicated caching scheme.
-->
<replicated-scheme>
<scheme-name>default-replicated</scheme-name>
<service-name>ReplicatedCache</service-name>
<backing-map-scheme>
<class-scheme>
<scheme-ref>default-backing-map</scheme-ref>
</class-scheme>
</backing-map-scheme>
</replicated-scheme>
<!--
Default Distributed caching scheme.
-->
<distributed-scheme>
<scheme-name>default-distributed</scheme-name>
<service-name>DistributedCache</service-name>
<backing-map-scheme>
<local-scheme>
<scheme-ref>default-eviction</scheme-ref>
<eviction-policy>LRU</eviction-policy>
<high-units>5</high-units>
<expiry-delay>60</expiry-delay>
</local-scheme>
</backing-map-scheme>
</distributed-scheme>
<!--
Default backing map scheme definition used by all
The caches that do not require any eviction policies
-->
<class-scheme>
<scheme-name>default-backing-map</scheme-name>
<class-name>com.tangosol.util.SafeHashMap</class-name>
</class-scheme>
<!--
Default eviction policy scheme.
-->
<local-scheme>
<scheme-name>default-eviction</scheme-name>
<eviction-policy>HYBRID</eviction-policy>
<high-units>0</high-units>
<expiry-delay>3600</expiry-delay>
</local-scheme>
</caching-schemes>
</cache-config>
Please note that we defined a general purpose local-scheme 'default-eviction' (with no size limit, 5 minute expiry and a HYBRID eviction policy) and then used it by reference (using scheme-ref) for our default-distributed scheme definition, overriding it's configuration settings to match our requirements.
After the changes are saved, the test intances are restarted and you have had a chance to do some test data entry to see how the cache behaves, you should see the following in the cache configuration section of the tests:
- SimpleCacheExplorer.java:
Command: info
>> VirtualCache cache is using a cache-scheme named 'default-distributed' defined as:
default-distributed
DistributedCache
default-eviction
LRU
5
60
>> The following member nodes are currently active:
Member(Id=1, Timestamp=Mon Jun 27 09:50:07 EDT 2005, Address=192.168.0.247, Port=8088, MachineId=26871)
Member(Id=2, Timestamp=Mon Jun 27 09:50:17 EDT 2005, Address=192.168.0.247, Port=8089, MachineId=26871) <-- this node
Command:
Try doing some puts and gets, carefully noting the time you last updated the specific entries. You should see that the number of entries does not exceed 5 entries per node (so if you have 2 nodes running the number of entries should not exceed 10, for 3 nodes - 15, and so on) and entries either expire after they have not been updated for 60 seconds, or when you add the 6th entry (with the least recently touched entries being 'evicted' from the cache first. (Hint: use the 'keys' command in the SimpleCacheExplorer.java to see the list of keys in the cache.)
These examples show you the general approach to modifying the cache configurations without making any code changes (as you no doubt noticed we did not touch our test application's code). Please refer to the cache-config.dtd, which can be found in the coherence.jar for full details on the available cache configuration descriptor settings and the explanation of their meaning and possible settings.
Configuring and Using Coherence*Extend-JMS
Overview
Coherence*Extend-JMS allows you to use Coherence caching from outside of a Coherence cluster, using your existing JMS infrastructure as the means to connect to the cluster. Coherence*Extend-JMS uses a JMS-based protocol to invoke cache operations on a remote cluster node, but the details of doing so are hidden behind a local interface. Coherence*Extend-JMS includes support for the CacheStore and NamedCache interfaces.
The client (non-clustered) portion of Coherence*Extend-JMS is configured using the <jms-scheme> caching scheme. The <jms-scheme> can be used directly, from within a <cachestore-scheme>, or as the <back-scheme> of a near cache.
The clustered portion of Coherence*Extend-JMS can either be deployed as part of a J2EE application using the included NamedCacheProxyBean Message-Driven EJB or run in a stand-alone JVM using the included AdapterFactory command line application.
To configure and use Coherence*Extend-JMS:
- Specify a <jms-scheme> cache scheme in your Coherence cache configuration deployment descriptor.
- Create a properties file for your JNDI provider.
- Configure and deploy a JMS QueueConnectionFactory, TopicConnectionFactory, Queue, and Topic.
- Configure the cluster-side Coherence*Extend-JMS proxy.
The <jms-scheme> Cache Scheme
The <jms-scheme> cache scheme allows a non-clustered application JVM to access cached data from a Coherence cluster using a JMS-based protocol.
Consider the following scenario: there are a number of nodes on a local subnet running in a Coherence cluster. Each node of the cluster is reacheable by both UDP unicast and multicast and takes part in caching application data and performing various cluster-related tasks. Assume that you have another machine that you would like to be able to retrieve or update the cached application data, but due to network topology limitations the machine cannot be part of the Coherence cluster. In this case, the <jms-scheme> cache configuration descriptor element can be leveraged to access clustered application data from outside the Coherence cluster.
The following jms-cache-config.xml cache configuration descriptor is an example of using the <jms-scheme> element:
<cache-config>
<caching-scheme-mapping>
<cache-mapping>
<cache-name>dist-jms-direct</cache-name>
<scheme-name>jms-direct</scheme-name>
</cache-mapping>
<cache-mapping>
<cache-name>dist-jms-local</cache-name>
<scheme-name>jms-local</scheme-name>
</cache-mapping>
<cache-mapping>
<cache-name>dist-jms-near</cache-name>
<scheme-name>jms-near</scheme-name>
</cache-mapping>
</caching-scheme-mapping>
<caching-schemes>
<jms-scheme>
<scheme-name>jms-direct</scheme-name>
<queue-connection-factory-name>jms/tangosol/ConnectionFactory</queue-connection-factory-name>
<topic-connection-factory-name>jms/tangosol/ConnectionFactory</topic-connection-factory-name>
<queue-name>jms/tangosol/Queue</queue-name>
<topic-name>jms/tangosol/Topic</topic-name>
<request-timeout>10</request-timeout>
</jms-scheme>
<local-scheme>
<scheme-name>jms-local</scheme-name>
<eviction-policy>HYBRID</eviction-policy>
<expiry-delay>30</expiry-delay>
<flush-delay>30</flush-delay>
<cachestore-scheme>
<jms-scheme>
<scheme-ref>jms-direct</scheme-ref>
</jms-scheme>
</cachestore-scheme>
</local-scheme>
<near-scheme>
<scheme-name>jms-near</scheme-name>
<front-scheme>
<local-scheme>
<high-units>100</high-units>
</local-scheme>
</front-scheme>
<back-scheme>
<jms-scheme>
<scheme-ref>jms-direct</scheme-ref>
</jms-scheme>
</back-scheme>
<invalidation-strategy>all</invalidation-strategy>
</near-scheme>
</caching-schemes>
</cache-config>
Assuming one or more Coherence*Extend-JMS proxies are running in the cluster (see below), start your Java application pointing to this cache configuration file using the following Java command:
The NamedCache returned by the CacheFactory.getCache(String sCacheName) method will uses JMS to communicate with the Coherence cluster to retrieve and update data from the clustered cache with the same name.
Note that unlike other cache configurations, the <jms-scheme> will not cause any Coherence clustered service to be started.
Specifying JNDI Properties for your JNDI Provider
Coherence*Extend-JMS uses JNDI to obtain references to all JMS resources. To specify the JNDI properties that Coherence*Extend-JMS uses to create a JNDI InitialContext, create a file called jndi.properties that contains your JNDI provider's configuration properties and add the directory that contains the file to your classpath.
For example, if you are using WebLogic Server as your JNDI provider, your jndi.properties file would look something like the following:
Configuring JMS Resources for the JMS Adapter
Coherence*Extend-JMS uses a JMS Queue and Topic to pass messages between the JMS stub (non-clustered node) and proxy (clustered node). Therefore, you must deploy an appropriately configured JMS QueueConnectionFactory, TopicConnectionFactory, Queue, and Topic. You must also be sure to register the JMS resources under the JNDI names that you specified in the <jms-scheme> cache scheme configuration.
For example, if you are using WebLogic Server as your JMS provider:
- If you haven't already done so, create and configure a JMS server.
- Create and configure a new JMS ConnectionFactory called TangosolConnectionFactory and register it under the JNDI name that you specified in your <jms-scheme> (e.g. jms/tangosol/ConnectionFactory). Be sure to add your server to the list of deployment targets for the JMS ConnectionFactory.
- Create and configure a new JMS Queue called TangosolQueue and register it under the JNDI name that you specified in your <jms-scheme> (e.g. jms/tangosol/Queue). Be sure to add your server to the list of deployment targets for the JMS Queue.
- Create and configure a new JMS Topic called TangosolTopic and register it under the JNDI name that you specified in your <jms-scheme> (e.g. jms/tangosol/Topic). Be sure to add your server to the list of deployment targets for the JMS Topic.
- Create and configure a new JMS Template called TangosolTemplate and specify it as the temporary template for your JMS server.
Starting a Coherence*Extend-JMS Proxy
The cluster-side portion of Coherence*Extend-JMS is called a JMS proxy. You can run the JMS proxy as part of a J2EE application using an included Message-Driven EJB or in one or more stand-alone JVMs. Coherence includes an example of running the JMS proxy in a stand-alone JVM that launches a DefaultCacheServer, but you can create your own JMS proxy application using the com.tangosol.net.jms.AdapterFactory class. See the AdapterFactory JavaDoc for additional information.
To deploy the JMS proxy as part of a J2EE application:
- Change the current directory to the Tangosol library directory (%TANGOSOL_HOME%\lib on Windows and $TANGOSOL_HOME/lib on Unix).
- Make sure that the paths are configured so that the Java Jar command will run.
- Unjar the coherence-jms.jar Message-Driven EJB to a temporary directory.
- Edit the ejb-jar.xml EJB deployment descriptor in the META-INF directory under the temporary directory in which you extracted coherence-jms.jar. The following environment entries must be configured appropriately:
<env-entry>
<env-entry-name>QueueConnectionFactory</env-entry-name>
<env-entry-type>java.lang.String</env-entry-type>
<env-entry-value>jms/tangosol/ConnectionFactory</env-entry-value>
</env-entry>
<env-entry>
<env-entry-name>TopicConnectionFactory</env-entry-name>
<env-entry-type>java.lang.String</env-entry-type>
<env-entry-value>jms/tangosol/ConnectionFactory</env-entry-value>
</env-entry>
<env-entry>
<env-entry-name>Queue</env-entry-name>
<env-entry-type>java.lang.String</env-entry-type>
<env-entry-value>jms/tangosol/Queue</env-entry-value>
</env-entry>
<env-entry>
<env-entry-name>Topic</env-entry-name>
<env-entry-type>java.lang.String</env-entry-type>
<env-entry-value>jms/tangosol/Topic</env-entry-value>
</env-entry>
<env-entry>
<env-entry-name>ClusterOwned</env-entry-name>
<env-entry-type>java.lang.Boolean</env-entry-type>
<env-entry-value>true</env-entry-value>
</env-entry>
The QueueConnectionFactory environment entry must be set to the JNDI name of the JMS QueueConnectionFactory that you configured for your JMS provider. This entry defaults to jms/tangosol/ConnectionFactory.
The TopicConnectionFactory environment entry must be set to the JNDI name of the JMS TopicConnectionFactory that you configured for your JMS provider. This entry defaults to jms/tangosol/ConnectionFactory.
The Queue environment entry must be set to the JNDI name of the JMS Queue that you configured for your JMS provider. This entry defaults to jms/tangosol/Queue. Note that the destination of this MDB must be the same Queue as specified by the Queue environment entry.
The Topic environment entry must be set to the JNDI name of the JMS Topic that you configured for your JMS provider. This entry defaults to jms/tangosol/Topic.
The ClusterOwned environment entry indicates whether or not the Coherence cluster should be shut down fully by the Message-Driven EJB when it shuts down. This entry defaults to true.
- Add your container-specific EJB deployment descriptor to the META-INF directory. A WebLogic Server deployment descriptor is included for illustrative purposes. Be sure to specify the JNDI name of the JMS Queue that you configured for your JMS provider as the destination JNDI name of the Message-Driven EJB.
- Jar the contents of the temporary directory into a new coherence-jms.jar EJB JAR file, run any container-specific EJB deployment tools, and deploy the Message-Driven EJB to your J2EE application server.
To run the example of launching a JMS proxy in a stand-alone JVM:
- Change the current directory to the Tangosol library directory (%TANGOSOL_HOME%\lib on Windows and $TANGOSOL_HOME/lib on Unix).
- Make sure that the paths are configured so that the Java command will run.
- Add the directory that contains your jndi.properties file to the classpath environment variable.
- Start the AdapterFactory example command line application. For example, on Windows you would run the following command (note that it is broken up into multiple lines here only for formatting purposes; this is a single command typed on one line):
On Unix:
Advanced Configuration
The following table summarizes the various Java System properties that can be used to override advanced Coherence*Extend-JMS settings:
| Property |
Description |
Default |
| com.tangosol.coherence.jms.ttl |
This property can be used to override the default JMS Message time-to-live in milliseconds. |
10000 (10 seconds) |
| com.tangosol.coherence.jms.readonly |
If set to true, all JMS NamedCache proxy instances running within the JVM will reject any request from a JMS NamedCache stub that may potentially modify the contents of the target NamedCache. |
false |
Managing Coherence using JMX
Overview
Coherence includes facilities for managing and monitoring Coherence resources via the Java Management Extensions (JMX) API. JMX is a Java standard for managing and monitoring Java applications and services. It defines a management architecture, design patterns, APIs, and services for building general solutions to manage Java-enabled resources. This section assumes familiarity with JMX terminology. If you are new to JMX a good place to start is with this article.
To manage Coherence using JMX:
- Add JMX libraries to the Coherence classpath (if necessary)
- Configure the Coherence Management Framework
- View and manipulate Coherence MBeans using a JMX client of your choice
| JMX support
Coherence Enterprise Edition supports clustered JMX, allowing access to JMX statistics for the entire cluster from any member. Coherence Clustered Edition provides only local JMX information. |
Adding JMX libraries to the Coherence classpath
If you would like to manage a Coherence cluster using JMX you will need to ensure that you have the necessary JMX 1.0 or later classes (javax.management.*) in the classpath of at least one Coherence cluster node, known as an MBeanServer host. The cluster nodes that are not MBeanServer hosts will be managed by the MBeanServer host(s) via the Coherence Invocation service.
All compliant J2SE 5.0 JREs and J2EE application servers supply a JMX 1.0 or later implementation; therefore, if the MBeanServer host node is running within a J2SE 5.0 JVM or J2EE application server, no additional actions are necessary. However, for standalone applications running within a pre-J2SE 5.0 JVM, you can download the necessary JMX libraries here and add them to the classpath.
Configuring the Coherence Management Framework
In the majority of cases, enabling JMX management can be done by simply setting a Java system property on all Coherence cluster nodes that are acting as MBeanServer hosts:
and the following Java system property on all cluster nodes:
Note that the use of dedicated JMX cluster members is a common pattern. This approach avoids loading JMX software into every single cluster member, while still providing fault-tolerance should a single JMX member run into issues.
In general, the Coherence Management Framework is configured by the management-configuration operational configuration element in the Coherence Operational Configuration deployment descriptor (tangosol-coherence.xml). The following sub-elements control the behavior of the Management Framework:
- domain-name
Specifies the name of the JMX domain used to register MBeans exposed by the Coherence Management Framework.
- managed-nodes
Specifies whether or not a cluster node's JVM has an in-process MBeanServer and if so, whether or not the node allows management of other nodes' managed objects. Valid values are none, local-only, remote-only and all. For example, if a node has an in-process MBeanServer and you'd like this node to manage other nodes' MBeans, set this attribute to all.
- allow-remote-management
Specifies whether or not this cluster node will register its MBeans in a remote MBeanServer(s).
- read-only
Specifies whether or not the MBeans exposed by this cluster node allow operations that modify run-time attributes.
For additional information on each of these attributes, please see the Operational Configuration Elements.
Accessing Coherence MBeans
Once you have configured the Coherence Management Framework and launched one or more Coherence cluster nodes (at least one being an MBeanServer host) you will be able to view and manipulate the Coherence MBeans registered by all cluster nodes using standard JMX API calls. See the JavaDoc for the Registry class for details on the various MBean types registered by Coherence clustered services.
Coherence ships with two examples that demonstrate accessing Coherence MBeans via JMX. The first uses the HttpAdapter that is shipped as part of the JMX reference implementation (jmxtools.jar). To run the example on a pre-J2SE 5.0 JVM, start the Coherence command line application using the following command on Windows (note that it is broken up into multiple lines here only for formatting purposes; this is a single command typed on one line):
On Unix:
Once the Coherence command line application has started, type "jmx 8082" and hit enter. This starts the HttpAdapter on http://localhost:8082 in the cluster node's JVM and makes the cluster node an MBeanServer host. You can now use the HttpAdapter web application to view and manipulate Coherence MBeans registered by all cluster nodes:
Alternatively, you can run this example with the Sun J2SE 5.0 JVM and use the JConsole utility included with the Sun J2SE 5.0 JDK to view and manipulate Coherence MBeans. To do so, start the Coherence command line application using the following command (note that it is broken up into multiple lines here only for formatting purposes; this is a single command typed on one line):
Once the Coherence command line application has started, launch the JConsole utility (located in the bin directory of the Sun J2SE 5.0 JDK distribution) and open a new connection to the JVM running the Coherence command line application:
The second example is a JSP page (JmxCacheExplorer.jsp) that displays basic information on each running Coherence cache using JMX API calls. You can find this example in the examples/jsp/explore directory under the root of your Coherence installation.
Additional JMX examples may be found on the Tangosol Forums.
Technical Overview
Clustering
Overview
Coherence is built on a fully clustered architecture. Since "clustered" is an overused term in the industry, it is worth stating exactly what it means to say that Coherence is clustered. Coherence is based on a peer-to-peer clustering protocol, using a conference room model, in which servers are capable of:
- Speaking to Everyone: When a party enters the conference room, it is able to speak to all other parties in a conference room.
- Listening: Each party present in the conference room can hear messages that are intended for everyone, as well as messages that are intended for that particular party. It is also possible that a message is not heard the first time, thus a message may need to be repeated until it is heard by its intended recipients.
- Discovery: Parties can only communicate by speaking and listening; there are no other senses. Using only these means, the parties must determine exactly who is in the conference room at any given time, and parties must detect when new parties enter the conference room.
- Working Groups and Private Conversations: Although a party can talk to everyone, once a party is introduced to the other parties in the conference room (i.e. once discovery has completed), the party can communicate directly to any set of parties, or directly to an individual party.
- Death Detection: Parties in the conference room must quickly detect when parties leave the conference room – or die.
Using the conference room model provides the following benefits:
- There is no configuration required to add members to a cluster. Subject to configurable security restrictions, any JVM running Coherence will automatically join the cluster and be able to access the caches and other services provided by the cluster. This includes J2EE application servers, Cache Servers, dedicated cache loader processes, or any other JVM that is running with the Coherence software. When a JVM joins the cluster, it is called a cluster node, or alternatively, a cluster member.
- Since all cluster members are known, it is possible to provide redundancy within the cluster, such that the death of any one JVM or server machine does not cause any data to be lost.
- Since the death or departure of a cluster member is automatically and quickly detected, failover occurs very rapidly, and more importantly, it occurs transparently, which means that the application does not have to do any extra work to handle failover.
- Since all cluster members are known, it is possible to load balance responsibilities across the cluster. Coherence does this automatically with its Distributed Cache Service, for example. Load balancing automatically occurs to respond to new members joining the cluster, or existing members leaving the cluster.
- Communication can be very well optimized, since some communication is multi-point in nature (e.g. messages for everyone), and some is between two members.
Two of the terms used here describe processing for failed servers:
- Failover: Failover refers to the ability of a server to assume the responsibilities of a failed server. For example, "When the server died, its processes failed over to the backup server."
- Failback: Failback is an extension to failover that allows a server to reclaim its responsibilities once it restarts. For example, "When the server came back up, the processes that it was running previously were failed back to it."
All of the Coherence clustered services, including cache services and grid services, provide automatic and transparent failover and failback. While these features are transparent to the application, it should be noted that the application can sign up for events to be notified of all comings and goings in the cluster.
Cache Topologies
Overview
Coherence supports many different cache topologies, but generally they fall into a few general categories. Because Coherence uses the TCMP clustering protocol, Coherence can supports each of these without compromise. However, there are inherent advantages of each topology, and trade-offs between them.
Topology as a concept refers to where the data physically resides and how it is accessed in a distributed environment. It is important to understand that, regardless of where the data physically resides, and which topology is being used, every cluster participant has the same logical view of the data, and uses the same exact API to access the data. Generally, this means that the topology can be tuned or even selected at deployment time.
- Peer-to-Peer: For most purposes, a peer-to-peer topology is the easiest to configure and offers very good performance. A peer-to-peer topology is one in which each server both provides and consumes the same clustered services. For example, in a cluster of J2EE application servers, if those servers are all managing and consuming data, and the data are shared via replicated and/or distributed caches, that would be a peer-to-peer topology. This topology spreads the load evenly over as many servers as possible, minimizing configuration details and offering the cache services the greatest overall amounts of memory and CPU resources.
- Centralized (Cache Servers): In order to centralize cache management to a cluster of servers, yet provide access to other servers, a centralized cache topology is used. This has several benefits, including the ability to reduce the resource requirements of the servers that utilize the cache by completely offloading cache management from those servers. Additionally, the cache servers (those that actually manage the cache data) can be hosted on machines explicitly configured for that purpose, and those machines do not require a J2EE application server or any other software other than a standard JVM. The cluster of cache servers (so designated by their storage-enabled attribute) provides a unified cache image, such that this model could almost be described as a cache client/cache server architecture, with the exception being that the server part of it is composed of a cluster of any number of actual servers. The obvious benefits of the clustered cache server architecture is the transparent failover and failback, and the ability to provision new servers to expand the caching resources, both in terms of processing power and in-memory cache sizes. This topology uses the Coherence Distributed Cache Service, which is a cluster-partitioned cache.
- Multi-Tier (n-tier): While the Centralized topology is a two-tier architecture, it is possible to extend this topology to three or more tiers, by having each tier be a client of the tier behind it, and coupling these tiers either over a clustered protocol (such as TCMP) or via JMS in cases where real-time cache coherency is not required and communication protocols are limited (such as production server tiers in which only certain protocols are permitted.)
- Hybrid (Near Caching): To accelerate cache accesses for Centralized and Multi-Tier topologies, Coherence supports a hybrid topology using a Near Cache technology. A Near Cache provides local cache access to recently- and/or often-used data, backed by a centralized or multi-tiered cache that is used to load-on-demand for local cache misses. Near Caches have configurable levels of cache coherency, from the most basic expiry-based caches and invalidation-based caches, up to advanced data-versioning caches that can provide guaranteed coherency. The result is a tunable balance between the preservation of local memory resources and the performance benefits of truly local caches.
The extent of the cluster and of its tiers is fully definable in the tangosol-coherence.xml configuration file. This includes the ability to lock down the set of servers that can access and manage the cache for security purposes. The selection of which topology to use is typically driven by the cache configuration file, which by default is named coherence-cache-config.xml; however, the topology can also be driven entirely by the Coherence programmatic API, if the developer so chooses.
Cluster Services Overview
Overview
Coherence functionality is based on the concept of cluster services. Each cluster node can participate in (which implies both the ability to provide and to consume) any number of named services. These named services may already exist, which is to say that they may already be running on one or more other cluster nodes, or a cluster node can register new named services. Each named service has a service name that uniquely identifies the service within the cluster, and a service type, which defines what the service can do. There are several service types that are supported by Coherence:
- Cluster Service: This service is automatically started when a cluster node needs to join the cluster; each cluster node always has exactly one service of this type running. This service is responsible for the detection of other cluster nodes, for detecting the failure (death) of a cluster node, and for registering the availability of other services in the cluster. In other words, the Cluster Service keeps track of the membership and services in the cluster.
- Distributed Cache Service: This is the distributed cache service, which allows cluster nodes to distribute (partition) data across the cluster so that each piece of data in the cache is managed (held) by only one cluster node. The Distributed Cache Service supports pessimistic locking. Additionally, to support failover without any data loss, the service can be configured so that each piece of data will be backed up by one or more other cluster nodes. Lastly, some cluster nodes can be configured to hold no data at all; this is useful, for example, to limit the Java heap size of an application server process, by setting the application server processes to not hold any distributed data, and by running additional cache server JVMs to provide the distributed cache storage.
- Invocation Service: This service provides clustered invocation and supports grid computing architectures. Using the Invocation Service, application code can invoke agents on any node in the cluster, or any group of nodes, or across the entire cluster. The agent invocations can be request/response, fire and forget, or an asynchronous user-definable model.
- Optimistic Cache Service: This is the optimistic-concurrency version of the Replicated Cache Service, which fully replicates all of its data to all cluster nodes, and employs an optimization similar to optimistic database locking in order to maintain coherency. Coherency refers to the fact that all servers will end up with the same "current" value, even if multiple updates occur at the same exact time from different servers. The Optimistic Cache Service does not support pessimistic locking, so in general it should only be used for caching "most recently known" values for read-only uses.
- Replicated Cache Service: This is the synchronized replicated cache service, which fully replicates all of its data to all cluster nodes that are running the service. Furthermore, it supports pessimistic locking so that data can be modified in a cluster without encountering the classic missing update problem.
Regarding resources, a clustered service typically uses one daemon thread, and optionally has a thread pool that can be configured to provide the service with additional processing bandwidth. For example, the invocation service and the distributed cache service both fully support thread pooling in order to accelerate database load operations, parallel distributed queries, and agent invocations.
It is important to note that these are only the basic clustered services, and not the full set of types of caches provided by Coherence. By combining clustered services with cache features such as backing maps and overflow maps, Coherence can provide an extremely flexible, configurable and powerful set of options for clustered applications. For example, the Near Cache functionality uses a Distributed Cache as one of its components.
Within a cache service, there exists any number of named caches. A named cache provides the standard JCache API, which is based on the Java collections API for key-value pairs, known as java.util.Map. The Map interface is the same API that is implemented by the Java Hashtable class, for example.
Replicated Cache Service
Overview
The first type of cache that Coherence supported was the replicated cache, and it was an instant success due to its ability to handle data replication, concurrency control and failover in a cluster, all while delivering in-memory data access speeds. A clustered replicated cache is exactly what it says it is: a cache that replicates its data to all cluster nodes.
There are several challenges to building a reliable replicated cache. The first is how to get it to scale and perform well. Updates to the cache have to be sent to all cluster nodes, and all cluster nodes have to end up with the same data, even if multiple updates to the same piece of data occur at the same time. Also, if a cluster node requests a lock, it should not have to get all cluster nodes to agree on the lock, otherwise it will scale extremely poorly; yet in the case of cluster node failure, all of the data and lock information must be kept safely. Coherence handles all of these scenarios transparently, and provides the most scalable and highly available replicated cache implementation available for Java applications.
The best part of a replicated cache is its access speed. Since the data is replicated to each cluster node, it is available for use without any waiting. This is referred to as "zero latency access," and is perfect for situations in which an application requires the highest possible speed in its data access. Each cluster node (JVM) accesses the data from its own memory:
In contrast, updating a replicated cache requires pushing the new version of the data to all other cluster nodes:
Coherence implements its replicated cache service in such a way that all read-only operations occur locally, all concurrency control operations involve at most one other cluster node, and only update operations require communicating with all other cluster nodes. The result is excellent scalable performance, and as with all of the Coherence services, the replicated cache service provides transparent and complete failover and failback.
The limitations of the replicated cache service should also be carefully considered. First, however much data is managed by the replicated cache service is on each and every cluster node that has joined the service. That means that memory utilization (the Java heap size) is increased for each cluster node, which can impact performance. Secondly, replicated caches with a high incidence of updates will not scale linearly as the cluster grows; in other words, the cluster will suffer diminishing returns as cluster nodes are added.
Partitioned Cache Service
Overview
To address the potential scalability limits of the replicated cache service, both in terms of memory and communication bottlenecks, Coherence has provided a distributed cache service since release 1.2. Many products have used the term distributed cache to describe their functionality, so it is worth clarifying exactly what is meant by that term in Coherence. Coherence defines a distributed cache as a collection of data that is distributed (or, partitioned) across any number of cluster nodes such that exactly one node in the cluster is responsible for each piece of data in the cache, and the responsibility is distributed (or, load-balanced) among the cluster nodes.
There are several key points to consider about a distributed cache:
- Partitioned: The data in a distributed cache is spread out over all the servers in such a way that no two servers are responsible for the same piece of cached data. This means that the size of the cache and the processing power associated with the management of the cache can grow linearly with the size of the cluster. Also, it means that operations against data in the cache can be accomplished with a "single hop," in other words, involving at most one other server.
- Load-Balanced: Since the data is spread out evenly over the servers, the responsibility for managing the data is automatically load-balanced across the cluster.
- Location Transparency: Although the data is spread out across cluster nodes, the exact same API is used to access the data, and the same behavior is provided by each of the API methods. This is called location transparency, which means that the developer does not have to code based on the topology of the cache, since the API and its behavior will be the same with a local JCache, a replicated cache, or a distributed cache.
- Failover: All Coherence services provide failover and failback without any data loss, and that includes the distributed cache service. The distributed cache service allows the number of backups to be configured; as long as the number of backups is one or higher, any cluster node can fail without the loss of data.
Access to the distributed cache will often need to go over the network to another cluster node. All other things equals, if there are n cluster nodes, (n - 1) / n operations will go over the network:
Since each piece of data is managed by only one cluster node, an access over the network is only a "single hop" operation. This type of access is extremely scalable, since it can utilize point-to-point communication and thus take optimal advantage of a switched network.
Similarly, a cache update operation can utilize the same single-hop point-to-point approach, which addresses one of the two known limitations of a replicated cache, the need to push cache updates to all cluster nodes:
In figure 4, above, the data is being sent to a primary cluster node and a backup cluster node. This is for failover purposes, and corresponds to a backup count of one. (The default backup count setting is one.) If the cache data were not critical, which is to say that it could be re-loaded from disk, the backup count could be set to zero, which would allow some portion of the distributed cache data to be lost in the event of a cluster node failure. If the cache were extremely critical, a higher backup count, such as two, could be used. The backup count only affects the performance of cache modifications, such as those made by adding, changing or removing cache entries.
Modifications to the cache are not considered complete until all backups have acknowledged receipt of the modification. This means that there is a slight performance penalty for cache modifications when using the distributed cache backups; however it guarantees that if a cluster node were to unexpectedly fail, that data consistency is maintained and no data will be lost.
Failover of a distributed cache involves promoting backup data to be primary storage. When a cluster node fails, all remaining cluster nodes determine what data each holds in backup that the failed cluster node had primary responsible for when it died. Those data becomes the responsibility of whatever cluster node was the backup for the data:
If there are multiple levels of backup, the first backup becomes responsible for the data; the second backup becomes the new first backup, and so on. Just as with the replicated cache service, lock information is also retained in the case of server failure, with the sole exception being that the locks for the failed cluster node are automatically released.
The distributed cache service also allows certain cluster nodes to be configured to store data, and others to be configured to not store data. The name of this setting is local storage enabled. Cluster nodes that are configured with the local storage enabled option will provide the cache storage and the backup storage for the distributed cache. Regardless of this setting, all cluster nodes will have the same exact view of the data, due to location transparency.
There are several benefits to the local storage enabled option:
- The Java heap size of the cluster nodes that have turned off local storage enabled will not be affected at all by the amount of data in the cache, because that data will be cached on other cluster nodes. This is particularly useful for application server processes running on older JVM versions with large Java heaps, because those processes often suffer from garbage collection pauses that grow exponentially with the size of the heap.
- Coherence allows each cluster node to run any supported version of the JVM. That means that cluster nodes with local storage enabled turned on could be running a newer JVM version that supports larger heap sizes, or Coherence's off-heap storage using the Java NIO features.
- The local storage enabled option allows some cluster nodes to be used just for storing the cache data; such cluster nodes are called Coherence cache servers. Cache servers are commonly used to scale up Coherence's distributed query functionality.
Local Storage
Overview
The Coherence architecture is modular, allowing almost any piece to be extended or even replaced with a custom implementation. One of the responsibilities of the Coherence system that is completely configurable, extendable and replaceable is local storage. Local storage refers to the data structures that actually store or cache the data that is managed by Coherence. For an object to provide local storage, it must support the same standard collections interface, java.util.Map. When a local storage implementation is used by Coherence to store replicated or distributed data, it is called a backing map, because Coherence is actually backed by that local storage implementation. The other common uses of local storage is in front of a distributed cache and as a backup behind the distributed cache.
Typically, Coherence uses one of the following local storage implementations:
- Safe HashMap: This is the default lossless implementation. A lossless implementation is one, like Java's Hashtable class, that is neither size-limited nor auto-expiring. In other words, it is an implementation that never evicts ("loses") cache items on its own. This particular HashMap implementation is optimized for extremely high thread-level concurrency. (For the default implementation, use class com.tangosol.util.SafeHashMap; when an implementation is required that provides cache events, use com.tangosol.util.ObservableHashMap. These implementations are thread-safe.)
- Local Cache: This is the default size-limiting and/or auto-expiring implementation. The local cache is covered in more detail below, but the primary points to remember about it are that it can limit the size of the cache, and it can automatically expire cache items after a certain period of time. (For the default implementation, use com.tangosol.net.cache.LocalCache; this implementation is thread safe and supports cache events, com.tangosol.net.CacheLoader, CacheStore and configurable/pluggable eviction policies.)
- Read/Write Backing Map: This is the default backing map implementation for caches that load from a database on a cache miss. It can be configured as a read-only cache (consumer model) or as either a write-through or a write-behind cache (for the consumer/producer model). The write-through and write-behind modes are intended only for use with the distributed cache service. If used with a near cache and the near cache must be kept in sync with the distributed cache, it is possible to combine the use of this backing map with a Seppuku-based near cache (for near cache invalidation purposes); however, given these requirements, it is suggested that the versioned implementation be used. (For the default implementation, use class com.tangosol.net.cache.ReadWriteBackingMap.)
- Versioned Backing Map: This is an optimized version of the read/write backing map that optimizes its handling of the data by utilizing a data versioning technique. For example, to invalidate near caches, it simply provides a version change notification, and to determine whether cached data needs to be written back to the database, it can compare the persistent (database) version information with the transient (cached) version information. The versioned implementation can provide very balanced performance in large scale clusters, both for read-intensive and write-intensive data. (For the default implementation, use class com.tangosol.net.cache.VersionedBackingMap; with this backing map, you can optionally use the com.tangosol.net.cache.VersionedNearCache as a near cache implementation.)
- Binary Map (Java NIO): This is a backing map implementation that can store its information in memory but outside of the Java heap, or even in memory-mapped files, which means that it does not affect the Java heap size and the related JVM garbage-collection performance that can be responsible for application pauses. This implementation is also available for distributed cache backups, which is particularly useful for read-mostly and read-only caches that require backup for high availability purposes, because it means that the backup does not affect the Java heap size yet it is immediately available in case of failover.
- Serialization Map: This is a backing map implementation that translates its data to a form that can be stored on disk, referred to as a serialized form. It requires a separate com.tangosol.io.BinaryStore object into which it stores the serialized form of the data; usually, this is the built-in LH disk store implementation, but the Serialization Map supports any custom implementation of BinaryStore. (For the default implementation of Serialization Map, use com.tangosol.net.cache.SerializationMap.)
- Serialization Cache: This is an extension of the SerializationMap that supports an LRU eviction policy. This can be used to limit the size of disk files, for example. (For the default implementation of Serialization Cache, use com.tangosol.net.cache.SerializationCache.)
- Overflow Map: An overflow map doesn't actually provide storage, but it deserves mention in this section because it can tie together two local storage implementations so that when the first one fills up, it will overflow into the second. (For the default implementation of OverflowMap, use com.tangosol.net.cache.OverflowMap.)
Local Cache
Overview
While it is not a clustered service, the Coherence local cache implementation is often used in combination with various Coherence clustered cache services. The Coherence local cache is just that: A cache that is local to (completely contained within) a particular cluster node. There are several attributes of the local cache that are particularly interesting:
- The local cache implements the same standard collections interface that the clustered caches implement, meaning that there is no programming difference between using a local or a clustered cache. Just like the clustered caches, the local cache is tracking to the JCache API, which itself is based on the same standard collections API that the local cache is based on.
- The local cache can be size-limited. This means that the local cache can restrict the number of entries that it caches, and automatically evict entries when the cache becomes full. Furthermore, both the sizing of entries and the eviction policies are customizable, for example allowing the cache to be size-limited based on the memory utilized by the cached entries. The default eviction policy uses a combination of Most Frequently Used (MFU) and Most Recently Used (MRU) information, scaled on a logarithmic curve, to determine what cache items to evict. This algorithm is the best general-purpose eviction algorithm because it works well for short duration and long duration caches, and it balances frequency versus recentness to avoid cache thrashing. The pure LRU and pure LFU algorithms are also supported, as well as the ability to plug in custom eviction policies.
- The local cache supports automatic expiration of cached entries, meaning that each cache entry can be assigned a time to live in the cache. Furthermore, the entire cache can be configured to flush itself on a periodic basis or at a preset time.
- The local cache is thread safe and highly concurrent, allowing many threads to simultaneously access and update entries in the local cache.
- The local cache supports cache notifications. These notifications are provided for additions (entries that are put by the client, or automatically loaded into the cache), modifications (entries that are put by the client, or automatically reloaded), and deletions (entries that are removed by the client, or automatically expired, flushed, or evicted.) These are the same cache events supported by the clustered caches.
- The local cache maintains hit and miss statistics. These runtime statistics can be used to accurately project the effectiveness of the cache, and adjust its size-limiting and auto-expiring settings accordingly while the cache is running.
The local cache is important to the clustered cache services for several reasons, including as part of Coherence's near cache technology, and with the modular backing map architecture.
Best Practices
Overview
Coherence supports several cache topologies, but the following options cover the vast majority of use cases. All are fully coherent and support cluster-wide locking and transactions:
- Replicated - Each machine contains a full copy of the dataset. Read access is instantaneous.
- Partitioned (Distributed) - Each machine contains a unique partition of the dataset. Adding machines to the cluster will increase the capacity of the cache. Both read and write access involve network transfer and serialization/deserialization.
- Near - Each machine contains a small local cache which is synchronized with a larger Partitioned cache, optimizing read performance. There is some overhead involved with synchronizing the caches.
Data Access Patterns
Data access distribution (hot spots)
When caching a large dataset, typically a small portion of that dataset will be responsible for most data accesses. For example, in a 1000 object dataset, 80% of operations may be be against a 100 object subset. The remaining 20% of operations may be against the other 900 objects. Obviously the most effective return on investment will be gained by caching the 100 most active objects; caching the remaining 900 objects will provide 25% more effective caching while requiring a 900% increase in resources.
On the other hand, if every object is accessed equally often (for example in sequential scans of the dataset), then caching will require more resources for the same level of effectiveness. In this case, achieving 80% cache effectiveness would require caching 80% of the dataset versus 10%. (Note that sequential scans of partially cached data sets will generally defeat MRU, LFU and MRU-LFU eviction policies). In practice, almost all non-synthetic (benchmark) data access patterns are uneven, and will respond well to caching subsets of data.
In cases where a subset of data is active, and a smaller subset is particularly active, Near caching can be very beneficial when used with the "all" invalidation strategy (this is effectively a two-tier extension of the above rules).
Cluster-node affinity
Coherence's Near cache technology will transparently take advantage of cluster-node affinity, especially when used with the "present" invalidation strategy. This topology is particularly useful when used in conjunction with a sticky load-balancer. Note that the "present" invalidation strategy results in higher overhead (as opposed to "all") when the front portion of the cache is "thrashed" (very short lifespan of cache entries); this is due to the higher overhead of adding/removing key-level event listeners. In general, a cache should be tuned to avoid thrashing and so this is usually not an issue.
Read-write ratio and data sizes
Generally speaking, the following cache topologies are best for the following use cases:
Replicated cache - small amounts of read-heavy data (e.g. metadata)
Partitioned cache - large amounts of read-write data (e.g. large data caches)
Near cache - similar to Partitioned, but has further benefits from read-heavy tiered access patterns (e.g. large data caches with hotspots) and "sticky" data access (e.g. sticky HTTP session data). Depending on the synchronization method (expiry, asynchronous, synchronous), the worst case performance may range from similar to a Partitioned cache to considerably worse.
Interleaving
Interleaving refers to the number of cache reads between each cache write. The Partitioned cache is not affected by interleaving (as it is designed for 1:1 interleaving). The Replicated and Near caches by contrast are optimized for read-heavy caching, and prefer a read-heavy interleave (e.g. 10 reads between every write). This is because they both locally cache data for subsequent read access. Writes to the cache will force these locally cached items to be refreshed, a comparatively expensive process (relative to the near-zero cost of fetching an object off the local memory heap). Note that with the Near cache technology, worst-case performance is still similar to the Partitioned cache; the loss of performance is relative to best-case scenarios.
Note that interleaving is related to read-write ratios, but only indirectly. For example, a Near cache with a 1:1 read-write ratio may be extremely fast (all writes followed by all reads) or much slower (1:1 interleave, write-read-write-read...).
Heap Size Considerations
Using several small heaps
For large datasets, Partitioned or Near caches are recommended. As the scalability of the Partitioned cache is linear for both reading and writing, varying the number of Coherence JVMs will not significantly affect cache performance. On the other hand, JVM memory management routines show worse than linear scalability. For example, increasing JVM heap size from 512MB to 2GB may substantially increase garbage collection (GC) overhead and pauses.
For this reason, it is common to use multiple Coherence instances per physical machine. As a general rule of thumb, current JVM technology works well up to 512MB heap sizes. Thererfore, using a number of 512MB Coherence instances will provide optimal performance without a great deal of JVM configuration or tuning.
For performance-sensitive applications, experimentation may provide better tuning. When considering heap size, it is important to find the right balance. The lower bound is determined by per-JVM overhead (and also, manageability of a potentially large number of JVMs). For example, if there is a fixed overhead of 100MB for infrastructure software (e.g. JMX agents, connection pools, internal JVM structures), then the use of JVMs with 256MB heap sizes will result in close to 40% overhead for non-cache data. The upper bound on JVM heap size is governed by memory management overhead, specifically the maximum duration of GC pauses and the percentage of CPU allocated to GC (and other memory management tasks).
For Java 5 VMs running on commodity systems, the following rules generally hold true (with no JVM configuration tuning). With a heap size of 512MB, GC pauses will not exceed one second. With a heap size of 1GB, GC pauses are limited to roughly 2-3 seconds. With a heap size of 2GB, GC pauses are limited to roughly 5-6 seconds. It is important to note that GC tuning will have an enormous impact on GC throughput and pauses. In all configurations, initial (-Xms) and maximum (-Xmx) heap sizes should be identical. There are many variations that can substantially impact these numbers, including machine architecture, CPU count, CPU speed, JVM configuration, object count (object size), object access profile (short-lived versus long-lived objects).
For allocation-intensive code, GC can theoretically consume close to 100% of CPU usage. For both cache server and client configurations, most CPU resources will typically be consumed by application-specific code. It may be worthwhile to view verbose garbage collection statistics (e.g. -verbosegc). Use the profiling features of the JVM to get profiling information including CPU usage by GC (e.g. -Xprof).
Moving the cache out of the application heap
Using dedicated Coherence cache server instances for Partitioned cache storage will minimize the heap size of application JVMs as the data is no longer stored locally. As most Partitioned cache access is remote (with only 1/N of data being held locally), using dedicated cache servers does not generally impose much additional overhead. Near cache technology may still be used, and it will generally have a minimal impact on heap size (as it is caching an even smaller subset of the Partitioned cache). Many applications are able to dramatically reduce heap sizes, resulting in better responsiveness.
Local partition storage may be enabled (for cache servers) or disabled (for application server clients) with the tangosol.coherence.distributed.localstorage Java property (e.g. -Dtangosol.coherence.distributed.localstorage=false).
It may also be disabled by modifying the <local-storage> setting in the tangosol-coherence.xml (or tangosol-coherence-override.xml) file as follows:
<coherence>
<services>
<service>
<service-type>DistributedCache</service-type>
<service-component>DistributedCache</service-component>
<init-params>
<init-param>
<param-name>local-storage</param-name>
<param-value system-property="tangosol.coherence.distributed.localstorage">false<param-value>
</init-param>
</init-params>
</service>
</services>
</coherence>
At least one storage-enabled JVM must be started before any storage-disabled clients access the cache.
Network Protocols
Overview
Coherence uses TCMP, a clustered IP-based protocol, for server discovery, cluster management, service provisioning and data transmission. To ensure true scalability, the TCMP protocol is completely asychronous, meaning that communication is never blocking, even when many threads on a server are communicating at the same time. Further, the asynchronous nature also means that the latency of the network (for example, on a routed network between two different sites) does not affect cluster throughput, although it will affect the speed of certain operations.
TCMP uses a combination of UDP/IP multicast, UDP/IP unicast and TCP/IP as follows:
- Multicast
- Cluster discovery: Is there a cluster already running that a new member can join?
- Cluster heartbeat: The most senior member in the cluster issues a periodic heartbeat via multi-cast; the rate is configurable and defaults to once per second.
- Message delivery: Messages that need to be delivered to multiple cluster members will often be sent via multicast, instead of unicasting the message one time to each member.
- Unicast
- Direct member-to-member ("point-to-point") communication, including messages, asynchronous acknowledgements (ACKs), asynchronous negative acknowledgements (NACKs) and peer-to-peer heartbeats.
- Under some circumstances, a message may be sent via unicast even if the message is directed to multiple members. This is done to shape traffic flow and to reduce CPU load in very large clusters.
- TCP
- An optional TCP/IP ring is used as an additional "death detection" mechanism, to differentiate between actual node failure and an unresponsive node, such as when a JVM conducts a full GC.
- TCP/IP is not used as a data transfer mechanism due to the intrinsic overhead of the protocol and its synchronous nature.
Protocol Reliability
The TCMP protocol provides fully reliable, in-order delivery of all messages. Since the underlying UDP/IP protocol does not provide for either reliable or in-order delivery, TCMP utilizes a queued, fully asynchronous ACK- and NACK-based mechanism for reliable delivery of messages, with unique integral identity for guaranteed ordering of messages.
Protocol Resource Utilization
The TCMP protocol requires only two UDP/IP sockets (one multicast, one unicast) and four threads per JVM, regardless of the cluster size. This is a key element in the scalability of Coherence, in that regardless of the number of servers, each node in the cluster can still communicate either point-to-point or with collections of cluster members without requiring additional network connections.
The optional TCP/IP ring will use a few additional TCP/IP sockets, and a total of one additional thread.
Protocol Tunability
The TCMP protocol is very tunable to take advantage of specific network topologies, or to add tolerance for low-bandwidth and/or high-latency segments in a geographically distributed cluster. Coherence comes with a pre-set configuration, some of which is dynamically self-configuring at runtime, but all attributes of TCMP can be overridden and locked down for deployment purposes.
Multicast Scope
Multicast UDP/IP packets are configured with a time-to-live value (TTL) that designates how far those packets can travel on a network. The TTL is expressed in terms of how many "hops" a packet will survive; each network interface, router and managed switch is considered one hop. Coherence provides a TTL setting to limit the scope of multicast messages.
Disabling Multicast
In most WAN environments, and some LAN environments, multicast traffic is disallowed. To prevent Coherence from using multicast, configure a list of well-known-addresses (WKA). This will disable multicast discovery, and also disable multicast for all data transfer. Coherence is designed to use point-to-point communication as much as possible, so most application profiles will not see a substantial performance impact.
Operational Configuration
Operational Configuration Elements
Operational Configuration Deployment Descriptor Elements
Description
The following sections describe the elements that control the operational and runtime settings used by Tangosol Coherence to create, configure and maintain its clustering, communication, and data management services. These elements may be specified in either the tangosol-coherence.xml operational descriptor, or the tangosol-coherence-override.xml override file. For information on configuring caches see the cache configuration descriptor section.
Document Location
When deploying Coherence, it is important to make sure that the tangosol-coherence.xml descriptor is present and situated in the application classpath (like with any other resource, Coherence will use the first one it finds in the classpath). By default (as Tangosol ships the software) tangosol-coherence.xml is packaged into in the coherence.jar.
Document Root
The root element of the operational descriptor is coherence, this is where you may begin configuring your cluster and services.
Document Format
Coherence Operational Configuration deployment descriptor should begin with the following DOCTYPE declaration:
<!DOCTYPE coherence PUBLIC "-//Tangosol, Inc.//DTD Tangosol Coherence 3.0//EN" "http://www.tangosol.com/dtd/coherence_3_0.dtd">
| When deploying Coherence into environments where the default character set is EBCDIC rather than ASCII, please make sure that this descriptor file is in ASCII format and is deployed into its runtime environment in the binary format. |
Operational Override File (tangosol-coherence-override.xml)
Though it is acceptable to supply an alternate definition of the default tangosol-coherence.xml file, the preferred approach to operational configuration is to specify an override file. The override file contains only the subset of the operational descriptor which you wish to adjust. The default name for the override file is tangosol-coherence-override.xml, and the first instance found in the classpath will be used. The format of the override file is the same as for the operational descriptor, except that all elements are optional, any missing element will simply be loaded from the operational descriptor.
| It is recommended that you supply an override file rather then a custom operational descriptor, thus specifing only the settings you wish to adjust. |
Command Line Override
Tangosol Coherence provides a very powerful Command Line Setting Override Feature, which allows for any element defined in this descriptor to be overridden from the Java command line if it has a system-property attribute defined in the descriptor. This feature allows you to use the same operational descriptor (and override file) across all cluster nodes, and provide per-node customizations as system properties.
Element Index
The following table lists all non-terminal elements which may be used from within the operational configuration.
access-controller
Used in: security-config.
The following table describes the elements you can define within the access-controller element.
| Element |
Required/Optional |
Description |
| <class-name> |
Required |
Specifies the name of a Java class that implements com.tangosol.net.security.AccessController interface, which will be used by the Coherence Security Framework to check access rights for clustered resources and encrypt/decrypt node-to-node communications regarding those rights.
Default value is com.tangosol.net.security.DefaultController. |
| <init-params> |
Optional |
Contains one or more initialization parameter(s) for a class that implements the AccessController interface.
For the default AccessController implementation the parameters are the paths to the key store file and permissions description file, specified as follows:
<init-params>
<init-param id="1">
<param-type>java.io.File</param-type>
<param-value system-property="tangosol.coherence.security.keystore"></param-value>
</init-param>
<init-param id="2">
<param-type>java.io.File</param-type>
<param-value system-property="tangosol.coherence.security.permissions"></param-value>
</init-param>
</init-params>
Preconfigured overrides based on the default AccessController implementation and the default parameters as specified above are tangosol.coherence.security.keystore and tangosol.coherence.security.permissions.
For more information on the elements you can define within the init-param element, refer to init-param.
|
authorized-hosts
Used in: cluster-config.
Description
If specified, restricts cluster membership to the cluster nodes specified in the collection of unicast addresses, or address range. The unicast address is the address value from the authorized cluster nodes' unicast-listener element. Any number of host-address and host-range elements may be specified.
Elements
The following table describes the elements you can define within the authorized-hosts element.
| Element |
Required/Optional |
Description |
| <host-address> |
Optional |
Specifies an IP address or hostname. If any are specified, only hosts with specified host-addresses or within the specified host-ranges will be allowed to join the cluster.
The content override attributes id can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document. |
| <host-range> |
Optional |
Specifies a range of IP addresses. If any are specified, only hosts with specified host-addresses or within the specified host-ranges will be allowed to join the cluster.
The content override attributes id can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document. |
The content override attributes xml-override and id can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
burst-mode
Used in: packet-publisher.
Description
The burst-mode element is used to control the rate at which the Packet publisher will transmit packets on the network, by specif icing the maximum number of packets to transmit without pausing.
Elements
The following table describes the elements you can define within the burst-mode element.
| Element |
Required/Optional |
Description |
| <maximum-packets> |
Required |
Specifies the maximum number of packets that the packet publisher will send in a row without pausing. Zero indicates no limit. By setting this value relatively low, the publisher is forced to pause when sending a large number of packets, which may reduce collisions in some instances or allow incoming traffic to be more quickly processed.
Default value is 1024. |
| <pause-milliseconds> |
Required |
Specifies the minimum number of milliseconds that the Publisher will pause between long bursts of packets. By increasing this value, the packet publisher is forced to pause longer when sending a large number of packets, which may reduce collisions in some instances or allow incoming traffic to be more quickly processed.
Default value is 4. |
callback-handler
Used in: security-config.
The following table describes the elements you can define within the callback-handler element.
| Element |
Required/Optional |
Description |
| <class-name> |
Required |
Specifies the name of a Java class that provides the implementation for the javax.security.auth.callback.CallbackHandler interface. |
| <init-params> |
Optional |
Contains one or more initialization parameter(s) for a CallbackHandler implementation.
For more information on the elements you can define within the init-param element, refer to init-param. |
cluster-config
Used in: coherence.
Description
Contains the cluster configuration information, including communication and service parameters.
Elements
The following table describes the elements you can define within the cluster-config element.
| Element |
Required/Optional |
Description |
| <cluster-name> |
Optional |
Specifies the name for the cluster. In order for a new node to join a running cluster it must be configured with the same cluster name as the running cluster. This prevents separate uniquely named clusters, which are unknowingly using the same multicast address from erroneously forming a single cluster.
reconfigured override is tangosol.coherence.clustername.
Default value is an empty string. |
| <unicast-listener> |
Required |
Specifies the configuration information for the Unicast listener, used for receiving point-to-point network communications. |
| <multicast-listener> |
Required |
Specifies the configuration information for the Multicast listener, used for receiving point-to-multipoint network communications. |
| <shutdown-listener> |
Required |
Specifies the action to take upon receiving an external shutdown request. |
| <tcp-ring-listener> |
Required |
Specifies configuration information for the TCP Ring listener, used to death detection. |
| <packet-publisher> |
Required |
Specifies configuration information for the Packet publisher, used for network data transmission. |
| <incoming-message-handler> |
Required |
Specifies configuration information for the Incoming message handler, used for dispatching incoming cluster communications. |
| <outgoing-message-handler> |
Required |
Specifies configuration information for the Outgoing message handler, used for dispatching outgoing cluster communications. |
| <authorized-hosts> |
Optional |
Specifies the hosts which are allowed to join the cluster. |
| <services> |
Required |
Specifies the declarative data for all available Coherence services. |
| <filters> |
Optional |
Specifies data transformation filters, which can be used to perform custom transformations on data being transfered between cluster nodes. |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
coherence
Description
The coherence element is the root element of the operational deployment descriptor.
Elements
The following table describes the elements you can define within the coherence element.
| Element |
Required/Optional |
Description |
| <cluster-config> |
Required |
Contains the cluster configuration information. This element is where most communication and service parameters are defined. |
| <logging-config> |
Required |
Contains the configuration information for the logging facility. |
| <configurable-cache-factory-config> |
Required |
Contains configuration information for the configurable cache factory. It controls where, from, and how the cache configuration settings are loaded. |
| <management-config> |
Required |
Contains the configuration information for the Coherence Management Framework. |
| <security-config> |
Required |
Contains the configuration information for the Coherence Security Framework. |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
configurable-cache-factory-config
Used in: coherence.
Elements
The following table describes the elements you can define within the configurable-cache-factory-config element.
| Element |
Required/Optional |
Description |
| <class-name> |
Required |
Specifies the name of a Java class that provides the cache configuration factory.
Default value is
com.tangosol.net.DefaultConfigurableCacheFactory. |
| <init-params> |
Optional |
Contains one or more initialization parameter(s) for a cache configuration factory class which implements the
com.tangosol.run.xml.XmlConfigurable interface.
For the default cache configuration factory class (DefaultConfigurableCacheFactory) the parameters are specified as follows:
<init-param>
<param-type>java.lang.String</param-type>
<param-value system-property="tangosol.coherence.cacheconfig">
coherence-cache-config.xml
</param-value>
</init-param>
Preconfigured override is tangosol.coherence.cacheconfig.
Unless an absolute or relative path is specified, such as with ./path/to/config.xml, the application's classpath will be used to find the specified descriptor. |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
filters
Used in: cluster-config.
Description
Data transformation filters can be used by services to apply a custom transformation on data being transfered between cluster nodes. This can be used for instance to compress or encrypt Coherence network traffic.
Implementation
Data transformation filters are implementations of the
com.tangosol.util.WrapperStreamFactory interface.
| Data transformation filters are not related to
com.tangosol.util.Filter, which is part of the Coherence API for querying caches. |
Elements
The following table describes the elements you can define within each filter element.
| Element |
Required/Optional |
Description |
| <filter-name> |
Required |
Specifies the canonical name of the filter. This name is unique within the cluster.
For example: gzip.
The content override attributes id can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document. |
| <filter-class> |
Required |
Specifies the class name of the filter implementation. This class must have a zero-parameter public constructor and must implement the
com.tangosol.util.WrapperStreamFactory interface. |
| <init-params> |
Optional |
Specifies initialization parameters, for configuring filters which implement the
com.tangosol.run.xml.XmlConfigurable interface.
For example when using a
com.tangosol.net.CompressionFilter the parameters are specified as follows:
<init-param>
<param-name>strategy</param-name>
<param-value>gzip</param-value>
</init-param>
<init-param>
<param-name>level</param-name>
<param-value>default</param-value>
</init-param>
For more information on the parameter values for the Compression Filter, refer to Compression Filter Parameters. |
The content override attributes xml-override and id can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
host-range
Used in: authorized-hosts.
Description
Specifies a range of unicast addresses of nodes which are allowed to join the cluster.
Elements
The following table describes the elements you can define within each host-range element.
| Element |
Required/Optional |
Description |
| <from-address> |
Required |
Specifies the starting IP address for a range of host addresses.
For example: 198.168.1.1. |
| <to-address> |
Required |
Specifies to-address element specifies the ending IP address (inclusive) for a range of hosts.
For example: 198.168.2.255. |
The content override attributes id can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
incoming-message-handler
Used in: cluster-config.
Description
The incoming-message-handler assembles UDP packets into logical messages and dispatches them to the appropriate Coherence service for processing.
Elements
The following table describes the elements you can define within the incoming-message-handler element.
| Element |
Required/Optional |
Description |
| <maximum-time-variance> |
Required |
Specifies the maximum time variance between sending and receiving broadcast Messages when trying to determine the difference between a new cluster Member's system time and the cluster time.
The smaller the variance, the more certain one can be that the cluster time will be closer between multiple systems running in the cluster; however, the process of joining the cluster will be extended until an exchange of Messages can occur within the specified variance.
Normally, a value as small as 20 milliseconds is sufficient, but with heavily loaded clusters and multiple network hops it is possible that a larger value would be necessary.
Default value is 16. |
| <use-nack-packets> |
Required |
Specifies whether the packet receiver will use negative acknowledgments (packet requests) to pro-actively respond to known missing packets.
Legal values are true or false.
Default value is true. |
| <priority> |
Required |
Specifies a priority of the incoming message handler execution thread.
Legal values are from 1 to 10.
Default value is 7. |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
init-param
Used in: init-params.
Description
Defines an individual initialization parameter.
Elements
The following table describes the elements you can define within the init-param element.
| Element |
Required/Optional |
Description |
| <param-name> |
Optional |
Specifies the name of the parameter passed to the class. The param-type or param-name must be specified.
For example: thread-count.
For more information on the pre-defined parameter values available for the specific elements , refer to Parameters. |
| <param-type> |
Optional |
Specifies the data type of the parameter passed to the class. The param-type or param-name must be specified.
For example: int |
| <param-value> |
Required |
Specifies the value passed in the parameter.
For example: 8.
For more information on the pre-defined parameter values available for the specific elements, refer to Parameters. |
The content override attributes id can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
init-params
Used in: filters, services, configurable-cache-factory-config, access-controller and callback-handler.
Description
Defines a series of initialization parameters.
Elements
The following table describes the elements you can define within the init-params element.
| Element |
Required/Optional |
Description |
| <init-param> |
Optional |
Defines an individual initialization parameter. |
logging-config
Used in: coherence.
Elements
The following table describes the elements you can define within the logging-config element.
| Element |
Required/Optional |
Description |
| <destination> |
Required |
Specifies the output device used by the logging system.
Legal values are:
stdout
stderr
jdk
log4j
a file name
Default value is stderr.
If jdk is specified as the destination, Coherence must be run using JDK 1.4 or later; likewise, if log4j is specified, the Log4j libraries must be in the classpath. In both cases, the appropriate logging configuration mechanism (system properties, property files, etc.) should be used to configure the JDK/Log4j logging libraries.
Preconfigured override is tangosol.coherence.log |
| <severity-level> |
Required |
Specifies which logged messages will be output to the log destination.
Legal values are:
0 - only output without a logging severity level specified will be logged
1 - all the above plus errors
2 - all the above plus warnings
3 - all the above plus informational messages
4-9 - all the above plus internal debugging messages (the higher the number, the more the messages)
-1 - no messages
Default value is 3.
Preconfigured override is tangosol.coherence.log.level |
| <message-format> |
Required |
Specifies how messages that have a logging level specified will be formatted before passing them to the log destination.
The value of the message-format element is static text with the following replaceable parameters:
{date} - the date/time format (to a millisecond) at which the message was logged
{version} - the Tangosol Coherence exact version and build details
{level} - the logging severity level of the message
{thread} - the thread name that logged the message
{member} - the cluster member id (if the cluster is currently running)
{text} - the text of the message
Default value is:
{date} Tangosol Coherence {version} <{level}> (thread={thread}, member={member}): {text} |
| <character-limit> |
Required |
Specifies the maximum number of characters that the logger daemon will process from the message queue before discarding all remaining messages in the queue. Note that the message that caused the total number of characters to exceed the maximum will NOT be truncated, and all messages that are discarded will be summarized by the logging system with a single log entry detailing the number of messages that were discarded and their total size. The truncation of the logging is only temporary, since once the queue is processed (emptied), the logger is reset so that subsequent messages will be logged.
The purpose of this setting is to avoid a situation where logging can itself prevent recovery from a failing condition. For example, with tight timings, logging can actually change the timings, causing more failures and probably more logging, which becomes a vicious cycle. A limit on the logging being done at any one point in time is a "pressure valve" that prevents such a vicious cycle from occurring. Note that logging occurs on a dedicated low-priority thread to even further reduce its impact on the critical portions of the system.
Legal values are positive integers or zero. Zero implies no limit.
Default value is 4096.
Preconfigured override is tangosol.coherence.log.limit |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
management-config
Used in: coherence.
Elements
The following table describes the elements you can define within the management-config element.
| Element |
Required/Optional |
Description |
| <domain-name> |
Required |
Specifies the name of the JMX domain used to register MBeans exposed by the Coherence Management Framework. |
| <managed-nodes> |
Required |
Specifies whether or not a cluster node's JVM has an [in-process] MBeanServer and if so, whether or not this node allows management of other nodes' managed objects.
Legal values are:
- none - No MBeanServer is instantiated.
- local-only - Manage only MBeans which are local to the cluster node (i.e. within the same JVM).
- remote-only - Manage MBeans on other remotely manageable cluster nodes. Requires a Coherence Enterprise License
- all - Manage both local and remotely manageable cluster nodes. Requires a Coherence Enterprise License
Default value is none.
Preconfigured override is tangosol.coherence.management |
| <allow-remote-management> |
Required |
Specifies whether or not this cluster node exposes its managed objects to remote MBeanServer(s).
Legal values are: true or false.
Default value is false.
Preconfigured override is tangosol.coherence.management.remote |
| <read-only> |
Required |
Specifies whether or not the managed objects exposed by this cluster node allow operations that modify run-time attributes.
Legal values are: true or false.
Default value is false.
Preconfigured override is tangosol.coherence.management.readonly |
| <service-name> |
Required |
Specifies the name of the Invocation Service used for remote management.
This element is used only if allow-remote-management is set to true. |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
multicast-listener
Used in: cluster-config.
Description
Specifies the configuration information for the Multicast listener. This element is used to specify the address and port that a cluster will use for cluster wide and point-to-multipoint communications. All nodes in a cluster must use the same multicast address and port, whereas distinct clusters on the same network should use different multicast addresses.
Multicast-Free Clustering
By default Coherence uses a multicast protocol to discover other nodes when forming a cluster. If multicast networking is undesirable, or unavailable in your environment, the well-known-addresses feature may be used to eliminate the need for multicast traffic. Note: The use of the Well Known Addressing feature requires an Coherence Enterprise License. If you are having difficulties in establishing a cluster via multicast, see the Multicast Test.
Elements
The following table describes the elements you can define within the multicast-listener element.
| Element |
Required/Optional |
Description |
| <address> |
Required |
Specifies the multicast IP address that a Socket will listen or publish on.
Legal values are from 224.0.0.0 to 239.255.255.255.
Default value depends on the release and build level and typically follows the convention of {build}.{major version}.{minor version}.{patch}. For example, for Coherence Release 2.2 build 255 it is 225.2.2.0.
Preconfigured override is tangosol.coherence.clusteraddress |
| <port> |
Required |
Specifies the port that the Socket will listen or publish on.
Legal values are from 1 to 65535.
Default value depends on the release and build level and typically follows the convention of {version}+{{{build}. For example, for Coherence Release 2.2 build 255 it is 22255.
Preconfigured override is tangosol.coherence.clusterport |
| <time-to-live> |
Required |
Specifies the time-to-live setting for the multicast. This determines the maximum number of "hops" a packet may traverse, where a hop is measured as a traversal from one network segment to another via a router.
Legal values are from from 0 to 255.
| For production use, this value should be set to the lowest integer value that works. On a single server cluster, it should work at 0; on a simple switched backbone, it should work at 1; on an advanced backbone with intelligent switching, it may require a value of 2 or more. Setting the value too high can use unnecessary bandwidth on other LAN segments and can even cause the OS or network devices to disable multicast traffic. While a value of 0 is meant to keep packets from leaving the originating machine, some OSs do not implement this correctly, and the packets may in fact be transmitted on the network. |
Default value is 4.
Preconfigured override is tangosol.coherence.ttl |
| <packet-buffer> |
Required |
Specifies how many incoming packets the OS will be requested to buffer. |
| <priority> |
Required |
Specifies a priority of the multicast listener execution thread.
Legal values are from 1 to 10.
Default value is 8. |
| <join-timeout-milliseconds> |
Required |
Specifies the number of milliseconds that a new member will wait without finding any evidence of a cluster before starting its own cluster and electing itself as the senior cluster member.
Legal values are from 1 to 1000000.
Note: For production use, the recommended value is 30000.
Default value is 6000. |
| <multicast-threshold-percent> |
Required |
Specifies the threshold percentage value used to determine whether a packet will be sent via unicast or multicast. It is a percentage value and is in the range of 1% to 100%. In a cluster of "n" nodes, a particular node sending a packet to a set of other (i.e. not counting self) destination nodes of size "d" (in the range of 0 to n-1), the packet will be sent multicast if and only if the following both hold true:
- The packet is being sent over the network to more than one other node, i.e. (d > 1).
- The number of nodes is greater than the threshold,i.e. (d > (n-1) * (threshold/100)).
Setting this value to 1 will allow the implementation to use multicast for basically all multi-point traffic. Setting it to 100 will force the implementation to use unicast for all multi-point traffic except for explicit broadcast traffic (e.g. cluster heartbeat and discovery) because the 100% threshold will never be exceeded. With the setting of 25 the implementation will send the packet using unicast if it is destined for less than one-fourth of all nodes, and send it using multicast if it is destined for the one-fourth or more of all nodes.
Note: This element is only used if the well-known-addresses element is empty.
Legal values are from 1 to 100.
Default value is 25.
|
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
notification-queueing
Used in: packet-publisher.
Description
The notification-queueing element is used to specificy the timing of notifications packets sent to other cluster nodes. Notification packets are used to acknowledge the receipt of packets which require confirmation.
Batched Acknowledgments
Rather then sending an individual ACK for each received packet which requires confirmation, Coherence will batch a series of acknowledgments for a given sender into a single ACK. The ack-delay-milliseconds specifies the maximum amount of time that an acknowledgment will be delayed before an ACK notification is sent. By batching the acknowledgments Coherence avoids wasting network bandwidth with many small ACK packets.
Elements
The following table describes the elements you can define within the notification-queueing element.
| Element |
Required/Optional |
Description |
| <ack-delay-milliseconds> |
Required |
Specifies the maximum number of milliseconds that the packet publisher will delay before sending an ACK packet. The ACK packet may be transmitted earlier if number of batched acknowledgments fills the ACK packet.
This value should be substantially lower then the remote node's packet-delivery resend timeout, to allow ample time for the ACK to be received and processed by the remote node before the resend timeout expires.
Default value is 64. |
| <nack-delay-milliseconds> |
Required |
Specifies the number of milliseconds that the packet publisher will delay before sending a NACK packet.
Default value is 16. |
outgoing-message-handler
Used in: cluster-config.
Description
The outgoing-message-handler splits logical messages into packets for transmission on the network, and enqueues them on the packet-publisher.
Elements
The following table describes the elements you can define within the outgoing-message-handler element.
| Element |
Required/Optional |
Description |
| <use-daemon> |
Required |
Specifies whether or not a daemon thread will be created to perform the outgoing message handling.
If the daemon thread is NOT created, then any thread that sends a message will itself do the work of splitting the message into packets and putting them into the packet publisher's queue. This is the recommended value as it spreads the processing over more threads.
Legal values are:
Legal values are true or false.
Default value is false. |
| <use-filters> |
Optional |
Contains the list of filter names to be used by this outgoing message handler.
For example, specifying use-filter as follows
<use-filters>
<filter-name>gzip</filter-name>
</use-filters>
will activate gzip compression for all network messages, which can help substantially with WAN and low-bandwidth networks.
|
| <priority> |
Required |
Specifies a priority of the outgoing message handler execution thread.
Legal values are from 1 to 10.
Default value is 7. |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
packet-buffer
Used in: unicast-listener, multicast-listener, packet-publisher.
Description
Specifies the size of the OS buffer for datagram sockets.
Performance Impact
Large inbound buffers help insulate the Coherence network layer from JVM pauses caused by the Java Garbage Collector. While the JVM is paused, Coherence is unable to dequeue packets from any inbound socket. If the pause is long enough to cause the packet buffer to overflow, the packet reception will be delayed as the the originating node will need to detect the packet loss and retransmit the packet(s).
It's just a hint
The OS will only treat the specified value as a hint, and is not required to allocate the specified amount. In the event that less space is allocated then requested Coherence will issue a warning and continue to operate with the constrained buffer, which may degrade performance. See http://forums.tangosol.com/thread.jspa?threadID=616 for details on configuring your OS to allow larger buffers.
Elements
The following table describes the elements you can define within the packet-buffer element.
| Element |
Required/Optional |
Description |
| <maximum-packets> |
Required |
For unicast-listener, multicast-listener and packet-publisher: Specifies the number of packets of maximum size that the datagram socket will be asked to size itself to buffer. See SO_SNDBUF and SO_RCVBUF. Actual buffer sizes may be smaller if the underlying socket implementation cannot support more than a certain size. Defaults are 16 for publishing, 64 for multicast listening, and 1428 for unicast listening. |
packet-delivery
Used in: packet-publisher.
Description
Specifies timing parameters related to reliable packet delivery.
Death Detection
The timeout-milliseconds and heartbeat-milliseconds are used in detecting the death of other cluster nodes.
Elements
The following table describes the elements you can define within the packet-delivery element.
| Element |
Required/Optional |
Description |
| <resend-milliseconds> |
Required |
For packets which require confirmation, specifies the minimum amount of time in milliseconds to wait for a corresponding ACK packet, before resending a packet.
Default value is 200. |
| <timeout-milliseconds> |
Required |
For packets which require confirmation, specifies the maximum amount of time, in milliseconds, that a packet will be resent. After this timeout expires Coherence will make a determination if the recipient is to be considered "dead". This determination takes additional data into account, such as if other nodes are still able to communicate with the recipient.
Default value is 60000.
Note: For production use, the recommended value is the greater of 60000 and two times the maximum expected full GC duration. |
| <heartbeat-milliseconds> |
Required |
Specifies the interval between heartbeats. Each member issues a unicast heartbeat, and the most senior member issues the cluster heartbeat, which is a broadcast message. The heartbeat is used by the tcp-ring-listener as part of fast death detection.
Default value is 1000. |
packet-publisher
Used in: cluster-config.
Description
Specifies configuration information for the Packet publisher, used for network data transmission.
Reliable packet delivery
The Packet publisher is responsible for ensuring that transmitted packets reach the destination cluster node. The publisher maintains a set of packets which are waiting to be acknowledged, and if the ACK does not arrive by the packet-delivery resend timeout, the packet will be retransmitted. The recipient node will delay the ACK, in order to batch a series of ACKs into a single response.
Throttling
The rate at which the publisher will accept and transmit packet may be controlled via the burst-mode and traffic-jam settings. Throttling may be necessary when dealing with slow networks, or small packet-buffers.
Elements
The following table describes the elements you can define within the packet-publisher element.
| Element |
Required/Optional |
Description |
| <packet-size> |
Required |
Specifies the UDP packet sizes to utilize. |
| <packet-delivery> |
Required |
Specifies timing parameters related to reliable packet delivery. |
| <notification-queueing> |
Required |
Contains the notification queue related configuration info. |
| <burst-mode> |
Required |
Specifies the maximum number of packets the publisher may transmit without pausing. |
| <traffic-jam> |
Required |
Specifies the maximum number of packets which can be enqueued on the publisher before client threads block. |
| <packet-buffer> |
Required |
Specifies how many outgoing packets the OS will be requested to buffer. |
| <priority> |
Required |
Specifies a priority of the packet publisher execution thread.
Legal values are from 1 to 10.
Default value is 6. |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
packet-size
Used in: packet-publisher.
Description
The packet-size element specifies the maximum and preferred UDP packet sizes. All cluster nodes must use identical maximum packet sizes. For optimal network utilization this value should be 32 bytes less then the network MTU.
Elements
The following table describes the elements you can define within the packet-size element.
| Element |
Required/Optional |
Description |
| <maximum-length> |
Required |
Specifies the maximum size, in bytes, of the UDP packets that will be sent and received on the unicast and multicast sockets.
This value should be at least 512; recommended value is 1468 for 100Mb, and 1Gb Ethernet. This value must be identical on all cluster nodes.
Note: Some network equipment cannot handle packets larger than 1472 bytes (IPv4) or 1468 bytes (IPv6), particularly under heavy load. If you encounter this situation on your network, this value should be set to 1472 or 1468 respectively.
Default value is 1468. |
| <preferred-length> |
Required |
Specifies the preferred size, in bytes, of UDP packets that will be sent and received on the unicast and multicast sockets.
This value should be at least 512 and cannot be greater than the maximum-length value; it is recommended to set the value to the same as the maximum-length value.
Default value is 1468. |
security-config
Used in: coherence.
Elements
The following table describes the elements you can define within the security-config element.
| Element |
Required/Optional |
Description |
| <enabled> |
Required |
Specifies whether the security features are enabled. All other configuration elements in the security-config group will be verified for validity and used if and only if the value of this element is true.
Legal values are true or false.
Default value is false.
Preconfigured override is tangosol.coherence.security |
| <login-module-name> |
Required |
Specifies the name of the JAAS LoginModule that should be used to authenticate the caller. This name should match a module in a configuration file will be used by the JAAS (for example specified via the -Djava.security.auth.login.config Java command line attribute).
For details please refer to the Sun Login Module Developer's Guide. |
| <access-controller> |
Required |
Contains the configuration information for the class that implements com.tangosol.net.security.AccessController interface, which will be used by the Coherence Security Framework to check access rights for clustered resources and encrypt/decrypt node-to-node communications regarding those rights. |
| <callback-handler> |
Optional |
Contains the configuration information for the class that implements javax.security.auth.callback.CallbackHandler interace which will be called if an attempt is made to access a protected clustered resource when there is no identity associated with the caller. |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
services
Used in: cluster-config.
Description
Specifies the configuration for Coherence services.
Service Components
The types of services which can be configured includes:
- ReplicatedCache - A cach service which maintains copies of all cache entries on all cluster nodes which run the service.
- ReplicatedCache.Optimistic - A version of the ReplicatedCache which uses optimistic locking.
- DistributedCache - A cache service which evenly partitions cache entries across the cluster nodes which run the service.
- SimpleCache - A version of the ReplicatedCache which lacks concurrent control.
- LocalCache - A cache service for caches where all cache entries reside in a single cluster node.
- InvocationService - A service used for performing custom operations on remote cluster nodes.
Elements
The following table describes the elements you can define for each service element.
| Element |
Required/Optional |
Description |
| <service-type> |
Required |
Specifies the canonical name for a service, allowing the service to be referenced from the service-name element in cache configuration caching-schemes. |
| <service-component> |
Required |
Specifies either the fully qualified class name of the service or the relocatable component name relative to the base Service component.
Legal values are:
- ReplicatedCache
- ReplicatedCache.Optimistic
- DistributedCache
- SimpleCache
- LocalCache
- InvocationService
|
| <use-filters> |
Optional |
Contains the list of filter names to be used by this service.
For example, specifying use-filter as follows
<use-filters>
<filter-name>gzip</filter-name>
</use-filters>
will activate gzip compression for the network messages used by this service, which can help substantially with WAN and low-bandwidth networks. |
| <init-params> |
Optional |
Specifies the initialization parameters that are specific to each service-component.
For more service specific parameter information see:
|
The content override attributes xml-override and id can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
shutdown-listener
Used in: cluster-config.
Description
Specifies the action a cluster node should take upon receiving an external shutdown request. External shutdown includes the "kill" command on Unix and "Ctrl-C" on Windows and Unix.
Elements
The following table describes the elements you can define within the shutdown-listener element.
| Element |
Required/Optional |
Description |
| <enabled> |
Required |
Specifies the type of action to take upon an external JVM shutdown.
Legal Values:
- none - perform no explicit shutdown actions
- force - perform "hard-stop" the node by calling Cluster.stop()
- graceful - perform a "normal" shutdown by calling Cluster.shutdown()
- true - same as force
- false - same as none
Note: For production use, the suggested value is none unless testing has verified that the behavior on external shutdown is exactly what is desired.
Default value is force.
Preconfigured override is tangosol.coherence.shutdownhook |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
socket-address
Used in: well-known-addresses.
Elements
The following table describes the elements you can define within the socket-address element.
| Element |
Required/Optional |
Description |
| <address> |
Required |
Specifies the IP address that a Socket will listen or publish on.
Note: The localhost setting may not work on systems that define localhost as the loopback address; in that case, specify the machine name or the specific IP address. |
| <port> |
Required |
Specifies the port that the Socket will listen or publish on.
Legal values are from 1 to 65535. |
tcp-ring-listener
Used in: cluster-config.
Description
The TCP-ring provides a means for fast death detection of another node within the cluster. When enabled the cluster nodes form a single "ring" of TCP connections spanning the entire cluster. A cluster node is able to utilize the TCP connection to detect the death of another node within a heartbeat interval (default one second). If disabled the cluster node must rely on detecting that another node has stopped responding to UDP packets for a considerately longer interval. Once the death has been detected it is communicated to all other cluster nodes.
Elements
The following table describes the elements you can define within the tcp-ring-listener element.
| Element |
Required/Optional |
Description |
| <enabled> |
Required |
Specifies whether the tcp ring listener should be enabled to defect node failures faster.
Legal values are true and false.
Default value is true.
Preconfigured override is tangosol.coherence.tcpring |
| <maximum-socket-closed-exceptions> |
Required |
Specifies the maximum number of tcp ring listener exceptions that will be tolerated before a particular member is considered really gone and is removed from the cluster.
This value is used only if the value of tcp-ring-listener/enabled is true.
Legal values are integers greater than zero.
Default value is 2. |
| <priority> |
Required |
Specifies a priority of the tcp ring listener execution thread.
Legal values are from 1 to 10.
Default value is 6. |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
traffic-jam
Used in: packet-publisher.
Description
The traffic-jam element is used to control the rate at which client threads enqueue packets for the Packet publisher to transmit on the network. Once the limit is exceeded any client thread will be forced to pause until the number of pending packets drops below the specified limit. To limit the rate at which the Publisher transmits packets see the burst-mode element.
Tuning
Specifying a limit which is to low, or a pause which is to long may result in the publisher transmitting all pending packets, and being left without packets to send. An ideal value will ensure that the publisher is never left without work to do, but at the same time prevent the queue from growing uncontrollably. It is therefore recommended that the pause remain quite short (singles of milliseconds), and that the limit on the number of packets be kept high (i.e. > 5000).
Elements
The following table describes the elements you can define within the traffic-jam element.
| Element |
Required/Optional |
Description |
| <maximum-packets> |
Required |
Specifies the maximum number of pending packets that the Publisher will tolerate before determining that it is clogged and must slow down client requests (requests from local non-system threads). Zero means no limit. This property prevents most unexpected out-of-memory conditions by limiting the size of the resend queue.
Default value is 8192. |
| <pause-milliseconds> |
Required |
Number of milliseconds that the Publisher will pause a client thread that is trying to send a message when the Publisher is clogged. The Publisher will not allow the message to go through until the clog is gone, and will repeatedly sleep the thread for the duration specified by this property.
Default value is 4. |
unicast-listener
Used in: cluster-config.
Description
Specifies the configuration information for the Unicast listener. This element is used to specify the address and port that a cluster node will bind to, in order to listen for point-to-point cluster communications.
Automatic Address Settings
By default Coherence will attempt to obtain the IP to bind to using the java.net.InetAddress.getLocalHost() call. On machines with multiple IPs or NICs you may need to explicitly specify the address. Additionally if the specified port is already in use, Coherence will by default auto increment the port number until the binding succeeds.
Multicast-Free Clustering
By default Coherence uses a multicast protocol to discover other nodes when forming a cluster. If multicast networking is undesirable, or unavailable in your environment, the well-known-addresses feature may be used to eliminate the need for multicast traffic. Note: Use of the Well Known Addresses (WKA) feature requires a Coherence Enterprise License. If you are having difficulties in establishing a cluster via multicast, see the Multicast Test.
Elements
The following table describes the elements you can define within the unicast-listener element.
| Element |
Required/Optional |
Description |
| <well-known-addresses> |
Optional |
Contains a list of "well known" addresses (WKA) that are used by the cluster discovery protocol in place of multicast broadcast. |
| <machine-id> |
Required |
Specifies an identifier that should uniquely identify each server machine. If not specified, a default value is generated from the address of the default network interface.
The machine id for each machine in the cluster can be used by cluster services to plan for failover by making sure that each member is backed up by a member running on a different machine. |
| <address> |
Required |
Specifies the IP address that a Socket will listen or publish on.
Note: The localhost setting may not work on systems that define localhost as the loopback address; in that case, specify the machine name or the specific IP address.
Default value is localhost.
Preconfigured override is tangosol.coherence.localhost |
| <port> |
Required |
Specifies the port that the Socket will listen or publish on.
Legal values are from 1 to 65535.
Default value is 8088.
Preconfigured override is tangosol.coherence.localport |
| <port-auto-adjust> |
Required |
Specifies whether or not the unicast port will be automatically incremented if the specified port cannot be bound to because it is already in use.
Legal values are true or false.
It is recommended that this value be configured to false for production environments.
Default value is true.
Preconfigured override is tangosol.coherence.localport.adjust |
| <ignore-socket-closed> |
Required |
Specifies whether or not the unicast listener will ignore socket exceptions that indicate that a Member is unreachable. Ignoring these exceptions may make the cluster more fault tolerant, but the result will be that unexpected Member deaths will have a detection lag equal to the value of packet-publisher/packet-delivery/resend-milliseconds element.
Legal values are true or false.
Default value is true. |
| <maximum-socket-closed-exceptions> |
Required |
Specifies the maximum number of unicast listener exceptions that will be tolerated before a particular member is considered really gone and is removed from the cluster.
This value is used only if the value of unicast-listener/ignore-socket-closed is false.
Legal values are integers greater than zero.
Default value is 4. |
| <packet-buffer> |
Required |
Specifies how many incoming packets the OS will be requested to buffer. |
| <priority> |
Required |
Specifies a priority of the unicast listener execution thread.
Legal values are from 1 to 10.
Default value is 8. |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
well-known-addresses
Used in: unicast-listener.
Description
Specifies a list of "well known" addresses to use in forming a cluster.
By default Coherence uses a multicast protocol to discover other nodes when forming a cluster. If multicast networking is undesirable, or unavailable in your environment, the Well Known Addresses feature may be used to eliminate the need for multicast traffic.
| Use of the Well Known Addresses (WKA) feature requires a Coherence Enterprise License. |
If you are having difficulties in establishing a cluster via multicast, see the Multicast Test.
Elements
The following table describes the elements you can define within the well-known-addresses element.
| Element |
Required/Optional |
Description |
| <socket-address> |
Required |
Specifies a list of "well known" addresses (WKA) that are used by the cluster discovery protocol in place of multicast broadcast. If one or more WKA is specified, for a member to join the cluster it will either have to be a WKA or there will have to be at least one WKA member running. Additionally, all cluster communication will be performed using unicast. If empty or unspecified multicast communications will be used.
Preconfigured overrides are tangosol.coherence.wka and tangosol.coherence.wka.port. |
The content override attribute xml-override can be optionally used to fully or partially override the contents of this element with XML document that is external to the base document.
Parameter Setttings
Parameter Settings for the Coherence Operational Configuration deployment descriptor init-param Element
This section describes the possible predefined parameter settings for the init-param element in a number of elements where parameters may be specified.
In the following tables, Parameter Name column refers to the value of the param-name element and Value Description column refers to the possible values for the corresponding param-value element.
For example when you see:
| Parameter Name |
Value Description |
| local-storage |
Specifies whether or not this member of the DistributedCache service enables the local storage.
Legal values are true or false.
Default value is true.
Preconfigured override is tangosol.coherence.distributed.localstorage |
it means that the init-params element may look as follows
<init-params>
<init-param>
<param-name>local-storage</param-name>
<param-value>false</param-value>
</init-param>
</init-params>
or as follows:
<init-params>
<init-param>
<param-name>local-storage</param-name>
<param-value>true</param-value>
</init-param>
</init-params>
Parameters
Used in: init-param.
The following table describes the specific parameter <param-name> - <param-value> pairs that can be configured for various elements.
ReplicatedCache Service Parameters
Description
ReplicatedCache service elements support the following parameters:
These settings may also be specified as part of the replicated-scheme element in the cache configuration descriptor.
Parameters
| Parameter Name |
Value Description |
| standard-lease-milliseconds |
Specifies the duration of the standard lease in milliseconds. Once a lease has aged past this number of milliseconds, the lock will automatically be released. Set this value to zero to specify a lease that never expires. The purpose of this setting is to avoid deadlocks or blocks caused by stuck threads; the value should be set higher than the longest expected lock duration (e.g. higher than a transaction timeout). It's also recommended to set this value higher then packet-delivery/timeout-milliseconds value.
Legal values are from positive long numbers or zero.
Default value is 0. |
| lease-granularity |
Specifies the lease ownership granularity. Available since release 2.3.
Legal values are:
A value of thread means that locks are held by a thread that obtained them and can only be released by that thread. A value of member means that locks are held by a cluster node and any thread running on the cluster node that obtained the lock can release it.
Default value is thread. |
| mobile-issues |
Specifies whether or not the lease issues should be transfered to the most recent lock holders.
Legal values are true or false.
Default value is false. |
DistributedCache Service Parameters
Description
DistributedCache service elements support the following parameters:
These settings may also be specified as part of the distributed-scheme element in the cache configuration descriptor.
Parameters
| Parameter Name |
Value Description |
| thread-count |
Specifies the number of daemon threads used by the distributed cache service.
If zero, all relevant tasks are performed on the service thread.
Legal values are from positive integers or zero.
Default value is 0.
Preconfigured override is tangosol.coherence.distributed.threads |
| standard-lease-milliseconds |
Specifies the duration of the standard lease in milliseconds. Once a lease has aged past this number of milliseconds, the lock will automatically be released. Set this value to zero to specify a lease that never expires. The purpose of this setting is to avoid deadlocks or blocks caused by stuck threads; the value should be set higher than the longest expected lock duration (e.g. higher than a transaction timeout). It's also recommended to set this value higher then packet-delivery/timeout-milliseconds value.
Legal values are from positive long numbers or zero.
Default value is 0. |
| lease-granularity |
Specifies the lease ownership granularity. Available since release 2.3.
Legal values are:
A value of thread means that locks are held by a thread that obtained them and can only be released by that thread. A value of member means that locks are held by a cluster node and any thread running on the cluster node that obtained the lock can release it.
Default value is thread. |
| transfer-threshold |
Specifies the threshold for the primary buckets distribution in kilo-bytes. When a new node joins the distributed cache service or when a member of the service leaves, the remaining nodes perform a task of bucket ownership re-destribution. During this process, the existing data gets re-balanced along with the ownership information. This parameter indicates a preferred message size for data transfer communications. Setting this value lower will make the distribution process take longer, but will reduce network bandwith utilization during this activity.
Legal values are integers greater then zero.
Default value is 512 (0.5MB). |
| partition-count |
Specifies the number of partitions that a distributed cache will be "chopped up" into. Each member running the distributed cache service that has the local-storage option set to true will manage a "fair" (balanced) number of partitions. The number of partitions should be larger than the square of the number of cluster members to achieve a good balance, and it is suggested that the number be prime. Good defaults include 257 and 1021 and prime numbers in-between, depending on the expected cluster size. A list of first 1,000 primes can be found at http://www.utm.edu/research/primes/lists/small/1000.txt
Legal values are prime numbers.
Default value is 257. |
| local-storage |
Specifies whether or not this member of the DistributedCache service enables the local storage.
| Normally this value should be left unspecified within the configuration file, and instead set on a per-process basis using the tangosol.coherence.distributed.localstorage system property. This allows cache clients and servers to use the same configuration descriptor. |
Legal values are true or false.
Default value is true.
Preconfigured override is tangosol.coherence.distributed.localstorage |
| backup-count |
Specifies the number of members of the DistributedCache service that hold the backup data for each unit of storage in the cache.
Value of 0 means that in the case of abnormal termination, some portion of the data in the cache will be lost. Value of N means that if up to N cluster nodes terminate at once, the cache data will be preserved.
To maintain the distributed cache of size M, the total memory usage in the cluster does not depend on the number of cluster nodes and will be in the order of M*(N+1).
Recommended values are 0, 1 or 2.
Default value is 1. |
| backup-storage/type |
Specifies the type of the storage used to hold the backup data.
Legal values are:
- on-heap - The corresponding implementations class is java.util.HashMap.
- off-heap - The corresponding implementations class is com.tangosol.util.nio.BinaryMap using {com.tangosol.util.nio.DirectBufferManager}}. Only available with JDK 1.4 and later.
- file-mapped - The corresponding implementations class is com.tangosol.util.nio.BinaryMap using com.tangosol.util.nio.MappedBufferManager. Only available with JDK 1.4 and later.
- custom - The corresponding implementations class is the class specified by the backup-storage/class element.
- scheme - The corresponding implementations class is the map returned by the ConfigurableCacheFactory for the scheme referred to by the backup-storage/scheme-name element.
Default value is on-heap.
Preconfigured override is tangosol.coherence.distributed.backup |
| backup-storage/initial-size |
Only applicable with the off-heap and file-mapped types.
Specifies the initial buffer size in bytes.
The value of this element must be in the following format:
[\d]+[[.][\d]]?[K|k|M|m|G|g|T|t]?[B|b]?
where the first non-digit (from left to right) indicates the factor with which the preceeding decimal value should be multiplied:
- K or k (kilo, 210)
- M or m (mega, 220)
- G or g (giga, 230)
- T or t (tera, 240)
If the value does not contain a factor, a factor of mega is assumed.
Legal values are positive integers between 1 and Integer.MAX_VALUE - 1023.
Default value is 1MB. |
| backup-storage/maximum-size |
Only applicable with the off-heap and file-mapped types.
Specifies the maximum buffer size in bytes.
The value of this element must be in the following format:
[\d]+[[.][\d]]?[K|k|M|m|G|g|T|t]?[B|b]?
where the first non-digit (from left to right) indicates the factor with which the preceeding decimal value should be multiplied:
- K or k (kilo, 210)
- M or m (mega, 220)
- G or g (giga, 230)
- T or t (tera, 240)
If the value does not contain a factor, a factor of mega is assumed.
Legal values are positive integers between 1 and Integer.MAX_VALUE - 1023.
Default value is 1024MB. |
| backup-storage/directory |
Only applicable with the file-mapped type.
Specifies the pathname for the directory that the disk persistence manager (com.tangosol.util.nio.MappedBufferManager) will use as "root" to store files in. If not specified or specifies a non-existent directory, a temporary file in the default location is used.
Default value is the default temporary directory designated by the Java runtime. |
| backup-storage/class-name |
Only applicable with the custom type.
Specifies a class name for the custom storage implementation. If the class implements com.tangosol.run.xml.XmlConfigurable interface then upon construction the setConfig method is called passing the entire backup-storage element. |
| backup-storage/scheme-name |
Only applicable with the scheme type.
Specifies a scheme name for the ConfigurableCacheFactory. |
| key-associator/class-name |
Specifies the name of a class that implements the com.tangosol.net.partition.KeyAssociator interface. This implementation must have a zero-parameter public constructor. |
| key-partitioning/class-name |
Specifies the name of a class that implements the com.tangosol.net.partition.KeyPartitioningStrategy interface. This implementation must have a zero-parameter public constructor. |
InvocationService Parameters
Description
InvocationService service elements support the following parameters:
These settings may also be specified as part of the invocation-scheme element in the cache configuration descriptor.
Parameters
| Parameter Name |
Value Description |
| thread-count |
Specifies the number of daemon threads to be used by the invocation service.
If zero, all relevant tasks are performed on the service thread.
Legal values are from positive integers or zero.
Default value is 0. |
Preconfigured override is tangosol.coherence.invocation.threads
Compression Filter Parameters
The compression filter
com.tangosol.net.CompressionFilter, supports the following parameters (see java.util.zip.Deflater for details):
Parameters
| Parameter Name |
Value Description |
| buffer-length |
Spesifies compression buffer length in bytes.
Legal values are from positive integers or zero.
Default value is 0. |
| strategy |
Specifies the compressions strategy.
Legal values are:
- gzip
- huffman-only
- filtered
- default
Default value is gzip. |
| level |
Specifies the compression level.
Legal values are:
- default
- compression
- speed
- none
Default value is default. |
Element Attributes
Element Attributes
The following table describes the attributes that can be used with some of the elements described above.
Used in: coherence, cluster-config, logging-config, configurable-cache-factory-config, unicast-listener, multicast-listener, tcp-ring-listener, shutdown-listener, packet-publisher, incoming-message-handler, authorized-hosts, host-range, services, filters, filter-name, init-param (operational).
| Attribute |
Required/Optional |
Description |
| xml-override |
Optional |
Allows the content of this elements to be fully or partially overridden with XML documents that are external to the base document.
Legal value of this attribute is the resource name of such an override document that should be accessible using the ClassLoader.getResourceAsStream(String name) by the classes contained in coherence.jar library. In general that means that resource name should be prefixed with '/' and located in the classpath.
The override XML document referred by this attribute does not have to exist. However, if it does exist then its root element must have the same name as the element it overrides.
In cases where there are multiple elements with the same name (e.g. <services>) the id attribute should be used to identify the base element that will be overridden as well as the override element itself. The elements of the override document that do not have a match in the base document are just appended to the base. |
| id |
Optional |
Used in conjunciton with the xml-override attribute in cases where there are multiple elements with the same name (e.g. <services>) to identify the base element that will be overridden as well as the override element itself. The elements of the override document that do not have a match in the base document are just appended to the base. |
Command Line Setting Override Feature
Command Line Setting Override Feature
Both the Coherence Operational Configuration deployment descriptor and the Coherence Cache Configuration deployment descriptor support the ability to assign java command line option name to any element defined in the descriptor. Some of the elements already have these Command Line Setting Overrides defined. You can create your own or change the ones that are already defined.
This feature is very useful when you need to change the settings just for a single JVM, or be able to start different applications with different settings without making them use different descriptors. The most commonplace application is passing different multicast address and/or port to allow different applications to create separate clusters.
To create a Commmand Line Setting Override all you need to do is add system-property attribute specifying the string you would like to assign as the name for the java command line option to the element you want to create an override to. Then you just need to specify it in the java command line prepended with "-D".
For example:
Let's say that we want to create an override for the IP address of the multi-home server to avoid using the default localhost, and instead specify a specific the IP address of the interface we want Coherence to use (let's say it is 192.168.0.301). We would like to call this override tangosol.coherence.localhost.
In order to do that we first add a system-property to the cluster-config/unicast-listener/address element:
<address>localhost</address>
which will look as follows with the property we added:
<address system-property="tangosol.coherence.localhost">localhost</address>
Then we use it by modifying our java command line:
