Load balancing¶

Host distance¶

For each host, the policy computes a distance that determines how the driver will establish connections to it:

• LOCAL and REMOTE are “active” distances, meaning that the driver will keep open connections to the host. They differ in the number of connections opened, depending on your pooling options. Also, the control connection will favor local nodes if possible.

• IGNORED, as the name suggests, means that the driver will not attempt to connect.

Typically, the distance will reflect network topology (e.g. local vs. remote datacenter), although that is entirely up to your policy. The distance can be dynamic: the driver re-checks it whenever connection pools are created (e.g. at startup or if a node was down and just came back up); you can also trigger it with refreshConnectedHosts:

// Re-evaluate all host distances:
cluster.getConfiguration().getPoolingOptions().refreshConnectedHosts();

// Re-evaluate the distance for a given host:
cluster.getConfiguration().getPoolingOptions().refreshConnectedHost(host);

Query plan¶

Each time the driver executes a query, it asks the policy to compute a query plan, which is a list of hosts. The driver will then try each host in sequence, according to the retry policy and speculative execution policy.

The contents and order of query plans are entirely up to your policy, but implementations typically return plans that:

• are different for each query, in order to balance the load across the cluster;

• only contain hosts that are known to be able to process queries, i.e. neither ignored nor down;

• favor local hosts over remote ones.

The next sections describe the implementations that are provided with the driver.

Requirements¶

In order for token awareness to work, you should first ensure that metadata is enabled in the driver. That is the case by default, unless it’s been explicitly disabled by QueryOptions#setMetadataEnabled.

Then you need to consider whether routing information (provided by Statement#getKeyspace and Statement#getRoutingKey) can be computed automatically for your statements; if not, you may provide it yourself (if a statement has no routing information, the query will still be executed, but token awareness will not work, so the driver might not pick the best coordinator).

The examples assume the following CQL schema:

CREATE TABLE testKs.sensor_data(id int, year int, ts timestamp, data double,
                                PRIMARY KEY ((id, year), ts));

For simple statements, routing information can never be computed automatically:

SimpleStatement statement = new SimpleStatement(
        "SELECT * FROM testKs.sensor_data WHERE id = 1 and year = 2016");

// No routing info available:
assert statement.getKeyspace() == null;
assert statement.getRoutingKey() == null;

// Set the keyspace manually:
statement.setKeyspace("testKs");

// Set the routing key manually: serialize each partition key component to its target CQL type
ProtocolVersion protocolVersion = cluster.getConfiguration().getProtocolOptions().getProtocolVersionEnum();
statement.setRoutingKey(
        TypeCodec.cint().serialize(1, protocolVersion),
        TypeCodec.cint().serialize(2016, protocolVersion));

session.execute(statement);

For built statements, the keyspace is available if it was provided while building the query; the routing key is available only if the statement was built using the table metadata, and all components of the partition key appear in the query:

TableMetadata tableMetadata = cluster.getMetadata()
        .getKeyspace("testKs")
        .getTable("sensor_data");

// Built from metadata: all info available
BuiltStatement statement1 = select().from(tableMetadata)
        .where(eq("id", 1))
        .and(eq("year", 2016));

assert statement1.getKeyspace() != null;
assert statement1.getRoutingKey() != null;

// Built from keyspace and table name: only keyspace available
BuiltStatement statement2 = select().from("testKs", "sensor")
        .where(eq("id", 1))
        .and(eq("year", 2016));

assert statement2.getKeyspace() != null;
assert statement2.getRoutingKey() == null;

For bound statements, the keyspace is always available; the routing key is only available if all components of the partition key are bound as variables:

// All components bound: all info available
PreparedStatement pst1 = session.prepare("SELECT * FROM testKs.sensor_data WHERE id = :id and year = :year");
BoundStatement statement1 = pst1.bind(1, 2016);

assert statement1.getKeyspace() != null;
assert statement1.getRoutingKey() != null;

// 'id' hard-coded, only 'year' is bound: only keyspace available
PreparedStatement pst2 = session.prepare("SELECT * FROM testKs.sensor_data WHERE id = 1 and year = :year");
BoundStatement statement2 = pst2.bind(2016);

assert statement2.getKeyspace() != null;
assert statement2.getRoutingKey() == null;

For batch statements, the routing information of each child statement is inspected; the first non-null keyspace is used as the keyspace of the batch, and the first non-null routing key as its routing key (the idea is that all childs should have the same routing information, since batches are supposed to operate on a single partition). All children might have null information, in which case you need to provide the information manually as shown previously.

Behavior¶

For any host, the distance returned by TokenAwarePolicy is always the same as its child policy.

When the policy computes a query plan, it will first inspect the statement’s routing information. If there is none, the policy simply acts as a pass-through, and returns the query plan computed by its child policy.

If the statement has routing information, the policy uses it to determine the replicas that hold the corresponding data. Then it returns a query plan containing:

1. the replicas for which the child policy returns distance LOCAL, shuffled in a random order;

2. followed by the query plan of the child policy, skipping any host that were already returned by the previous step.

Finally, the shuffleReplicas constructor parameter allows you to control whether the policy shuffles the replicas in step 1:

new TokenAwarePolicy(anotherPolicy, false); // no shuffling

Shuffling will distribute writes better, and can alleviate hotspots caused by “fat” partitions. On the other hand, setting it to false might increase the effectiveness of caching, since data will always be retrieved from the “primary” replica. Shuffling is enabled by default.