Scylla Documentation Logo Documentation
  • Server
    • Scylla Open Source
    • Scylla Enterprise
    • Scylla Alternator
  • Cloud
    • Scylla Cloud
    • Scylla Cloud Docs
  • Tools
    • Scylla Manager
    • Scylla Monitoring Stack
    • Scylla Operator
  • Drivers
    • CQL Drivers
    • DynamoDB Drivers
Download
Menu

Caution

You're viewing documentation for a previous version of Scylla Java Driver. Switch to the latest stable version.

Scylla Java Driver Manual Developer docs Common infrastructure Concurrency

Concurrency¶

The driver is a highly concurrent environment. We try to use thread confinement to simplify the code, when that does not impact performance.

Hot path¶

The hot path is everything that happens for a session.execute call. In a typical client application, this is where the driver will likely spend the majority of its time, so it must be fast.

Write path:

  1. convert the statement into a protocol-level Message (CqlRequestHandler constructor);

  2. find a node and a connection, and write the message to it (CqlRequestHandler.sendRequest);

  3. assign a stream id and wrap the message into a frame (InflightHandler.write);

  4. encode the frame into a binary payload (FrameEncoder).

Read path:

  1. decode the binary payload into a frame (FrameDecoder);

  2. find the handler that corresponds to the stream id (InFlightHandler.channelRead);

  3. complete the client’s future (CqlRequestHandler.NodeResponseCallback.onResponse).

Various policies are also invoked along the way (load balancing, retry, speculative execution, timestamp generator…), they are considered on the hot path too.

Steps 1 and 2 of the write path happen on the client thread, and 3 and 4 on the Netty I/O thread (which is one of the threads in NettyOptions.ioEventLoopGroup()). On the read path, everything happens on the Netty I/O thread. Beyond that, we want to avoid context switches for performance reasons: in early prototypes, we tried confining CqlRequestHandler to a particular thread, but that did not work well; so you will find that the code is fairly similar to driver 3 in terms of concurrency control (reliance on atomic structures, volatile fields, etc).

Note: code on the hot path should prefer the TRACE log level.

Cold path¶

The cold path is everything else: initialization and shutdown, metadata refreshes, tracking node states, etc. They will typically be way less frequent than user requests, so we can tolerate a small performance hit in order to make concurrency easier to handle.

One pattern we use a lot is a confined inner class:

public class ControlConnection {
  // some content omitted for brevity
  
  private final EventExecutor adminExecutor;
  private final SingleThreaded singleThreaded;
  
  // Called from other components, from any thread
  public void reconnectNow() {
    RunOrSchedule.on(adminExecutor, singleThreaded::reconnectNow);
  }
  
  private class SingleThreaded {
    private void reconnectNow() {
      assert adminExecutor.inEventLoop();
      // this method is only ever called from one thread, much easier to handle concurrency
    }
  }
}

Public outer methods such as reconnectNow() are called concurrently. But they delegate to a method of the internal class, that always runs on the same adminExecutor thread. RunOrSchedule.on calls the method directly if we’re already on the target thread, otherwise it schedules a task. If we need to propagate a result, the outer method injects a future that the inner method completes.

adminExecutor is picked randomly from NettyOptions.adminEventExecutorGroup() at construction time.

Confining SingleThreaded simplifies the code tremendously: we can use regular, non-volatile fields, and methods are guaranteed to always run in isolation, eliminating subtle race conditions (this idea was borrowed from actor systems).

Non-blocking¶

Whether on the hot or cold path, internal code is almost 100% lock-free. The driver guarantees on lock-freedom are detailed in the core manual.

If an internal component needs to execute a query, it does so asynchronously, and registers callbacks to process the results. Examples of this can be found in ReprepareOnUp and DefaultTopologyMonitor (among others).

The only place where the driver blocks is when using the synchronous API (methods declared in SyncCqlSession), and when calling other synchronous wrapper methods in the public API, for example, ExecutionInfo.getQueryTrace():

public interface ExecutionInfo {
  // some content omitted for brevity
  
  default QueryTrace getQueryTrace() {
    BlockingOperation.checkNotDriverThread();
    return CompletableFutures.getUninterruptibly(getQueryTraceAsync());
  }
}

When a public API method is blocking, this is generally clearly stated in its javadocs.

BlockingOperation is a utility to check that those methods aren’t called on I/O threads, which could introduce deadlocks.

Keeping the internals fully asynchronous is another major improvement over driver 3, where internal requests were synchronous, and required multiple internal executors to avoid deadlocks.

In driver 4, there are only two executors: NettyOptions.ioEventLoopGroup() and NettyOptions.adminEventLoopGroup(), that are guaranteed to never run blocking tasks. They can be shared with application code, or across multiple sessions, or can even be one and the same (in theory, it’s possible to use a single 1-thread executor, although there’s probably no practical reason to do that).

To be exhaustive, NettyOptions.getTimer() also uses its own thread; we tried scheduling request timeouts and speculative executions on I/O threads in early alphas, but that didn’t perform as well as Netty’s HashedWheelTimer.

So the total number of threads created by a session is

advanced.netty.io-group.size + advanced.netty.admin-group.size + 1
PREVIOUS
Common infrastructure
NEXT
Driver context
  • 4.11.1.x
    • 4.13.0.x
    • 4.12.0.x
    • 4.11.1.x
    • 4.10.0.x
    • 4.7.2.x
    • 3.11.2.x
    • 3.11.0.x
    • 3.10.2.x
    • 3.7.2.x
  • Java Driver for Scylla and Apache Cassandra®
  • API Documentation
  • Manual
    • API conventions
    • Case sensitivity
    • Core driver
      • Address resolution
      • Asynchronous programming
      • Authentication
      • Bill of Materials (BOM)
      • Compression
      • Configuration
        • Reference configuration
      • Control connection
      • Custom codecs
      • Detachable types
      • Query idempotence
      • Integration
      • Load balancing
      • Logging
      • Metadata
        • Node metadata
        • Schema metadata
        • Token metadata
      • Metrics
      • Native protocol
      • Non-blocking programming
      • Paging
      • Performance
      • Connection pooling
      • Query timestamps
      • Reactive Style Programming
      • Reconnection
      • Request tracker
      • Retries
      • Using the shaded JAR
      • Speculative query execution
      • SSL
      • Statements
        • Batch statements
        • Per-query keyspace
        • Prepared statements
        • Simple statements
      • Temporal types
      • Request throttling
      • Query tracing
      • Tuples
      • User-defined types
    • Developer docs
      • Administrative tasks
      • Common infrastructure
        • Concurrency
        • Driver context
        • Event bus
      • Native protocol layer
      • Netty pipeline
      • Request execution
    • Mapper
      • Integration
        • Kotlin
        • Lombok
        • Java 14 Records
        • Scala
      • DAOs
        • Custom result types
        • Delete methods
        • GetEntity methods
        • Increment methods
        • Insert methods
        • Null saving strategy
        • Query methods
        • Query provider methods
        • Select methods
        • SetEntity methods
        • Statement attributes
        • Update methods
      • Entities
      • Mapper interface
    • OSGi
    • Query builder
      • Conditions
      • DELETE
      • Idempotence in the query builder
      • INSERT
      • Relations
      • Schema builder
        • Aggregate
        • Function
        • Index
        • Keyspace
        • Materialized View
        • Table
        • Type
      • SELECT
      • Terms
      • TRUNCATE
      • UPDATE
  • Upgrade guide
  • Frequently asked questions
  • Changelog
  • Create an issue
  • Edit this page

On this page

  • Concurrency
    • Hot path
    • Cold path
    • Non-blocking
Logo
Docs Contact Us About Us
Mail List Icon Slack Icon
© 2022, ScyllaDB. All rights reserved.
Last updated on 25 May 2022.
Powered by Sphinx 4.3.2 & ScyllaDB Theme 1.2.2