Class PendingClusterStatesQueue


  • public class PendingClusterStatesQueue
    extends java.lang.Object
    A queue that holds all "in-flight" incoming cluster states from the master. Once a master commits a cluster state, it is made available via getNextClusterStateToProcess(). The class also takes care of batching cluster states for processing and failures.

    The queue is bound by maxQueueSize. When the queue is at capacity and a new cluster state is inserted the oldest cluster state will be dropped. This is safe because: 1) Under normal operations, master will publish & commit a cluster state before processing another change (i.e., the queue length is 1) 2) If the master fails to commit a change, it will step down, causing a master election, which will flush the queue. 3) In general it's safe to process the incoming cluster state as a replacement to the cluster state that's dropped. a) If the dropped cluster is from the same master as the incoming one is, it is likely to be superseded by the incoming state (or another state in the queue). This is only not true in very extreme cases of out of order delivery. b) If the dropping cluster state is not from the same master, it means that: i) we are no longer following the master of the dropped cluster state but follow the incoming one ii) we are no longer following any master, in which case it doesn't matter which cluster state will be processed first.

    The class is fully thread safe and can be used concurrently.

    • Constructor Detail

      • PendingClusterStatesQueue

        public PendingClusterStatesQueue​(org.apache.logging.log4j.Logger logger,
                                         int maxQueueSize)
    • Method Detail

      • addPending

        public void addPending​(ClusterState state)
        Add an incoming, not yet committed cluster state
      • markAsCommitted

        public ClusterState markAsCommitted​(java.lang.String stateUUID,
                                            org.elasticsearch.discovery.zen.PendingClusterStatesQueue.StateProcessedListener listener)
        Mark a previously added cluster state as committed. This will make it available via getNextClusterStateToProcess() When the cluster state is processed (or failed), the supplied listener will be called
      • markAsFailed

        public void markAsFailed​(ClusterState state,
                                 java.lang.Exception reason)
        mark that the processing of the given state has failed. All committed states that are ClusterState.supersedes(ClusterState)-ed by this failed state, will be failed as well
      • markAsProcessed

        public void markAsProcessed​(ClusterState state)
        indicates that a cluster state was successfully processed. Any committed state that is ClusterState.supersedes(ClusterState)-ed by the processed state will be marked as processed as well.

        NOTE: successfully processing a state indicates we are following the master it came from. Any committed state from another master will be failed by this method

      • failAllStatesAndClear

        public void failAllStatesAndClear​(java.lang.Exception reason)
        clear the incoming queue. any committed state will be failed
      • getNextClusterStateToProcess

        public ClusterState getNextClusterStateToProcess()
        Gets the next committed state to process.

        The method tries to batch operation by getting the cluster state the highest possible committed states which succeeds the first committed state in queue (i.e., it comes from the same master).

      • pendingClusterStates

        public ClusterState[] pendingClusterStates()
        returns all pending states, committed or not