BlobStoreRepository (server 6.3.2 API)

java.lang.Object
- org.elasticsearch.common.component.AbstractComponent
- - org.elasticsearch.common.component.AbstractLifecycleComponent
  - - org.elasticsearch.repositories.blobstore.BlobStoreRepository

All Implemented Interfaces:: java.io.Closeable, java.lang.AutoCloseable, LifecycleComponent, Releasable, Repository

Direct Known Subclasses:: FsRepository

public abstract class BlobStoreRepository
extends AbstractLifecycleComponent
implements Repository

BlobStore - based implementation of Snapshot Repository

This repository works with any BlobStore implementation. The blobStore should be initialized in the derived class before doStart() is called.

BlobStoreRepository maintains the following structure in the blob store

 
   STORE_ROOT
   |- index-N           - list of all snapshot ids and the indices belonging to each snapshot, N is the generation of the file
   |- index.latest      - contains the numeric value of the latest generation of the index file (i.e. N from above)
   |- incompatible-snapshots - list of all snapshot ids that are no longer compatible with the current version of the cluster
   |- snap-20131010 - JSON serialized Snapshot for snapshot "20131010"
   |- meta-20131010.dat - JSON serialized MetaData for snapshot "20131010" (includes only global metadata)
   |- snap-20131011 - JSON serialized Snapshot for snapshot "20131011"
   |- meta-20131011.dat - JSON serialized MetaData for snapshot "20131011"
   .....
   |- indices/ - data for all indices
      |- Ac1342-B_x/ - data for index "foo" which was assigned the unique id of Ac1342-B_x in the repository
      |  |- meta-20131010.dat - JSON Serialized IndexMetaData for index "foo"
      |  |- 0/ - data for shard "0" of index "foo"
      |  |  |- __1 \
      |  |  |- __2 |
      |  |  |- __3 |- files from different segments see snapshot-* for their mappings to real segment files
      |  |  |- __4 |
      |  |  |- __5 /
      |  |  .....
      |  |  |- snap-20131010.dat - JSON serialized BlobStoreIndexShardSnapshot for snapshot "20131010"
      |  |  |- snap-20131011.dat - JSON serialized BlobStoreIndexShardSnapshot for snapshot "20131011"
      |  |  |- list-123 - JSON serialized BlobStoreIndexShardSnapshot for snapshot "20131011"
      |  |
      |  |- 1/ - data for shard "1" of index "foo"
      |  |  |- __1
      |  |  .....
      |  |
      |  |-2/
      |  ......
      |
      |- 1xB0D8_B3y/ - data for index "bar" which was assigned the unique id of 1xB0D8_B3y in the repository
      ......

Nested Class Summary
- Nested classes/interfaces inherited from interface org.elasticsearch.repositories.Repository
  Repository.Factory

Field Summary

Fields
Modifier and Type Field Description

protected RepositoryMetaData metadata

protected NamedXContentRegistry namedXContentRegistry
- Fields inherited from class org.elasticsearch.common.component.AbstractComponent
  deprecationLogger, logger, settings
- Fields inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent
  lifecycle

Fields
Modifier and Type	Field	Description
`protected RepositoryMetaData`	`metadata`
`protected NamedXContentRegistry`	`namedXContentRegistry`

Constructor Summary

Constructors
Modifier	Constructor	Description
`protected`	`BlobStoreRepository(RepositoryMetaData metadata, Settings globalSettings, NamedXContentRegistry namedXContentRegistry)`	Constructs new BlobStoreRepository

Method Summary

All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method	Description
`protected abstract BlobPath`	`basePath()`	Returns base path of the repository
`protected abstract BlobStore`	`blobStore()`	Returns the BlobStore to read and write data.
`protected ByteSizeValue`	`chunkSize()`	Returns data file chunk size.
`void`	`deleteSnapshot(SnapshotId snapshotId, long repositoryStateId)`	Deletes snapshot
`protected void`	`doClose()`
`protected void`	`doStart()`
`protected void`	`doStop()`
`void`	`endVerification(java.lang.String seed)`	Called at the end of repository verification process.
`SnapshotInfo`	`finalizeSnapshot(SnapshotId snapshotId, java.util.List<IndexId> indices, long startTime, java.lang.String failure, int totalShards, java.util.List<SnapshotShardFailure> shardFailures, long repositoryStateId, boolean includeGlobalState)`	Finalizes snapshotting process
`RepositoryMetaData`	`getMetadata()`	Returns metadata about this repository.
`RepositoryData`	`getRepositoryData()`	Returns a `RepositoryData` to describe the data in the repository, including the snapshots and the indices across all snapshots found in the repository.
`long`	`getRestoreThrottleTimeInNanos()`	Returns restore throttle time in nanoseconds
`IndexShardSnapshotStatus`	`getShardSnapshotStatus(SnapshotId snapshotId, Version version, IndexId indexId, ShardId shardId)`	Retrieve shard snapshot status for the stored snapshot
`SnapshotInfo`	`getSnapshotInfo(SnapshotId snapshotId)`	Reads snapshot description from repository.
`MetaData`	`getSnapshotMetaData(SnapshotInfo snapshot, java.util.List<IndexId> indices)`	Returns global metadata associate with the snapshot.
`long`	`getSnapshotThrottleTimeInNanos()`	Returns snapshot throttle time in nanoseconds
`void`	`initializeSnapshot(SnapshotId snapshotId, java.util.List<IndexId> indices, MetaData clusterMetaData)`	Starts snapshotting process
`protected boolean`	`isCompress()`	Returns true if metadata and snapshot files should be compressed
`boolean`	`isReadOnly()`	Returns true if the repository supports only read operations
`void`	`restoreShard(IndexShard shard, SnapshotId snapshotId, Version version, IndexId indexId, ShardId snapshotShardId, RecoveryState recoveryState)`	Restores snapshot of the shard.
`void`	`snapshotShard(IndexShard shard, SnapshotId snapshotId, IndexId indexId, org.apache.lucene.index.IndexCommit snapshotIndexCommit, IndexShardSnapshotStatus snapshotStatus)`	Creates a snapshot of the shard based on the index commit point.
`java.lang.String`	`startVerification()`	Verifies repository on the master node and returns the verification token.
`static java.lang.String`	`testBlobPrefix(java.lang.String seed)`
`java.lang.String`	`toString()`
`void`	`verify(java.lang.String seed, DiscoveryNode localNode)`	Verifies repository settings on data node.
`protected void`	`writeIndexGen(RepositoryData repositoryData, long repositoryStateId)`

Methods inherited from class org.elasticsearch.common.component.AbstractComponent
logDeprecatedSetting, logRemovedSetting, nodeName

Methods inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent
addLifecycleListener, close, lifecycleState, removeLifecycleListener, start, stop

Methods inherited from interface org.elasticsearch.common.component.LifecycleComponent
addLifecycleListener, lifecycleState, removeLifecycleListener, start, stop

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.elasticsearch.common.lease.Releasable
close

- Field Detail
  - metadata
```
protected final RepositoryMetaData metadata
```
  - namedXContentRegistry
```
protected final NamedXContentRegistry namedXContentRegistry
```
- Constructor Detail
  - BlobStoreRepository
```
protected BlobStoreRepository(RepositoryMetaData metadata,
                              Settings globalSettings,
                              NamedXContentRegistry namedXContentRegistry)
```
    Constructs new BlobStoreRepository
    
    Parameters:
    
    metadata - The metadata for this repository including name and settings
    
    globalSettings - Settings for the node this repository object is created on
- Method Detail
  - doStart
```
protected void doStart()
```
    Specified by:
    
    doStart in class AbstractLifecycleComponent
  - doStop
```
protected void doStop()
```
    Specified by:
    
    doStop in class AbstractLifecycleComponent
  - doClose
```
protected void doClose()
```
    Specified by:
    
    doClose in class AbstractLifecycleComponent
  - blobStore
```
protected abstract BlobStore blobStore()
```
    Returns the BlobStore to read and write data.
  - basePath
```
protected abstract BlobPath basePath()
```
    Returns base path of the repository
  - isCompress
```
protected boolean isCompress()
```
    Returns true if metadata and snapshot files should be compressed
    
    Returns:
    
    true if compression is needed
  - chunkSize
```
protected ByteSizeValue chunkSize()
```
    Returns data file chunk size.
    This method should return null if no chunking is needed.
    
    Returns:
    
    chunk size
  - getMetadata
```
public RepositoryMetaData getMetadata()
```
    Description copied from interface: Repository
    
    Returns metadata about this repository.
    
    Specified by:
    
    getMetadata in interface Repository
  - initializeSnapshot
```
public void initializeSnapshot(SnapshotId snapshotId,
                               java.util.List<IndexId> indices,
                               MetaData clusterMetaData)
```
    Description copied from interface: Repository
    
    Starts snapshotting process
    
    Specified by:
    
    initializeSnapshot in interface Repository
    
    Parameters:
    
    snapshotId - snapshot id
    
    indices - list of indices to be snapshotted
    
    clusterMetaData - cluster metadata
  - deleteSnapshot
```
public void deleteSnapshot(SnapshotId snapshotId,
                           long repositoryStateId)
```
    Description copied from interface: Repository
    
    Deletes snapshot
    
    Specified by:
    
    deleteSnapshot in interface Repository
    
    Parameters:
    
    snapshotId - snapshot id
    
    repositoryStateId - the unique id identifying the state of the repository when the snapshot deletion began
  - finalizeSnapshot
```
public SnapshotInfo finalizeSnapshot(SnapshotId snapshotId,
                                     java.util.List<IndexId> indices,
                                     long startTime,
                                     java.lang.String failure,
                                     int totalShards,
                                     java.util.List<SnapshotShardFailure> shardFailures,
                                     long repositoryStateId,
                                     boolean includeGlobalState)
```
    Finalizes snapshotting process
    This method is called on master after all shards are snapshotted.
    
    Specified by:
    
    finalizeSnapshot in interface Repository
    
    Parameters:
    
    snapshotId - snapshot id
    
    indices - list of indices in the snapshot
    
    startTime - start time of the snapshot
    
    failure - global failure reason or null
    
    totalShards - total number of shards
    
    shardFailures - list of shard failures
    
    repositoryStateId - the unique id identifying the state of the repository when the snapshot began
    
    includeGlobalState - include cluster global state
    
    Returns:
    
    snapshot description
  - getSnapshotMetaData
```
public MetaData getSnapshotMetaData(SnapshotInfo snapshot,
                                    java.util.List<IndexId> indices)
                             throws java.io.IOException
```
    Description copied from interface: Repository
    
    Returns global metadata associate with the snapshot.
    The returned meta data contains global metadata as well as metadata for all indices listed in the indices parameter.
    
    Specified by:
    
    getSnapshotMetaData in interface Repository
    
    Parameters:
    
    snapshot - snapshot
    
    indices - list of indices
    
    Returns:
    
    information about snapshot
    
    Throws:
    
    java.io.IOException
  - getSnapshotInfo
```
public SnapshotInfo getSnapshotInfo(SnapshotId snapshotId)
```
    Description copied from interface: Repository
    
    Reads snapshot description from repository.
    
    Specified by:
    
    getSnapshotInfo in interface Repository
    
    Parameters:
    
    snapshotId - snapshot id
    
    Returns:
    
    information about snapshot
  - getSnapshotThrottleTimeInNanos
```
public long getSnapshotThrottleTimeInNanos()
```
    Description copied from interface: Repository
    
    Returns snapshot throttle time in nanoseconds
    
    Specified by:
    
    getSnapshotThrottleTimeInNanos in interface Repository
  - getRestoreThrottleTimeInNanos
```
public long getRestoreThrottleTimeInNanos()
```
    Description copied from interface: Repository
    
    Returns restore throttle time in nanoseconds
    
    Specified by:
    
    getRestoreThrottleTimeInNanos in interface Repository
  - startVerification
```
public java.lang.String startVerification()
```
    Description copied from interface: Repository
    
    Verifies repository on the master node and returns the verification token.
    If the verification token is not null, it's passed to all data nodes for verification. If it's null - no additional verification is required
    
    Specified by:
    
    startVerification in interface Repository
    
    Returns:
    
    verification token that should be passed to all Index Shard Repositories for additional verification or null
  - endVerification
```
public void endVerification(java.lang.String seed)
```
    Description copied from interface: Repository
    
    Called at the end of repository verification process.
    This method should perform all necessary cleanup of the temporary files created in the repository
    
    Specified by:
    
    endVerification in interface Repository
    
    Parameters:
    
    seed - verification request generated by Repository.startVerification() command
  - getRepositoryData
```
public RepositoryData getRepositoryData()
```
    Description copied from interface: Repository
    
    Returns a RepositoryData to describe the data in the repository, including the snapshots and the indices across all snapshots found in the repository. Throws a RepositoryException if there was an error in reading the data.
    
    Specified by:
    
    getRepositoryData in interface Repository
  - testBlobPrefix
```
public static java.lang.String testBlobPrefix(java.lang.String seed)
```
  - isReadOnly
```
public boolean isReadOnly()
```
    Description copied from interface: Repository
    
    Returns true if the repository supports only read operations
    
    Specified by:
    
    isReadOnly in interface Repository
    
    Returns:
    
    true if the repository is read/only
  - writeIndexGen
```
protected void writeIndexGen(RepositoryData repositoryData,
                             long repositoryStateId)
                      throws java.io.IOException
```
    Throws:
    
    java.io.IOException
  - snapshotShard
```
public void snapshotShard(IndexShard shard,
                          SnapshotId snapshotId,
                          IndexId indexId,
                          org.apache.lucene.index.IndexCommit snapshotIndexCommit,
                          IndexShardSnapshotStatus snapshotStatus)
```
    Description copied from interface: Repository
    
    Creates a snapshot of the shard based on the index commit point.
    The index commit point can be obtained by using Engine.acquireLastIndexCommit(boolean) method. Repository implementations shouldn't release the snapshot index commit point. It is done by the method caller.
    As snapshot process progresses, implementation of this method should update IndexShardSnapshotStatus object and check IndexShardSnapshotStatus.isAborted() to see if the snapshot process should be aborted.
    
    Specified by:
    
    snapshotShard in interface Repository
    
    Parameters:
    
    shard - shard to be snapshotted
    
    snapshotId - snapshot id
    
    indexId - id for the index being snapshotted
    
    snapshotIndexCommit - commit point
    
    snapshotStatus - snapshot status
  - restoreShard
```
public void restoreShard(IndexShard shard,
                         SnapshotId snapshotId,
                         Version version,
                         IndexId indexId,
                         ShardId snapshotShardId,
                         RecoveryState recoveryState)
```
    Description copied from interface: Repository
    
    Restores snapshot of the shard.
    The index can be renamed on restore, hence different shardId and snapshotShardId are supplied.
    
    Specified by:
    
    restoreShard in interface Repository
    
    Parameters:
    
    shard - the shard to restore the index into
    
    snapshotId - snapshot id
    
    version - version of elasticsearch that created this snapshot
    
    indexId - id of the index in the repository from which the restore is occurring
    
    snapshotShardId - shard id (in the snapshot)
    
    recoveryState - recovery state
  - getShardSnapshotStatus
```
public IndexShardSnapshotStatus getShardSnapshotStatus(SnapshotId snapshotId,
                                                       Version version,
                                                       IndexId indexId,
                                                       ShardId shardId)
```
    Description copied from interface: Repository
    
    Retrieve shard snapshot status for the stored snapshot
    
    Specified by:
    
    getShardSnapshotStatus in interface Repository
    
    Parameters:
    
    snapshotId - snapshot id
    
    version - version of elasticsearch that created this snapshot
    
    indexId - the snapshotted index id for the shard to get status for
    
    shardId - shard id
    
    Returns:
    
    snapshot status
  - verify
```
public void verify(java.lang.String seed,
                   DiscoveryNode localNode)
```
    Description copied from interface: Repository
    
    Verifies repository settings on data node.
    
    Specified by:
    
    verify in interface Repository
    
    Parameters:
    
    seed - value returned by Repository.startVerification()
    
    localNode - the local node information, for inclusion in verification errors
  - toString
```
public java.lang.String toString()
```
    Overrides:
    
    toString in class java.lang.Object

Class BlobStoreRepository

Nested Class Summary

Nested classes/interfaces inherited from interface org.elasticsearch.repositories.Repository

Field Summary

Fields inherited from class org.elasticsearch.common.component.AbstractComponent

Fields inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent

Constructor Summary

Method Summary

Methods inherited from class org.elasticsearch.common.component.AbstractComponent

Methods inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent

Methods inherited from interface org.elasticsearch.common.component.LifecycleComponent

Methods inherited from class java.lang.Object

Methods inherited from interface org.elasticsearch.common.lease.Releasable

Field Detail

metadata

namedXContentRegistry

Constructor Detail

BlobStoreRepository

Method Detail

doStart

doStop

doClose

blobStore

basePath

isCompress

chunkSize

getMetadata

initializeSnapshot

deleteSnapshot

finalizeSnapshot

getSnapshotMetaData

getSnapshotInfo

getSnapshotThrottleTimeInNanos

getRestoreThrottleTimeInNanos

startVerification

endVerification

getRepositoryData

testBlobPrefix

isReadOnly

writeIndexGen

snapshotShard

restoreShard

getShardSnapshotStatus

verify

toString