Package org.elasticsearch.indices

Class IndicesService

java.lang.Object

org.elasticsearch.common.component.AbstractLifecycleComponent

org.elasticsearch.indices.IndicesService

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<IndexService>, LifecycleComponent, Releasable, IndexService.ShardStoreDeleter, IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>

public class IndicesService
extends AbstractLifecycleComponent
implements IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>, IndexService.ShardStoreDeleter

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	IndicesService.ShardDeletionCheckResult	result type returned by signaling different reasons why a shard can / cannot be deleted

Nested classes/interfaces inherited from interface org.elasticsearch.indices.cluster.IndicesClusterStateService.AllocatedIndices

IndicesClusterStateService.AllocatedIndices.IndexRemovalReason

Field Summary

Fields

Modifier and Type	Field	Description
static final Setting<TimeValue>	INDICES_CACHE_CLEAN_INTERVAL_SETTING
static final Setting<Boolean>	INDICES_ID_FIELD_DATA_ENABLED_SETTING
static final String	INDICES_SHARDS_CLOSED_TIMEOUT
static final Setting<Boolean>	WRITE_DANGLING_INDICES_INFO_SETTING

Fields inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent

lifecycle

Constructor Summary

Constructors

Constructor

Description

IndicesService(Settings settings, PluginsService pluginsService, NodeEnvironment nodeEnv, NamedXContentRegistry xContentRegistry, AnalysisRegistry analysisRegistry, IndexNameExpressionResolver indexNameExpressionResolver, MapperRegistry mapperRegistry, NamedWriteableRegistry namedWriteableRegistry, ThreadPool threadPool, IndexScopedSettings indexScopedSettings, CircuitBreakerService circuitBreakerService, BigArrays bigArrays, ScriptService scriptService, ClusterService clusterService, Client client, MetaStateService metaStateService, Collection<Function<IndexSettings,Optional<EngineFactory>>> engineFactoryProviders, Map<String,IndexStorePlugin.DirectoryFactory> directoryFactories, ValuesSourceRegistry valuesSourceRegistry, Map<String,IndexStorePlugin.RecoveryStateFactory> recoveryStateFactories, List<IndexStorePlugin.IndexFoldersDeletionListener> indexFoldersDeletionListeners, Map<String,IndexStorePlugin.SnapshotCommitSupplier> snapshotCommitSuppliers, CheckedBiConsumer<ShardSearchRequest,StreamOutput,IOException> requestCacheKeyDifferentiator)

Method Summary

Modifier and Type	Method	Description
void	addPendingDelete(Index index, IndexSettings settings)	Adds a pending delete for the given index.
void	addPendingDelete(ShardId shardId, IndexSettings settings)	Adds a pending delete for the given index shard.
boolean	allPendingDanglingIndicesWritten()
boolean	awaitClose(long timeout, TimeUnit timeUnit)	Wait for this IndicesService to be effectively closed.
AliasFilter	buildAliasFilter(ClusterState state, String index, Set<String> resolvedExpressions)
boolean	canCache(ShardSearchRequest request, SearchContext context)	Can the shard request be cached at all?
boolean	canDeleteIndexContents(Index index)	This method returns true if the current node is allowed to delete the given index.
IndicesService.ShardDeletionCheckResult	canDeleteShardContent(ShardId shardId, IndexSettings indexSettings)	Returns ShardDeletionCheckResult signaling whether the shards content for the given shard can be deleted.
void	clearIndexShardCache(ShardId shardId, boolean queryCache, boolean fieldDataCache, boolean requestCache, String... fields)	Clears the caches for the given shard id if the shard is still allocated on this node
ClusterService	clusterService()
IndexService	createIndex(IndexMetadata indexMetadata, List<IndexEventListener> builtInListeners, boolean writeDanglingIndices)	Creates a new IndexService for the given metadata.
MapperService	createIndexMapperService(IndexMetadata indexMetadata)	creates a new mapper service for the given index, in order to do administrative work like mapping updates.
IndexShard	createShard(ShardRouting shardRouting, PeerRecoveryTargetService recoveryTargetService, PeerRecoveryTargetService.RecoveryListener recoveryListener, RepositoriesService repositoriesService, Consumer<IndexShard.ShardFailure> onShardFailure, Consumer<ShardId> globalCheckpointSyncer, RetentionLeaseSyncer retentionLeaseSyncer, DiscoveryNode targetNode, DiscoveryNode sourceNode)	Creates a shard for the specified shard routing and starts recovery.
void	deleteShardStore(String reason, ShardLock lock, IndexSettings indexSettings)	Deletes the shard with an already acquired shard lock.
void	deleteShardStore(String reason, ShardId shardId, ClusterState clusterState)	This method deletes the shard contents on disk for the given shard ID.
void	deleteUnassignedIndex(String reason, IndexMetadata oldIndexMetadata, ClusterState clusterState)	Deletes an index that is not assigned to this node.
protected void	doClose()
protected void	doStart()
protected void	doStop()
Set<String>	getAllMetadataFields()	Returns the registered metadata field names for all compatible versions.
AnalysisRegistry	getAnalysis()
CircuitBreakerService	getCircuitBreakerService()
CoordinatorRewriteContextProvider	getCoordinatorRewriteContextProvider(LongSupplier nowInMillis)
Function<String,Predicate<String>>	getFieldFilter()	Returns a function which given an index name, returns a predicate which fields must match in order to be returned by get mappings, get index, get field mappings and field capabilities API.
IndicesFieldDataCache	getIndicesFieldDataCache()
IndicesQueryCache	getIndicesQueryCache()
Set<String>	getMetadataFields(Version version)	Returns the registered metadata field names for the provided compatible Version.
QueryRewriteContext	getRewriteContext(LongSupplier nowInMillis)	Returns a new QueryRewriteContext with the given now provider
DateFieldMapper.DateFieldType	getTimestampFieldType(Index index)
ByteSizeValue	getTotalIndexingBufferBytes()
boolean	hasIndex(Index index)
boolean	hasUncompletedPendingDeletes()	Checks if all pending deletes have completed.
IndexService	indexService(Index index)	Returns an IndexService for the specified index if exists otherwise returns null.
IndexService	indexServiceSafe(Index index)	Returns an IndexService for the specified index if exists otherwise a IndexNotFoundException is thrown.
boolean	isIdFieldDataEnabled()	Returns true if fielddata is enabled for the IdFieldMapper field, false otherwise.
Iterator<IndexService>	iterator()
void	loadIntoContext(ShardSearchRequest request, SearchContext context, QueryPhase queryPhase)	Loads the cache result, computing it if needed by executing the query phase and otherwise deserializing the cached value into the context's query result.
void	processPendingDeletes(Index index, IndexSettings indexSettings, TimeValue timeout)	Processes all pending deletes for the given index.
void	removeIndex(Index index, IndicesClusterStateService.AllocatedIndices.IndexRemovalReason reason, String extraInfo)	Removes the given index from this service and releases all associated resources.
NodeIndicesStats	stats(CommonStatsFlags flags)
IndexMetadata	verifyIndexIsDeleted(Index index, ClusterState clusterState)	Verify that the contents on disk for the given index is deleted; if not, delete the contents.
void	verifyIndexMetadata(IndexMetadata metadata, IndexMetadata metadataUpdate)	This method verifies that the given metadata holds sane values to create an IndexService.
<T, E extends Exception> T	withTempIndexService(IndexMetadata indexMetadata, CheckedFunction<IndexService,T,E> indexServiceConsumer)

Methods inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent

addLifecycleListener, close, lifecycleState, removeLifecycleListener, start, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.elasticsearch.indices.cluster.IndicesClusterStateService.AllocatedIndices

getShardOrNull

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Field Details

INDICES_SHARDS_CLOSED_TIMEOUT

public static final String INDICES_SHARDS_CLOSED_TIMEOUT

See Also:

• Constant Field Values

INDICES_CACHE_CLEAN_INTERVAL_SETTING

public static final Setting<TimeValue> INDICES_CACHE_CLEAN_INTERVAL_SETTING

INDICES_ID_FIELD_DATA_ENABLED_SETTING

public static final Setting<Boolean> INDICES_ID_FIELD_DATA_ENABLED_SETTING

WRITE_DANGLING_INDICES_INFO_SETTING

public static final Setting<Boolean> WRITE_DANGLING_INDICES_INFO_SETTING

Constructor Details

IndicesService

public IndicesService(Settings settings,
 PluginsService pluginsService,
 NodeEnvironment nodeEnv,
 NamedXContentRegistry xContentRegistry,
 AnalysisRegistry analysisRegistry,
 IndexNameExpressionResolver indexNameExpressionResolver,
 MapperRegistry mapperRegistry,
 NamedWriteableRegistry namedWriteableRegistry,
 ThreadPool threadPool,
 IndexScopedSettings indexScopedSettings,
 CircuitBreakerService circuitBreakerService,
 BigArrays bigArrays,
 ScriptService scriptService,
 ClusterService clusterService,
 Client client,
 MetaStateService metaStateService,
 Collection<Function<IndexSettings,Optional<EngineFactory>>> engineFactoryProviders,
 Map<String,IndexStorePlugin.DirectoryFactory> directoryFactories,
 ValuesSourceRegistry valuesSourceRegistry,
 Map<String,IndexStorePlugin.RecoveryStateFactory> recoveryStateFactories,
 List<IndexStorePlugin.IndexFoldersDeletionListener> indexFoldersDeletionListeners,
 Map<String,IndexStorePlugin.SnapshotCommitSupplier> snapshotCommitSuppliers,
 CheckedBiConsumer<ShardSearchRequest,StreamOutput,IOException> requestCacheKeyDifferentiator)

Method Details

doStart

protected void doStart()

Specified by:

doStart in class AbstractLifecycleComponent

clusterService

public ClusterService clusterService()

doStop

protected void doStop()

Specified by:

doStop in class AbstractLifecycleComponent

doClose

protected void doClose()
                throws IOException

Specified by:

doClose in class AbstractLifecycleComponent

Throws:

IOException

awaitClose

public boolean awaitClose(long timeout,
 TimeUnit timeUnit)
                   throws InterruptedException

Wait for this IndicesService to be effectively closed. When this returns true, all shards and shard stores are closed and all shard closed listeners have run. However some shard closed listeners might not have run.

Returns:

true if all shards closed within the given timeout, false otherwise

Throws:

InterruptedException - if the current thread got interrupted while waiting for shards to close

stats

public NodeIndicesStats stats(CommonStatsFlags flags)

iterator

public Iterator<IndexService> iterator()

Specified by:

iterator in interface Iterable<IndexService>

hasIndex

public boolean hasIndex(Index index)

indexService

@Nullable
public IndexService indexService(Index index)

Returns an IndexService for the specified index if exists otherwise returns null.

Specified by:

indexService in interface IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>

indexServiceSafe

public IndexService indexServiceSafe(Index index)

Returns an IndexService for the specified index if exists otherwise a IndexNotFoundException is thrown.

createIndex

public IndexService createIndex(IndexMetadata indexMetadata,
 List<IndexEventListener> builtInListeners,
 boolean writeDanglingIndices)
                         throws IOException

Creates a new IndexService for the given metadata.

Specified by:

createIndex in interface IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>

Parameters:

indexMetadata - the index metadata to create the index for

builtInListeners - a list of built-in lifecycle IndexEventListener that should should be used along side with the per-index listeners

writeDanglingIndices - whether dangling indices information should be written

Throws:

ResourceAlreadyExistsException - if the index already exists.

IOException

withTempIndexService

public <T,
E extends Exception> T withTempIndexService(IndexMetadata indexMetadata,
 CheckedFunction<IndexService,T,E> indexServiceConsumer)
                                               throws IOException,
E

Throws:

IOException

E extends Exception

createIndexMapperService

public MapperService createIndexMapperService(IndexMetadata indexMetadata)
                                       throws IOException

creates a new mapper service for the given index, in order to do administrative work like mapping updates. This *should not* be used for document parsing. Doing so will result in an exception. Note: the returned MapperService should be closed when unneeded.

Throws:

IOException

verifyIndexMetadata

public void verifyIndexMetadata(IndexMetadata metadata,
 IndexMetadata metadataUpdate)
                         throws IOException

This method verifies that the given metadata holds sane values to create an IndexService. This method tries to update the meta data of the created IndexService if the given metadataUpdate is different from the given metadata. This method will throw an exception if the creation or the update fails. The created IndexService will not be registered and will be closed immediately.

Throws:

IOException

createShard

public IndexShard createShard(ShardRouting shardRouting,
 PeerRecoveryTargetService recoveryTargetService,
 PeerRecoveryTargetService.RecoveryListener recoveryListener,
 RepositoriesService repositoriesService,
 Consumer<IndexShard.ShardFailure> onShardFailure,
 Consumer<ShardId> globalCheckpointSyncer,
 RetentionLeaseSyncer retentionLeaseSyncer,
 DiscoveryNode targetNode,
 DiscoveryNode sourceNode)
                       throws IOException

Description copied from interface: IndicesClusterStateService.AllocatedIndices

Creates a shard for the specified shard routing and starts recovery.

Specified by:

createShard in interface IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>

Parameters:

shardRouting - the shard routing

recoveryTargetService - recovery service for the target

recoveryListener - a callback when recovery changes state (finishes or fails)

repositoriesService - service responsible for snapshot/restore

onShardFailure - a callback when this shard fails

globalCheckpointSyncer - a callback when this shard syncs the global checkpoint

retentionLeaseSyncer - a callback when this shard syncs retention leases

targetNode - the node where this shard will be recovered

sourceNode - the source node to recover this shard from (it might be null)

Returns:

a new shard

Throws:

IOException - if an I/O exception occurs when creating the shard

removeIndex

public void removeIndex(Index index,
 IndicesClusterStateService.AllocatedIndices.IndexRemovalReason reason,
 String extraInfo)

Description copied from interface: IndicesClusterStateService.AllocatedIndices

Removes the given index from this service and releases all associated resources. Persistent parts of the index like the shards files, state and transaction logs are kept around in the case of a disaster recovery.

Specified by:

removeIndex in interface IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>

Parameters:

index - the index to remove

reason - the reason to remove the index

extraInfo - extra information that will be used for logging and reporting

getIndicesFieldDataCache

public IndicesFieldDataCache getIndicesFieldDataCache()

getCircuitBreakerService

public CircuitBreakerService getCircuitBreakerService()

getIndicesQueryCache

public IndicesQueryCache getIndicesQueryCache()

deleteUnassignedIndex

public void deleteUnassignedIndex(String reason,
 IndexMetadata oldIndexMetadata,
 ClusterState clusterState)

Deletes an index that is not assigned to this node. This method cleans up all disk folders relating to the index but does not deal with in-memory structures. For those call removeIndex(Index, IndexRemovalReason, String)

Specified by:

deleteUnassignedIndex in interface IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>

deleteShardStore

public void deleteShardStore(String reason,
 ShardLock lock,
 IndexSettings indexSettings)
                      throws IOException

Deletes the shard with an already acquired shard lock.

Specified by:

deleteShardStore in interface IndexService.ShardStoreDeleter

Parameters:

reason - the reason for the shard deletion

lock - the lock of the shard to delete

indexSettings - the shards index settings.

Throws:

IOException - if an IOException occurs

deleteShardStore

public void deleteShardStore(String reason,
 ShardId shardId,
 ClusterState clusterState)
                      throws IOException,
ShardLockObtainFailedException

This method deletes the shard contents on disk for the given shard ID. This method will fail if the shard deleting is prevented by canDeleteShardContent(ShardId, IndexSettings) of if the shards lock can not be acquired. On data nodes, if the deleted shard is the last shard folder in its index, the method will attempt to remove the index folder as well.

Parameters:

reason - the reason for the shard deletion

shardId - the shards ID to delete

clusterState - . This is required to access the indexes settings etc.

Throws:

IOException - if an IOException occurs

ShardLockObtainFailedException

canDeleteIndexContents

public boolean canDeleteIndexContents(Index index)

This method returns true if the current node is allowed to delete the given index. This is the case if the index is deleted in the metadata or there is no allocation on the local node and the index isn't on a shared file system.

Parameters:

index - Index to check whether deletion is allowed

Returns:

true if the index can be deleted on this node

verifyIndexIsDeleted

@Nullable
public IndexMetadata verifyIndexIsDeleted(Index index,
 ClusterState clusterState)

Verify that the contents on disk for the given index is deleted; if not, delete the contents. This method assumes that an index is already deleted in the cluster state and/or explicitly through index tombstones.

Specified by:

verifyIndexIsDeleted in interface IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>

Parameters:

index - Index to make sure its deleted from disk

clusterState - ClusterState to ensure the index is not part of it

Returns:

IndexMetadata for the index loaded from disk

canDeleteShardContent

public IndicesService.ShardDeletionCheckResult canDeleteShardContent(ShardId shardId,
 IndexSettings indexSettings)

Returns ShardDeletionCheckResult signaling whether the shards content for the given shard can be deleted.

Parameters:

shardId - the shard to delete.

indexSettings - the shards's relevant IndexSettings. This is required to access the indexes settings etc.

addPendingDelete

public void addPendingDelete(ShardId shardId,
 IndexSettings settings)

Adds a pending delete for the given index shard.

Specified by:

addPendingDelete in interface IndexService.ShardStoreDeleter

addPendingDelete

public void addPendingDelete(Index index,
 IndexSettings settings)

Adds a pending delete for the given index.

processPendingDeletes

public void processPendingDeletes(Index index,
 IndexSettings indexSettings,
 TimeValue timeout)
                           throws IOException,
InterruptedException,
ShardLockObtainFailedException

Processes all pending deletes for the given index. This method will acquire all locks for the given index and will process all pending deletes for this index. Pending deletes might occur if the OS doesn't allow deletion of files because they are used by a different process ie. on Windows where files might still be open by a virus scanner. On a shared filesystem a replica might not have been closed when the primary is deleted causing problems on delete calls so we schedule there deletes later.

Specified by:

processPendingDeletes in interface IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>

Parameters:

index - the index to process the pending deletes for

timeout - the timeout used for processing pending deletes

Throws:

IOException

InterruptedException

ShardLockObtainFailedException

hasUncompletedPendingDeletes

public boolean hasUncompletedPendingDeletes()

Checks if all pending deletes have completed. Used by tests to ensure we don't check directory contents while deletion still ongoing. * The reason is that, on Windows, browsing the directory contents can interfere with the deletion process and delay it unnecessarily.

getAnalysis

public AnalysisRegistry getAnalysis()

canCache

public boolean canCache(ShardSearchRequest request,
 SearchContext context)

Can the shard request be cached at all?

loadIntoContext

public void loadIntoContext(ShardSearchRequest request,
 SearchContext context,
 QueryPhase queryPhase)
                     throws Exception

Loads the cache result, computing it if needed by executing the query phase and otherwise deserializing the cached value into the context's query result. The combination of load + compute allows to have a single load operation that will cause other requests with the same key to wait till its loaded an reuse the same cache.

Throws:

Exception

getTotalIndexingBufferBytes

public ByteSizeValue getTotalIndexingBufferBytes()

buildAliasFilter

public AliasFilter buildAliasFilter(ClusterState state,
 String index,
 Set<String> resolvedExpressions)

getRewriteContext

public QueryRewriteContext getRewriteContext(LongSupplier nowInMillis)

Returns a new QueryRewriteContext with the given now provider

getCoordinatorRewriteContextProvider

public CoordinatorRewriteContextProvider getCoordinatorRewriteContextProvider(LongSupplier nowInMillis)

clearIndexShardCache

public void clearIndexShardCache(ShardId shardId,
 boolean queryCache,
 boolean fieldDataCache,
 boolean requestCache,
 String... fields)

Clears the caches for the given shard id if the shard is still allocated on this node

getFieldFilter

public Function<String,Predicate<String>> getFieldFilter()

Returns a function which given an index name, returns a predicate which fields must match in order to be returned by get mappings, get index, get field mappings and field capabilities API. Useful to filter the fields that such API return. The predicate receives the field name as input argument. In case multiple plugins register a field filter through MapperPlugin.getFieldFilter(), only fields that match all the registered filters will be returned by get mappings, get index, get field mappings and field capabilities API.

getMetadataFields

public Set<String> getMetadataFields(Version version)

Returns the registered metadata field names for the provided compatible Version.

getAllMetadataFields

public Set<String> getAllMetadataFields()

Returns the registered metadata field names for all compatible versions.

isIdFieldDataEnabled

public boolean isIdFieldDataEnabled()

Returns true if fielddata is enabled for the IdFieldMapper field, false otherwise.

allPendingDanglingIndicesWritten

public boolean allPendingDanglingIndicesWritten()

getTimestampFieldType

@Nullable
public DateFieldMapper.DateFieldType getTimestampFieldType(Index index)

Returns:

the field type of the @timestamp field of the given index, or null if: - the index is not found, - the field is not found, or - the field is not a timestamp field.