Package org.elasticsearch.indices
Class IndicesService
- java.lang.Object
-
- org.elasticsearch.common.component.AbstractLifecycleComponent
-
- org.elasticsearch.indices.IndicesService
-
- All Implemented Interfaces:
java.io.Closeable,java.lang.AutoCloseable,java.lang.Iterable<IndexService>,LifecycleComponent,Releasable,IndexService.ShardStoreDeleter,IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>
public class IndicesService extends AbstractLifecycleComponent implements IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>, IndexService.ShardStoreDeleter
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classIndicesService.ShardDeletionCheckResultresult type returned bysignaling different reasons why a shard can / cannot be deleted-
Nested classes/interfaces inherited from interface org.elasticsearch.indices.cluster.IndicesClusterStateService.AllocatedIndices
IndicesClusterStateService.AllocatedIndices.IndexRemovalReason
-
-
Field Summary
Fields Modifier and Type Field Description static Setting<TimeValue>INDICES_CACHE_CLEAN_INTERVAL_SETTINGstatic java.lang.StringINDICES_SHARDS_CLOSED_TIMEOUT-
Fields inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent
lifecycle
-
-
Constructor Summary
Constructors Constructor Description IndicesService(Settings settings, PluginsService pluginsService, NodeEnvironment nodeEnv, NamedXContentRegistry xContentRegistry, AnalysisRegistry analysisRegistry, IndexNameExpressionResolver indexNameExpressionResolver, MapperRegistry mapperRegistry, NamedWriteableRegistry namedWriteableRegistry, ThreadPool threadPool, IndexScopedSettings indexScopedSettings, CircuitBreakerService circuitBreakerService, BigArrays bigArrays, ScriptService scriptService, Client client, MetaStateService metaStateService, java.util.Collection<java.util.function.Function<IndexSettings,java.util.Optional<EngineFactory>>> engineFactoryProviders, java.util.Map<java.lang.String,java.util.function.Function<IndexSettings,IndexStore>> indexStoreFactories)
-
Method Summary
Modifier and Type Method Description voidaddPendingDelete(Index index, IndexSettings settings)Adds a pending delete for the given index.voidaddPendingDelete(ShardId shardId, IndexSettings settings)Adds a pending delete for the given index shard.AliasFilterbuildAliasFilter(ClusterState state, java.lang.String index, java.util.Set<java.lang.String> resolvedExpressions)booleancanCache(ShardSearchRequest request, SearchContext context)Can the shard request be cached at all?booleancanDeleteIndexContents(Index index, IndexSettings indexSettings)This method returns true if the current node is allowed to delete the given index.IndicesService.ShardDeletionCheckResultcanDeleteShardContent(ShardId shardId, IndexSettings indexSettings)ReturnsShardDeletionCheckResultsignaling whether the shards content for the given shard can be deleted.static java.util.Optional<java.lang.String>checkShardLimit(int newShards, ClusterState state, DeprecationLogger deprecationLogger)Checks to see if an operation can be performed without taking the cluster over the cluster-wide shard limit.voidclearIndexShardCache(ShardId shardId, boolean queryCache, boolean fieldDataCache, boolean requestCache, java.lang.String... fields)Clears the caches for the given shard id if the shard is still allocated on this nodeIndexServicecreateIndex(IndexMetaData indexMetaData, java.util.List<IndexEventListener> builtInListeners)Creates a newIndexServicefor the given metadata.MapperServicecreateIndexMapperService(IndexMetaData indexMetaData)creates a new mapper service for the given index, in order to do administrative work like mapping updates.IndexShardcreateShard(ShardRouting shardRouting, RecoveryState recoveryState, PeerRecoveryTargetService recoveryTargetService, PeerRecoveryTargetService.RecoveryListener recoveryListener, RepositoriesService repositoriesService, java.util.function.Consumer<IndexShard.ShardFailure> onShardFailure, java.util.function.Consumer<ShardId> globalCheckpointSyncer, RetentionLeaseSyncer retentionLeaseSyncer)Creates a shard for the specified shard routing and starts recovery.voiddeleteShardStore(java.lang.String reason, ShardLock lock, IndexSettings indexSettings)Deletes the shard with an already acquired shard lock.voiddeleteShardStore(java.lang.String reason, ShardId shardId, ClusterState clusterState)This method deletes the shard contents on disk for the given shard ID.voiddeleteUnassignedIndex(java.lang.String reason, IndexMetaData metaData, ClusterState clusterState)Deletes an index that is not assigned to this node.protected voiddoClose()protected voiddoStart()protected voiddoStop()AnalysisRegistrygetAnalysis()CircuitBreakerServicegetCircuitBreakerService()java.util.function.Function<java.lang.String,java.util.function.Predicate<java.lang.String>>getFieldFilter()Returns a function which given an index name, returns a predicate which fields must match in order to be returned by get mappings, get index, get field mappings and field capabilities API.IndicesFieldDataCachegetIndicesFieldDataCache()IndicesQueryCachegetIndicesQueryCache()QueryRewriteContextgetRewriteContext(java.util.function.LongSupplier nowInMillis)Returns a newQueryRewriteContextwith the givennowproviderByteSizeValuegetTotalIndexingBufferBytes()booleanhasIndex(Index index)booleanhasUncompletedPendingDeletes()Checks if all pending deletes have completed.IndexServiceindexService(Index index)Returns an IndexService for the specified index if exists otherwise returnsnull.IndexServiceindexServiceSafe(Index index)Returns an IndexService for the specified index if exists otherwise aIndexNotFoundExceptionis thrown.booleanisMetaDataField(java.lang.String field)Returns true if the provided field is a registered metadata field (including ones registered via plugins), false otherwise.java.util.Iterator<IndexService>iterator()voidloadIntoContext(ShardSearchRequest request, SearchContext context, QueryPhase queryPhase)Loads the cache result, computing it if needed by executing the query phase and otherwise deserializing the cached value into thecontext's query result.voidprocessPendingDeletes(Index index, IndexSettings indexSettings, TimeValue timeout)Processes all pending deletes for the given index.voidremoveIndex(Index index, IndicesClusterStateService.AllocatedIndices.IndexRemovalReason reason, java.lang.String extraInfo)Removes the given index from this service and releases all associated resources.NodeIndicesStatsstats(boolean includePrevious)Returns the node stats indices stats.NodeIndicesStatsstats(boolean includePrevious, CommonStatsFlags flags)IndexMetaDataverifyIndexIsDeleted(Index index, ClusterState clusterState)Verify that the contents on disk for the given index is deleted; if not, delete the contents.voidverifyIndexMetadata(IndexMetaData metaData, IndexMetaData metaDataUpdate)This method verifies that the givenmetaDataholds sane values to create anIndexService.-
Methods inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent
addLifecycleListener, close, lifecycleState, removeLifecycleListener, start, stop
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.elasticsearch.indices.cluster.IndicesClusterStateService.AllocatedIndices
getShardOrNull
-
-
-
-
Field Detail
-
INDICES_SHARDS_CLOSED_TIMEOUT
public static final java.lang.String INDICES_SHARDS_CLOSED_TIMEOUT
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
IndicesService
public IndicesService(Settings settings, PluginsService pluginsService, NodeEnvironment nodeEnv, NamedXContentRegistry xContentRegistry, AnalysisRegistry analysisRegistry, IndexNameExpressionResolver indexNameExpressionResolver, MapperRegistry mapperRegistry, NamedWriteableRegistry namedWriteableRegistry, ThreadPool threadPool, IndexScopedSettings indexScopedSettings, CircuitBreakerService circuitBreakerService, BigArrays bigArrays, ScriptService scriptService, Client client, MetaStateService metaStateService, java.util.Collection<java.util.function.Function<IndexSettings,java.util.Optional<EngineFactory>>> engineFactoryProviders, java.util.Map<java.lang.String,java.util.function.Function<IndexSettings,IndexStore>> indexStoreFactories)
-
-
Method Detail
-
doStart
protected void doStart()
- Specified by:
doStartin classAbstractLifecycleComponent
-
doStop
protected void doStop()
- Specified by:
doStopin classAbstractLifecycleComponent
-
doClose
protected void doClose()
- Specified by:
doClosein classAbstractLifecycleComponent
-
stats
public NodeIndicesStats stats(boolean includePrevious)
Returns the node stats indices stats. TheincludePreviousflag controls if old shards stats will be aggregated as well (only for relevant stats, such as refresh and indexing, not for docs/store).
-
stats
public NodeIndicesStats stats(boolean includePrevious, CommonStatsFlags flags)
-
iterator
public java.util.Iterator<IndexService> iterator()
- Specified by:
iteratorin interfacejava.lang.Iterable<IndexService>
-
hasIndex
public boolean hasIndex(Index index)
-
indexService
@Nullable public IndexService indexService(Index index)
Returns an IndexService for the specified index if exists otherwise returnsnull.- Specified by:
indexServicein interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>
-
indexServiceSafe
public IndexService indexServiceSafe(Index index)
Returns an IndexService for the specified index if exists otherwise aIndexNotFoundExceptionis thrown.
-
createIndex
public IndexService createIndex(IndexMetaData indexMetaData, java.util.List<IndexEventListener> builtInListeners) throws java.io.IOException
Creates a newIndexServicefor the given metadata.- Specified by:
createIndexin interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>- Parameters:
indexMetaData- the index metadata to create the index forbuiltInListeners- a list of built-in lifecycleIndexEventListenerthat should should be used along side with the per-index listeners- Throws:
ResourceAlreadyExistsException- if the index already exists.java.io.IOException
-
createIndexMapperService
public MapperService createIndexMapperService(IndexMetaData indexMetaData) throws java.io.IOException
creates a new mapper service for the given index, in order to do administrative work like mapping updates. This *should not* be used for document parsing. Doing so will result in an exception. Note: the returnedMapperServiceshould be closed when unneeded.- Throws:
java.io.IOException
-
verifyIndexMetadata
public void verifyIndexMetadata(IndexMetaData metaData, IndexMetaData metaDataUpdate) throws java.io.IOException
This method verifies that the givenmetaDataholds sane values to create anIndexService. This method tries to update the meta data of the createdIndexServiceif the givenmetaDataUpdateis different from the givenmetaData. This method will throw an exception if the creation or the update fails. The createdIndexServicewill not be registered and will be closed immediately.- Throws:
java.io.IOException
-
createShard
public IndexShard createShard(ShardRouting shardRouting, RecoveryState recoveryState, PeerRecoveryTargetService recoveryTargetService, PeerRecoveryTargetService.RecoveryListener recoveryListener, RepositoriesService repositoriesService, java.util.function.Consumer<IndexShard.ShardFailure> onShardFailure, java.util.function.Consumer<ShardId> globalCheckpointSyncer, RetentionLeaseSyncer retentionLeaseSyncer) throws java.io.IOException
Description copied from interface:IndicesClusterStateService.AllocatedIndicesCreates a shard for the specified shard routing and starts recovery.- Specified by:
createShardin interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>- Parameters:
shardRouting- the shard routingrecoveryState- the recovery staterecoveryTargetService- recovery service for the targetrecoveryListener- a callback when recovery changes state (finishes or fails)repositoriesService- service responsible for snapshot/restoreonShardFailure- a callback when this shard failsglobalCheckpointSyncer- a callback when this shard syncs the global checkpointretentionLeaseSyncer- a callback when this shard syncs retention leases- Returns:
- a new shard
- Throws:
java.io.IOException- if an I/O exception occurs when creating the shard
-
removeIndex
public void removeIndex(Index index, IndicesClusterStateService.AllocatedIndices.IndexRemovalReason reason, java.lang.String extraInfo)
Description copied from interface:IndicesClusterStateService.AllocatedIndicesRemoves the given index from this service and releases all associated resources. Persistent parts of the index like the shards files, state and transaction logs are kept around in the case of a disaster recovery.- Specified by:
removeIndexin interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>- Parameters:
index- the index to removereason- the reason to remove the indexextraInfo- extra information that will be used for logging and reporting
-
getIndicesFieldDataCache
public IndicesFieldDataCache getIndicesFieldDataCache()
-
getCircuitBreakerService
public CircuitBreakerService getCircuitBreakerService()
-
getIndicesQueryCache
public IndicesQueryCache getIndicesQueryCache()
-
deleteUnassignedIndex
public void deleteUnassignedIndex(java.lang.String reason, IndexMetaData metaData, ClusterState clusterState)Deletes an index that is not assigned to this node. This method cleans up all disk folders relating to the index but does not deal with in-memory structures. For those callremoveIndex(Index, IndexRemovalReason, String)- Specified by:
deleteUnassignedIndexin interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>
-
deleteShardStore
public void deleteShardStore(java.lang.String reason, ShardLock lock, IndexSettings indexSettings) throws java.io.IOExceptionDeletes the shard with an already acquired shard lock.- Specified by:
deleteShardStorein interfaceIndexService.ShardStoreDeleter- Parameters:
reason- the reason for the shard deletionlock- the lock of the shard to deleteindexSettings- the shards index settings.- Throws:
java.io.IOException- if an IOException occurs
-
deleteShardStore
public void deleteShardStore(java.lang.String reason, ShardId shardId, ClusterState clusterState) throws java.io.IOException, ShardLockObtainFailedExceptionThis method deletes the shard contents on disk for the given shard ID. This method will fail if the shard deleting is prevented bycanDeleteShardContent(ShardId, IndexSettings)of if the shards lock can not be acquired. On data nodes, if the deleted shard is the last shard folder in its index, the method will attempt to remove the index folder as well.- Parameters:
reason- the reason for the shard deletionshardId- the shards ID to deleteclusterState- . This is required to access the indexes settings etc.- Throws:
java.io.IOException- if an IOException occursShardLockObtainFailedException
-
canDeleteIndexContents
public boolean canDeleteIndexContents(Index index, IndexSettings indexSettings)
This method returns true if the current node is allowed to delete the given index. This is the case if the index is deleted in the metadata or there is no allocation on the local node and the index isn't on a shared file system.- Parameters:
index-Indexto check whether deletion is allowedindexSettings-IndexSettingsfor the given index- Returns:
- true if the index can be deleted on this node
-
verifyIndexIsDeleted
@Nullable public IndexMetaData verifyIndexIsDeleted(Index index, ClusterState clusterState)
Verify that the contents on disk for the given index is deleted; if not, delete the contents. This method assumes that an index is already deleted in the cluster state and/or explicitly through index tombstones.- Specified by:
verifyIndexIsDeletedin interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>- Parameters:
index-Indexto make sure its deleted from diskclusterState-ClusterStateto ensure the index is not part of it- Returns:
- IndexMetaData for the index loaded from disk
-
canDeleteShardContent
public IndicesService.ShardDeletionCheckResult canDeleteShardContent(ShardId shardId, IndexSettings indexSettings)
ReturnsShardDeletionCheckResultsignaling whether the shards content for the given shard can be deleted.- Parameters:
shardId- the shard to delete.indexSettings- the shards's relevantIndexSettings. This is required to access the indexes settings etc.
-
addPendingDelete
public void addPendingDelete(ShardId shardId, IndexSettings settings)
Adds a pending delete for the given index shard.- Specified by:
addPendingDeletein interfaceIndexService.ShardStoreDeleter
-
addPendingDelete
public void addPendingDelete(Index index, IndexSettings settings)
Adds a pending delete for the given index.
-
processPendingDeletes
public void processPendingDeletes(Index index, IndexSettings indexSettings, TimeValue timeout) throws java.io.IOException, java.lang.InterruptedException, ShardLockObtainFailedException
Processes all pending deletes for the given index. This method will acquire all locks for the given index and will process all pending deletes for this index. Pending deletes might occur if the OS doesn't allow deletion of files because they are used by a different process ie. on Windows where files might still be open by a virus scanner. On a shared filesystem a replica might not have been closed when the primary is deleted causing problems on delete calls so we schedule there deletes later.- Specified by:
processPendingDeletesin interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>- Parameters:
index- the index to process the pending deletes fortimeout- the timeout used for processing pending deletes- Throws:
java.io.IOExceptionjava.lang.InterruptedExceptionShardLockObtainFailedException
-
hasUncompletedPendingDeletes
public boolean hasUncompletedPendingDeletes()
Checks if all pending deletes have completed. Used by tests to ensure we don't check directory contents while deletion still ongoing. * The reason is that, on Windows, browsing the directory contents can interfere with the deletion process and delay it unnecessarily.
-
getAnalysis
public AnalysisRegistry getAnalysis()
-
canCache
public boolean canCache(ShardSearchRequest request, SearchContext context)
Can the shard request be cached at all?
-
loadIntoContext
public void loadIntoContext(ShardSearchRequest request, SearchContext context, QueryPhase queryPhase) throws java.lang.Exception
Loads the cache result, computing it if needed by executing the query phase and otherwise deserializing the cached value into thecontext's query result. The combination of load + compute allows to have a single load operation that will cause other requests with the same key to wait till its loaded an reuse the same cache.- Throws:
java.lang.Exception
-
getTotalIndexingBufferBytes
public ByteSizeValue getTotalIndexingBufferBytes()
-
buildAliasFilter
public AliasFilter buildAliasFilter(ClusterState state, java.lang.String index, java.util.Set<java.lang.String> resolvedExpressions)
-
getRewriteContext
public QueryRewriteContext getRewriteContext(java.util.function.LongSupplier nowInMillis)
Returns a newQueryRewriteContextwith the givennowprovider
-
clearIndexShardCache
public void clearIndexShardCache(ShardId shardId, boolean queryCache, boolean fieldDataCache, boolean requestCache, java.lang.String... fields)
Clears the caches for the given shard id if the shard is still allocated on this node
-
getFieldFilter
public java.util.function.Function<java.lang.String,java.util.function.Predicate<java.lang.String>> getFieldFilter()
Returns a function which given an index name, returns a predicate which fields must match in order to be returned by get mappings, get index, get field mappings and field capabilities API. Useful to filter the fields that such API return. The predicate receives the the field name as input argument. In case multiple plugins register a field filter throughMapperPlugin.getFieldFilter(), only fields that match all the registered filters will be returned by get mappings, get index, get field mappings and field capabilities API.
-
isMetaDataField
public boolean isMetaDataField(java.lang.String field)
Returns true if the provided field is a registered metadata field (including ones registered via plugins), false otherwise.
-
checkShardLimit
public static java.util.Optional<java.lang.String> checkShardLimit(int newShards, ClusterState state, DeprecationLogger deprecationLogger)Checks to see if an operation can be performed without taking the cluster over the cluster-wide shard limit. Adds a deprecation warning or returns an error message as appropriate- Parameters:
newShards- The number of shards to be added by this operationstate- The current cluster statedeprecationLogger- The logger to use for deprecation warnings- Returns:
- If present, an error message to be given as the reason for failing an operation. If empty, a sign that the operation is valid.
-
-