Class Store.MetadataSnapshot

java.lang.Object
org.elasticsearch.index.store.Store.MetadataSnapshot
All Implemented Interfaces:
java.lang.Iterable<StoreFileMetaData>, Writeable
Enclosing class:
Store

public static final class Store.MetadataSnapshot
extends java.lang.Object
implements java.lang.Iterable<StoreFileMetaData>, Writeable
Represents a snapshot of the current directory build from the latest Lucene commit. Only files that are part of the last commit are considered in this datastructure. For backwards compatibility the snapshot might include legacy checksums that are derived from a dedicated checksum file written by older elasticsearch version pre 1.3

Note: This class will ignore the segments.gen file since it's optional and might change concurrently for safety reasons.

See Also:
StoreFileMetaData
  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.elasticsearch.common.io.stream.Writeable

    Writeable.Reader<V>, Writeable.Writer<V>
  • Field Summary

    Fields 
    Modifier and Type Field Description
    static Store.MetadataSnapshot EMPTY  
  • Constructor Summary

    Constructors 
    Constructor Description
    MetadataSnapshot​(java.util.Map<java.lang.String,​StoreFileMetaData> metadata, java.util.Map<java.lang.String,​java.lang.String> commitUserData, long numDocs)  
    MetadataSnapshot​(StreamInput in)
    Read from a stream.
  • Method Summary

    Modifier and Type Method Description
    java.util.Map<java.lang.String,​StoreFileMetaData> asMap()  
    boolean contains​(java.lang.String existingFile)
    Returns true iff this metadata contains the given file.
    StoreFileMetaData get​(java.lang.String name)  
    java.util.Map<java.lang.String,​java.lang.String> getCommitUserData()  
    java.lang.String getHistoryUUID()
    returns the history uuid the store points at, or null if nonexistent.
    long getNumDocs()
    Returns the number of documents in this store snapshot
    StoreFileMetaData getSegmentsFile()
    Returns the segments file that this metadata snapshot represents or null if the snapshot is empty.
    java.lang.String getSyncId()
    Returns the sync id of the commit point that this MetadataSnapshot represents.
    static void hashFile​(org.apache.lucene.util.BytesRefBuilder fileHash, java.io.InputStream in, long size)
    Computes a strong hash value for small files.
    java.util.Iterator<StoreFileMetaData> iterator()  
    Store.RecoveryDiff recoveryDiff​(Store.MetadataSnapshot recoveryTargetSnapshot)
    Returns a diff between the two snapshots that can be used for recovery.
    int size()
    Returns the number of files in this snapshot
    void writeTo​(StreamOutput out)
    Write this into the StreamOutput.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface java.lang.Iterable

    forEach, spliterator
  • Field Details

  • Constructor Details

    • MetadataSnapshot

      public MetadataSnapshot​(java.util.Map<java.lang.String,​StoreFileMetaData> metadata, java.util.Map<java.lang.String,​java.lang.String> commitUserData, long numDocs)
    • MetadataSnapshot

      public MetadataSnapshot​(StreamInput in) throws java.io.IOException
      Read from a stream.
      Throws:
      java.io.IOException
  • Method Details

    • writeTo

      public void writeTo​(StreamOutput out) throws java.io.IOException
      Description copied from interface: Writeable
      Write this into the StreamOutput.
      Specified by:
      writeTo in interface Writeable
      Throws:
      java.io.IOException
    • getNumDocs

      public long getNumDocs()
      Returns the number of documents in this store snapshot
    • hashFile

      public static void hashFile​(org.apache.lucene.util.BytesRefBuilder fileHash, java.io.InputStream in, long size) throws java.io.IOException
      Computes a strong hash value for small files. Note that this method should only be used for files < 1MB
      Throws:
      java.io.IOException
    • iterator

      public java.util.Iterator<StoreFileMetaData> iterator()
      Specified by:
      iterator in interface java.lang.Iterable<StoreFileMetaData>
    • get

      public StoreFileMetaData get​(java.lang.String name)
    • asMap

      public java.util.Map<java.lang.String,​StoreFileMetaData> asMap()
    • recoveryDiff

      public Store.RecoveryDiff recoveryDiff​(Store.MetadataSnapshot recoveryTargetSnapshot)
      Returns a diff between the two snapshots that can be used for recovery. The given snapshot is treated as the recovery target and this snapshot as the source. The returned diff will hold a list of files that are:
      • identical: they exist in both snapshots and they can be considered the same ie. they don't need to be recovered
      • different: they exist in both snapshots but their they are not identical
      • missing: files that exist in the source but not in the target
      This method groups file into per-segment files and per-commit files. A file is treated as identical if and on if all files in it's group are identical. On a per-segment level files for a segment are treated as identical iff:
      • all files in this segment have the same checksum
      • all files in this segment have the same length
      • the segments .si files hashes are byte-identical Note: This is a using a perfect hash function, The metadata transfers the .si file content as it's hash

      The .si file contains a lot of diagnostics including a timestamp etc. in the future there might be unique segment identifiers in there hardening this method further.

      The per-commit files handles very similar. A commit is composed of the segments_N files as well as generational files like deletes (_x_y.del) or field-info (_x_y.fnm) files. On a per-commit level files for a commit are treated as identical iff:

      • all files belonging to this commit have the same checksum
      • all files belonging to this commit have the same length
      • the segments file segments_N files hashes are byte-identical Note: This is a using a perfect hash function, The metadata transfers the segments_N file content as it's hash

      NOTE: this diff will not contain the segments.gen file. This file is omitted on recovery.

    • size

      public int size()
      Returns the number of files in this snapshot
    • getCommitUserData

      public java.util.Map<java.lang.String,​java.lang.String> getCommitUserData()
    • getHistoryUUID

      public java.lang.String getHistoryUUID()
      returns the history uuid the store points at, or null if nonexistent.
    • contains

      public boolean contains​(java.lang.String existingFile)
      Returns true iff this metadata contains the given file.
    • getSegmentsFile

      public StoreFileMetaData getSegmentsFile()
      Returns the segments file that this metadata snapshot represents or null if the snapshot is empty.
    • getSyncId

      public java.lang.String getSyncId()
      Returns the sync id of the commit point that this MetadataSnapshot represents.
      Returns:
      sync id if exists, else null