public class Translog extends AbstractIndexShardComponent implements IndexShardComponent, java.io.Closeable, org.apache.lucene.index.TwoPhaseCommit
InternalEngine
. The engine
records the current translog generation getGeneration()
in it's commit metadata using TRANSLOG_GENERATION_KEY
to reference the generation that contains all operations that have not yet successfully been committed to the engines lucene index.
Additionally, since Elasticsearch 2.0 the engine also records a TRANSLOG_UUID_KEY
with each commit to ensure a strong association
between the lucene index an the transaction log file. This UUID is used to prevent accidential recovery from a transaction log that belongs to a
different engine.
Each Translog has only one translog file open at any time referenced by a translog generation ID. This ID is written to a translog.ckp file that is designed to fit in a single disk block such that a write of the file is atomic. The checkpoint file is written on each fsync operation of the translog and records the number of operations written, the current tranlogs file generation and it's fsynced offset in bytes.
When a translog is opened the checkpoint is use to retrieve the latest translog file generation and subsequently to open the last written file to recovery operations.
The Translog.TranslogGeneration
, given when the translog is opened / constructed is compared against
the latest generation and all consecutive translog files singe the given generation and the last generation in the checkpoint will be recovered and preserved until the next
generation is committed using commit()
. In the common case the translog file generation in the checkpoint and the generation passed to the translog on creation are
the same. The only situation when they can be different is when an actual translog commit fails in between prepareCommit()
and commit()
. In such a case
the currently being committed translog file will not be deleted since it's commit was not successful. Yet, a new/current translog file is already opened at that point such that there is more than
one translog file present. Such an uncommitted translog file always has a translog-${gen}.ckp associated with it which is an fsynced copy of the it's last translog.ckp such that in
disaster recovery last fsynced offsets, number of operation etc. are still preserved.
Modifier and Type | Class and Description |
---|---|
static class |
Translog.Delete |
static class |
Translog.Durability |
static class |
Translog.Index |
static class |
Translog.Location |
static interface |
Translog.Operation
A generic interface representing an operation performed on the transaction log.
|
static interface |
Translog.Snapshot
A snapshot of the transaction log, allows to iterate over all the transaction log operations.
|
static class |
Translog.Source |
static class |
Translog.TranslogGeneration
References a transaction log generation
|
class |
Translog.View
a view into the translog, capturing all translog file at the moment of creation
and updated with any future translog.
|
Modifier and Type | Field and Description |
---|---|
static java.lang.String |
CHECKPOINT_FILE_NAME |
static java.lang.String |
CHECKPOINT_SUFFIX |
protected ReleasableLock |
readLock |
static java.lang.String |
TRANSLOG_FILE_PREFIX |
static java.lang.String |
TRANSLOG_FILE_SUFFIX |
static java.lang.String |
TRANSLOG_GENERATION_KEY |
static java.lang.String |
TRANSLOG_UUID_KEY |
protected ReleasableLock |
writeLock |
deprecationLogger, indexSettings, logger, shardId
Constructor and Description |
---|
Translog(TranslogConfig config,
Translog.TranslogGeneration translogGeneration)
Creates a new Translog instance.
|
Modifier and Type | Method and Description |
---|---|
Translog.Location |
add(Translog.Operation operation)
Adds a delete / index operations to the transaction log.
|
void |
close() |
long |
commit() |
long |
currentFileGeneration()
Returns the generation of the current transaction log.
|
boolean |
ensureSynced(java.util.stream.Stream<Translog.Location> locations)
Ensures that all locations in the given stream have been synced / written to the underlying storage.
|
boolean |
ensureSynced(Translog.Location location)
Ensures that the given location has be synced / written to the underlying storage.
|
TranslogConfig |
getConfig() |
static java.lang.String |
getFilename(long generation)
package private for testing
|
Translog.TranslogGeneration |
getGeneration()
Returns the current generation of this translog.
|
Translog.Location |
getLastWriteLocation()
The a Translog.Location that will sort after the Translog.Location returned by the last write but before any locations which
can be returned by the next write.
|
java.lang.Exception |
getTragicException()
If this
Translog was closed as a side-effect of a tragic exception,
e.g. |
java.lang.String |
getTranslogUUID()
Returns the translog uuid used to associate a lucene index with a translog.
|
boolean |
isCurrent(Translog.TranslogGeneration generation)
Returns
true iff the given generation is the current gbeneration of this translog |
boolean |
isOpen()
Returns
true if this Translog is still open. |
java.nio.file.Path |
location()
Returns all translog locations as absolute paths.
|
Translog.Snapshot |
newSnapshot()
Snapshots the current transaction log allowing to safely iterate over the snapshot.
|
Translog.View |
newView()
Returns a view into the current translog that is guaranteed to retain all current operations
while receiving future ones as well
|
static long |
parseIdFromFileName(java.nio.file.Path translogFile)
Extracts the translog generation from a file name.
|
long |
prepareCommit() |
static java.util.List<Translog.Operation> |
readOperations(StreamInput input)
Reads a list of operations written with
writeOperations(StreamOutput, List) |
void |
rollback() |
long |
sizeInBytes()
Returns the size in bytes of the translog files that aren't committed to lucene.
|
TranslogStats |
stats()
return stats
|
void |
sync()
Sync's the translog.
|
boolean |
syncNeeded() |
int |
totalOperations()
Returns the number of operations in the transaction files that aren't committed to lucene..
|
static void |
writeOperationNoSize(BufferedChecksumStreamOutput out,
Translog.Operation op) |
static void |
writeOperations(StreamOutput outStream,
java.util.List<Translog.Operation> toWrite)
Writes all operations in the given iterable to the given output stream including the size of the array
use
readOperations(StreamInput) to read it back. |
indexSettings, nodeName, shardId
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
indexSettings, shardId
public static final java.lang.String TRANSLOG_GENERATION_KEY
public static final java.lang.String TRANSLOG_UUID_KEY
public static final java.lang.String TRANSLOG_FILE_PREFIX
public static final java.lang.String TRANSLOG_FILE_SUFFIX
public static final java.lang.String CHECKPOINT_SUFFIX
public static final java.lang.String CHECKPOINT_FILE_NAME
protected final ReleasableLock readLock
protected final ReleasableLock writeLock
public Translog(TranslogConfig config, Translog.TranslogGeneration translogGeneration) throws java.io.IOException
TranslogConfig
has
a non-null Translog.TranslogGeneration
. If the generation is null this method
us destructive and will delete all files in the translog path given.config
- the configuration of this translogtranslogGeneration
- the translog generation to open. If this is null
a new translog is created. If non-null
the translog tries to open the given translog generation. The generation is treated as the last generation referenced
form already committed data. This means all operations that have not yet been committed should be in the translog
file referenced by this generation. The translog creation will fail if this generation can't be opened.java.io.IOException
TranslogConfig.getTranslogPath()
public static long parseIdFromFileName(java.nio.file.Path translogFile)
java.lang.IllegalArgumentException
- if the path doesn't match the expected pattern.public boolean isOpen()
true
if this Translog
is still open.public void close() throws java.io.IOException
close
in interface java.io.Closeable
close
in interface java.lang.AutoCloseable
java.io.IOException
public java.nio.file.Path location()
public long currentFileGeneration()
public int totalOperations()
public long sizeInBytes()
public Translog.Location add(Translog.Operation operation) throws java.io.IOException
java.io.IOException
Translog.Operation
,
Translog.Index
,
Translog.Delete
public Translog.Location getLastWriteLocation()
public Translog.Snapshot newSnapshot()
public Translog.View newView()
public void sync() throws java.io.IOException
java.io.IOException
public boolean syncNeeded()
public static java.lang.String getFilename(long generation)
public boolean ensureSynced(Translog.Location location) throws java.io.IOException
true
iff this call caused an actual sync operation otherwise false
java.io.IOException
public boolean ensureSynced(java.util.stream.Stream<Translog.Location> locations) throws java.io.IOException
true
iff this call caused an actual sync operation otherwise false
java.io.IOException
public TranslogStats stats()
public TranslogConfig getConfig()
public static java.util.List<Translog.Operation> readOperations(StreamInput input) throws java.io.IOException
writeOperations(StreamOutput, List)
java.io.IOException
public static void writeOperations(StreamOutput outStream, java.util.List<Translog.Operation> toWrite) throws java.io.IOException
readOperations(StreamInput)
to read it back.java.io.IOException
public static void writeOperationNoSize(BufferedChecksumStreamOutput out, Translog.Operation op) throws java.io.IOException
java.io.IOException
public long prepareCommit() throws java.io.IOException
prepareCommit
in interface org.apache.lucene.index.TwoPhaseCommit
java.io.IOException
public long commit() throws java.io.IOException
commit
in interface org.apache.lucene.index.TwoPhaseCommit
java.io.IOException
public void rollback() throws java.io.IOException
rollback
in interface org.apache.lucene.index.TwoPhaseCommit
java.io.IOException
public Translog.TranslogGeneration getGeneration()
public boolean isCurrent(Translog.TranslogGeneration generation)
true
iff the given generation is the current gbeneration of this translogpublic java.lang.Exception getTragicException()
Translog
was closed as a side-effect of a tragic exception,
e.g. disk full while flushing a new segment, this returns the root cause exception.
Otherwise (no tragic exception has occurred) it returns null.public java.lang.String getTranslogUUID()