Module org.elasticsearch.server
Package org.elasticsearch.ingest

Class IngestDocument

java.lang.Object
org.elasticsearch.ingest.IngestDocument
public final class IngestDocument extends Object
Represents a single document being captured before indexing and holds the source and metadata (like id, type and index).

  • Field Details

  • Constructor Details

    • IngestDocument

      public IngestDocument(String index, String id, long version, String routing, VersionType versionType, Map<String,Object> source)

    • IngestDocument

      public IngestDocument(IngestDocument other)
      Copy constructor that creates a new IngestDocument which has exactly the same properties as the one provided.

    • IngestDocument

      public IngestDocument(Map<String,Object> sourceAndMetadata, Map<String,Object> ingestMetadata)
      Constructor to create an IngestDocument from its constituent maps. The maps are shallow copied.

  • Method Details

    • getFieldValue

      public <T> T getFieldValue(String path, Class<T> clazz)
      Returns the value contained in the document for the provided path
      Parameters:
      path - The path within the document in dot-notation
      clazz - The expected class of the field value
      Returns:
      the value for the provided path if existing
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid, if the field doesn't exist or if the field that is found at the provided path is not of the expected type.

    • getFieldValue

      public <T> T getFieldValue(String path, Class<T> clazz, boolean ignoreMissing)
      Returns the value contained in the document for the provided path
      Parameters:
      path - The path within the document in dot-notation
      clazz - The expected class of the field value
      ignoreMissing - The flag to determine whether to throw an exception when `path` is not found in the document.
      Returns:
      the value for the provided path if existing, null otherwise.
      Throws:
      IllegalArgumentException - only if ignoreMissing is false and the path is null, empty, invalid, if the field doesn't exist or if the field that is found at the provided path is not of the expected type.

    • getFieldValue

      public <T> T getFieldValue(TemplateScript.Factory pathTemplate, Class<T> clazz)
      Returns the value contained in the document with the provided templated path
      Parameters:
      pathTemplate - The path within the document in dot-notation
      clazz - The expected class fo the field value
      Returns:
      the value for the provided path if existing, null otherwise
      Throws:
      IllegalArgumentException - if the pathTemplate is null, empty, invalid, if the field doesn't exist, or if the field that is found at the provided path is not of the expected type.

    • getFieldValueAsBytes

      public byte[] getFieldValueAsBytes(String path)
      Returns the value contained in the document for the provided path as a byte array. If the path value is a string, a base64 decode operation will happen. If the path value is a byte array, it is just returned
      Parameters:
      path - The path within the document in dot-notation
      Returns:
      the byte array for the provided path if existing
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid, if the field doesn't exist or if the field that is found at the provided path is not of the expected type.

    • getFieldValueAsBytes

      public byte[] getFieldValueAsBytes(String path, boolean ignoreMissing)
      Returns the value contained in the document for the provided path as a byte array. If the path value is a string, a base64 decode operation will happen. If the path value is a byte array, it is just returned
      Parameters:
      path - The path within the document in dot-notation
      ignoreMissing - The flag to determine whether to throw an exception when `path` is not found in the document.
      Returns:
      the byte array for the provided path if existing
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid, if the field doesn't exist or if the field that is found at the provided path is not of the expected type.

    • hasField

      public boolean hasField(TemplateScript.Factory fieldPathTemplate)
      Checks whether the document contains a value for the provided templated path
      Parameters:
      fieldPathTemplate - the template for the path within the document in dot-notation
      Returns:
      true if the document contains a value for the field, false otherwise
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid

    • hasField

      public boolean hasField(String path)
      Checks whether the document contains a value for the provided path
      Parameters:
      path - The path within the document in dot-notation
      Returns:
      true if the document contains a value for the field, false otherwise
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid.

    • hasField

      public boolean hasField(String path, boolean failOutOfRange)
      Checks whether the document contains a value for the provided path
      Parameters:
      path - The path within the document in dot-notation
      failOutOfRange - Whether to throw an IllegalArgumentException if array is accessed outside of its range
      Returns:
      true if the document contains a value for the field, false otherwise
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid.

    • removeField

      public void removeField(TemplateScript.Factory fieldPathTemplate)
      Removes the field identified by the provided path.
      Parameters:
      fieldPathTemplate - Resolves to the path with dot-notation within the document
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the field doesn't exist.

    • removeField

      public void removeField(String path)
      Removes the field identified by the provided path.
      Parameters:
      path - the path of the field to be removed
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the field doesn't exist.

    • appendFieldValue

      public void appendFieldValue(String path, Object value)
      Appends the provided value to the provided path in the document. Any non existing path element will be created. If the path identifies a list, the value will be appended to the existing list. If the path identifies a scalar, the scalar will be converted to a list and the provided value will be added to the newly created list. Supports multiple values too provided in forms of list, in that case all the values will be appended to the existing (or newly created) list.
      Parameters:
      path - The path within the document in dot-notation
      value - The value or values to append to the existing ones
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid.

    • appendFieldValue

      public void appendFieldValue(String path, Object value, boolean allowDuplicates)
      Appends the provided value to the provided path in the document. Any non existing path element will be created. If the path identifies a list, the value will be appended to the existing list. If the path identifies a scalar, the scalar will be converted to a list and the provided value will be added to the newly created list. Supports multiple values too provided in forms of list, in that case all the values will be appended to the existing (or newly created) list.
      Parameters:
      path - The path within the document in dot-notation
      value - The value or values to append to the existing ones
      allowDuplicates - When false, any values that already exist in the field will not be added
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid.

    • appendFieldValue

      public void appendFieldValue(TemplateScript.Factory fieldPathTemplate, ValueSource valueSource, boolean allowDuplicates)
      Appends the provided value to the provided path in the document. Any non existing path element will be created. If the path identifies a list, the value will be appended to the existing list. If the path identifies a scalar, the scalar will be converted to a list and the provided value will be added to the newly created list. Supports multiple values too provided in forms of list, in that case all the values will be appended to the existing (or newly created) list.
      Parameters:
      fieldPathTemplate - Resolves to the path with dot-notation within the document
      valueSource - The value source that will produce the value or values to append to the existing ones
      allowDuplicates - When false, any values that already exist in the field will not be added
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid.

    • setFieldValue

      public void setFieldValue(String path, Object value)
      Sets the provided value to the provided path in the document. Any non existing path element will be created. If the last item in the path is a list, the value will replace the existing list as a whole. Use appendFieldValue(String, Object) to append values to lists instead.
      Parameters:
      path - The path within the document in dot-notation
      value - The value to put in for the path key
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the value cannot be set to the item identified by the provided path.

    • setFieldValue

      public void setFieldValue(TemplateScript.Factory fieldPathTemplate, ValueSource valueSource)
      Sets the provided value to the provided path in the document. Any non existing path element will be created. If the last element is a list, the value will replace the existing list.
      Parameters:
      fieldPathTemplate - Resolves to the path with dot-notation within the document
      valueSource - The value source that will produce the value to put in for the path key
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the value cannot be set to the item identified by the provided path.

    • setFieldValue

      public void setFieldValue(TemplateScript.Factory fieldPathTemplate, ValueSource valueSource, boolean ignoreEmptyValue)
      Sets the provided value to the provided path in the document. Any non existing path element will be created. If the last element is a list, the value will replace the existing list.
      Parameters:
      fieldPathTemplate - Resolves to the path with dot-notation within the document
      valueSource - The value source that will produce the value to put in for the path key
      ignoreEmptyValue - The flag to determine whether to exit quietly when the value produced by TemplatedValue is null or empty
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the value cannot be set to the item identified by the provided path.

    • setFieldValue

      public void setFieldValue(TemplateScript.Factory fieldPathTemplate, Object value, boolean ignoreEmptyValue)
      Sets the provided value to the provided path in the document. Any non existing path element will be created. If the last element is a list, the value will replace the existing list.
      Parameters:
      fieldPathTemplate - Resolves to the path with dot-notation within the document
      value - The value to put in for the path key
      ignoreEmptyValue - The flag to determine whether to exit quietly when the value produced by TemplatedValue is null or empty
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the value cannot be set to the item identified by the provided path.

    • renderTemplate

      public String renderTemplate(TemplateScript.Factory template)

    • getSourceAndMetadata

      public Map<String,Object> getSourceAndMetadata()
      Get source and metadata map

    • getCtxMap

      public CtxMap<?> getCtxMap()
      Get the CtxMap

    • getMetadata

      public Metadata getMetadata()
      Get the strongly typed metadata

    • getSource

      public Map<String,Object> getSource()
      Get all source values in a Map

    • getIngestMetadata

      public Map<String,Object> getIngestMetadata()
      Returns the available ingest metadata fields, by default only timestamp, but it is possible to set additional ones. Use only for reading values, modify them instead using setFieldValue(String, Object) and removeField(String)

    • deepCopyMap

      public static <K, V> Map<K,V> deepCopyMap(Map<K,V> source)

    • deepCopy

      public static Object deepCopy(Object value)

    • getAllFields

      public static Set<String> getAllFields(Map<String,Object> input)

    • executePipeline

      public void executePipeline(Pipeline pipeline, BiConsumer<IngestDocument,Exception> handler)
      Executes the given pipeline with for this document unless the pipeline has already been executed for this document.
      Parameters:
      pipeline - the pipeline to execute
      handler - handles the result or failure

    • updateIndexHistory

      public boolean updateIndexHistory(String index)
      Adds an index to the index history for this document, returning true if the index was added to the index history (i.e. if it wasn't already in the index history).
      Parameters:
      index - the index to potentially add to the index history
      Returns:
      true if the index history did not already contain the index in question

    • getIndexHistory

      public Set<String> getIndexHistory()
      Returns:
      an unmodifiable view of the document's index history

    • doNoSelfReferencesCheck

      public boolean doNoSelfReferencesCheck()
      Returns:
      Whether a self referencing check should be performed

    • doNoSelfReferencesCheck

      public void doNoSelfReferencesCheck(boolean doNoSelfReferencesCheck)
      Whether the ingest framework should perform a self referencing check after this ingest document has been processed by all pipelines. Doing this check adds an extra tax to ingest and should only be performed when really needed. Only if a processor is executed that could add self referencing maps or lists then this check must be performed. Most processors will not be able to do this, hence the default is false.
      Parameters:
      doNoSelfReferencesCheck - Whether a self referencing check should be performed

    • equals

      public boolean equals(Object obj)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • reroute

      public void reroute(String destIndex)