OrientDB Manual 1.7.8

ETL Transformers

Transformer components are executed in pipeline. They work against the received input returning an output.

Before the execution, the $input variable is always assigned, so you can get at run-time and use if needed.

Available Transformers
CSV FIELD MERGE VERTEX
CODE LINK EDGE SKIP
LOG BLOCK COMMAND

CSV

Converts a String in a Document parsing it as CSV.

  • Component name: csv
  • Supported inputs types: [String]
  • Output: ODocument

Syntax

Parameter Description Type Mandatory Default value
separator Column separator char false ,
columnsOnFirstLine Columns are described in the first line boolean false true
columns Columns array containing names, and optionally types by postfixing names with :. Specifying type guarantee better performance string[] false -
nullValue value to consider as NULL. Default is not declared string false -
stringCharacter String character delimiter char false "
skipFrom Line number where start to skip integer true -
skipTo Line number where skip ends integer true -

Example

Transforms a row in CSV (as ODocument), using comma as separator, considering "NULL" as null value and skipping the rows 2-4:

{ "csv": { "separator": ",", "nullValue": "NULL",
           "skipFrom": 1, "skipTo": 3 } }

FIELD

Execute a SQL transformation against a field.

  • Component name: vertex
  • Supported inputs types: [ODocument]
  • Output: ODocument

Syntax

Parameter Description Type Mandatory Default value
fieldName Document's field name to assign string true -
expression Expression to evaluate. You can use OrientDB SQL syntax string true -

Examples

Transforms the field 'class' into the ODocument's class by prefixing it with '_':

{ "field": { "fieldName": "@class", "expression": "class.prefix('_')"} }

Applies the class name based on the value of another field:

{ "field": { "fieldName": "@class", "expression": "if( ( fileCount >= 0 ), 'D', 'F')"} }

Assigns to the "name" field the last part of a path:

{ "field": { "fieldName": "name",
      "expression": "path.substring( eval( '$current.path.lastIndexOf(\"/\") + 1') )" }

MERGE

Merges input ODocument with another one, loaded by a lookup. Lookup can be a lookup against an index or a SELECT query.

  • Component name: merge
  • Supported inputs types: [ODocument, OrientVertex]
  • Output: ODocument

Syntax

Parameter Description Type Mandatory Default value
joinFieldName Field name where the join value is saved string true -
lookup Can be the index name where to execute the lookup, or a SELECT query string true -
unresolvedLinkAction Action to execute in case the JOIN hasn't been resolved. Actions can be: 'NOTHING' (do nothing), WARNING (increment warnings), ERROR (increment errors), HALT (interrupt the process), SKIP (skip current row). string false NOTHING

Example

Merges current record against the record returned by the lookup on index "V.URI" with the value contained in the field "URI" of the input's document:

{ "merge": { "joinFieldName":"URI", "lookup":"V.URI" } }

VERTEX

Transforms a ODocument in a OrientVertex.

  • Component name: vertex
  • Supported inputs types: [ODocument, OrientVertex]
  • Output: OrientVertex

Syntax

Parameter Description Type Mandatory Default value
class Vertex class name to assign string false V

Example

Transform the ODocument in a Vertex setting as class the value of "$className" variable:

{ "vertex": { "class": "$className"} }

EDGE

Transform a JOIN value in one or more EDGEs between current vertex and all the vertices returned by the lookup. Lookup can be a lookup against an index or a SELECT query.

  • Component name: EDGE
  • Supported inputs types: [ODocument, OrientVertex]
  • Output: OrientVertex

Syntax

Parameter Description Type Mandatory Default value
joinFieldName Field name where the join value is saved string true -
direction Edge direction string false 'out'
class Edge's class name string false 'E'
lookup Can be the index name where to execute the lookup, or a SELECT query string true -
unresolvedLinkAction Action to execute in case the JOIN hasn't been resolved. Actions can be: 'NOTHING' (do nothing), CREATE (create a OrientVertex setting as primary key the join value), WARNING (increment warnings), ERROR (increment errors), HALT (interrupt the process), SKIP (skip current row). string false NOTHING

Example

Creates an EDGE from the current vertex, with class "Parent", to all the vertices returned by the lookup on "D.inode" index with the value contained in the field "inode_parent" of the input's vertex:

{ "edge": { "class": "Parent", "joinFieldName": "inode_parent",
            "lookup":"D.inode", "unresolvedLinkAction":"CREATE"} }

SKIP

Skip an execution in pipeline if the condition in "expression" field is true.

  • Component name: skip
  • Supported inputs types: [ODocument, OrientVertex]
  • Output: same type as input

Syntax

Parameter Description Type Mandatory Default value
expression SQL expression to evaluate. If true the current execution is skipped string true -

Example

Skip the current record if name is null:

{ "skip": { "expression": "name is null"} }

CODE

Executes a snippet of code in any of the JVM supported languages. Default is Javascript. Last object in the code is returned as output.

  • Component name: code
  • Supported inputs types: [Object]
  • Output: Object

Syntax

Parameter Description Mandatory Default value
language Programming language used string false Javascript
code Code to execute string true -

Example

Displays current record and returns the parent.

{ "code": { "language": "Javascript",
            "code": "print('Current record: ' + record); record.field('parent');"}
}

Transform a JOIN value in LINK in current record with the result of the lookup. Lookup can be a lookup against an index or a SELECT query.

  • Component name: link
  • Supported inputs types: [ODocument, OrientVertex]
  • Output: ODocument

Syntax

Parameter Description Type Mandatory Default value
joinFieldName Field name where the join value is saved string false -
joinValue Value to lookup string false -
linkFieldName Field name containing the link to set string true -
linkFieldType Type of link between: LINK, LINKSET and LINKLIST string true -
lookup Can be the index name where to execute the lookup, or a SELECT query string true -
unresolvedLinkAction Action to execute in case the JOIN hasn't been resolved. Actions can be: 'NOTHING' (do nothing), CREATE (create a ODocument setting as primary key the join value), WARNING (increment warnings), ERROR (increment errors), HALT (interrupt the process), SKIP (skip current row). string false NOTHING

Example

Transform a JOIN value in LINK in current record (set as "parent" of type LINK) with the result of the lookup on index "D.inode" with the value contained in the field "inode_parent" of the input's document:

{ "link": { "linkFieldName": "parent", "linkFieldType": "LINK",
            "joinFieldName": "inode_parent", "lookup":"D.inode", "unresolvedLinkAction":"CREATE"} }

LOG

Logs the input object to System.out.

  • Component name: log
  • Supported inputs types: [Any]
  • Output: Any

Syntax

Parameter Description Type Mandatory Default value
prefix Prefix to write before the content string false -
postfix Postfix to write after the content string false -

Example

Simply log current value:

{ "log": {} }

Log current value with "-> " as prefix:

{ "log": { "prefix" : "-> "} }

Block

Executes a Block as transformation step.

  • Component name: block
  • Supported inputs types: [Any]
  • Output: Any

Syntax

Parameter Description Type Mandatory Default value
block Block to execute document true -

Example

Simply log current value:

{ "block": {
    "let": {
      "name": "id",
      "value": "={eval('$input.amount * 2')}"
    }
  }
}

Command

Executes a command.

  • Component name: command
  • Supported inputs types: [ODocument]
  • Output: ODocument

Syntax

Parameter Description Type Mandatory Default value
language Command language. Available are: sql (default) and gremlin string false sql
command Command to execute string true -

Example

{
  "command" : {
    "command" : "select from E where id = ${edgeid}",
    "output" : "edge"
  }
}