Current protocol version for 2.0-SNAPSHOT: 28. Look at compatibility for retro-compatibility.
The OrientDB binary protocol is the fastest way to interface a client application to an OrientDB Server instance. The aim of this page is to provide a starting point from which to build a language binding, maintaining high-performance.
If you'd like to develop a new binding, please take a look to the available ones before starting a new project from scratch: Existent Drivers.
Also, check the available REST implementations.
Before starting, please note that:
For more in-depth information please look at the Java classes:
(Since 0.9.24-SNAPSHOT Nov 25th 2010) Once connected, the server sends a short number (2 byte) containing the binary protocol number. The client should check that it supports that version of the protocol. Every time the protocol changes the version is incremented.
After the connection has been established, a client can Connect to the server or request the opening of a database Database Open. Currently, only TCP/IP raw sockets are supported. For this operation use socket APIs appropriate to the language you're using. After the Connect and Database Open all the client's requests are sent to the server until the client closes the socket. When the socket is closed, OrientDB Server instance frees resources the used for the connection.
The first operation following the socket-level connection must be one of:
In both cases a Session-Id is sent back to the client. The server assigns a unique Session-Id to the client. This value must be used for all further operations against the server. You may open a database after connecting to the server, using the same Session-Id
The session managment is implemented in two different way, one stateful another stateless this is choosed in the open/connect operation with a flag, the stateful is based on a Session-id the stateless is based on a Token
All the operations that follow the open/connect must contain, as the first parameter, the client Session-Id (as Integer, 4 bytes) and it will be sent back on completion of the request just after the result field.
NOTE: In order to create a new server-side connection, the client must send a negative number into the open/connect calls.
This Session-Id can be used into the client to keep track of the requests if it handles multiple session bound to the same connection. In this way the client can implement a sharing policy to save resources. This requires that the client implementation handle the response returned and dispatch it to the correct caller thread.
All the operation in a stateless session are based on the token, the token is a byte[] that contains all the information for the interaction with the server, the token is acquired at the mement of open or connect, and need to be resend for each request. the session id used in the stateful requests is still there and is used to associate the request to the response. in the response can be resend a token in case of expire renew.
To make the development of a new client easier it's strongly suggested to activate debug mode on the binary channel. To activate this, edit the file orientdb-server-config.xml and configure the new parameter "network.binary.debug" on the "binary" or "distributed" listener. E.g.:
...
<listener protocol="distributed" port-range="2424-2430"
ip-address="127.0.0.1">
<parameters>
<parameter name="network.binary.debug" value="true" />
</parameters>
</listener>
...
In the log file (or the console if you have configured the orientdb-server-log.properties file) all the packets received will be printed.
This is the typical exchange of messages between client and server sides:
+------+ +------+
|Client| |Server|
+------+ +------+
| TCP/IP Socket connection |
+-------------------------->|
| DB_OPEN |
+-------------------------->|
| RESPONSE (+ SESSION-ID) |
+<--------------------------+
... ...
| REQUEST (+ SESSION-ID) |
+-------------------------->|
| RESPONSE (+ SESSION-ID) |
+<--------------------------+
... ...
| DB_CLOSE (+ SESSION-ID) |
+-------------------------->|
| TCP/IP Socket close |
+-------------------------->|
In explaining the network messages these conventions will be used:
(length:int)
The network protocol supports different types of information:
Type | Minimum length in bytes | Maximum length in bytes | Notes | Example |
---|---|---|---|---|
boolean | 1 | 1 | Single byte: 1 = true, 0 = false | 1 |
byte | 1 | 1 | Single byte, used to store small numbers and booleans | 1 |
short | 2 | 2 | Signed short type | 01 |
int | 4 | 4 | Signed integer type | 0001 |
long | 8 | 8 | Signed long type | 00000001 |
bytes | 4 | N | Used for binary data. The format is (length:int)bytes) . Send -1 as NULL | 000511111 |
string | 4 | N | Used for text messages.The format is: (length:int)bytes) . Send -1 as NULL | 0005Hello |
record | 2 | N | An entire record serialized. The format depends if a RID is passed or an entire record with its content. In case of null record then -2 as short is passed. In case of RID -3 is passes as short and then the RID: (-3:short)(cluster-id:short)(cluster-position:long) . In case of record: (0:short)(record-type:byte)(cluster-id:short)(cluster-position:long)(record-version:int)(record-content:bytes) | |
strings | 4 | N | Used for multiple text messages. The format is: (length:int)[(Nth-string:string)] | 00020005Hello0007World! |
The record format is choose during the CONNECT or DB_OPEN request, the formats available are:
CSV (serialization-impl value "ORecordDocument2csv") Binary (serialization-impl value "ORecordSerializerBinary")
The CSV format is the default for all the versions 0. and 1. or for any client with Network Protocol Version < 22
Each request has own format depending of the operation requested. The operation requested is indicated in the first byte:
Command | Value as byte | Description | Async | Since |
---|---|---|---|---|
Server (CONNECT Operations) | ||||
REQUEST_SHUTDOWN | 1 | Shut down server. | no | |
REQUEST_CONNECT | 2 | Required initial operation to access to server commands. | no | |
REQUEST_DB_OPEN | 3 | Required initial operation to access to the database. | no | |
REQUEST_DB_CREATE | 4 | Add a new database. | no | |
REQUEST_DB_EXIST | 6 | Check if database exists. | no | |
REQUEST_DB_DROP | 7 | Delete database. | no | |
REQUEST_CONFIG_GET | 70 | Get a configuration property. | no | |
REQUEST_CONFIG_SET | 71 | Set a configuration property. | no | |
REQUEST_CONFIG_LIST | 72 | Get a list of configuration properties. | no | |
REQUEST_DB_LIST | 74 | Get a list of databases. | no | 1.0rc6 |
Database (DB_OPEN Operations) | ||||
REQUEST_DB_CLOSE | 5 | Close a database. | no | |
REQUEST_DB_SIZE | 8 | Get the size of a database (in bytes). | no | 0.9.25 |
REQUEST_DB_COUNTRECORDS | 9 | Get total number of records in a database. | no | 0.9.25 |
REQUEST_DATACLUSTER_ADD | 10 | Add a data cluster. | no | |
REQUEST_DATACLUSTER_DROP | 11 | Delete a data cluster. | no | |
REQUEST_DATACLUSTER_COUNT | 12 | Get the total number of data clusters. | no | |
REQUEST_DATACLUSTER_DATARANGE | 13 | Get the data range of data clusters. | no | |
REQUEST_DATACLUSTER_COPY | 14 | Copy a data cluster. | no | |
REQUEST_DATACLUSTER_LH_CLUSTER_IS_USED | 16 | no | 1.2.0 | |
REQUEST_RECORD_METADATA | 29 | Get metadata from a record. | no | 1.4.0 |
REQUEST_RECORD_LOAD | 30 | Load a record. | no | |
REQUEST_RECORD_CREATE | 31 | Add a record. | yes | |
REQUEST_RECORD_UPDATE | 32 | yes | | |
REQUEST_RECORD_DELETE | 33 | Delete a record. | yes | |
REQUEST_RECORD_COPY | 34 | Copy a record. | yes | |
REQUEST_RECORD_CLEAN_OUT | 38 | Clean out record. | yes | 1.3.0 |
REQUEST_POSITIONS_FLOOR | 39 | Get the last record. | yes | 1.3.0 |
REQUEST_COUNT (DEPRECATED) | 40 | See REQUEST_DATACLUSTER_COUNT | no | |
REQUEST_COMMAND | 41 | Execute a command. | no | |
REQUEST_POSITIONS_CEILING | 42 | Get the first record. | no | 1.3.0 |
REQUEST_TX_COMMIT | 60 | Commit transaction. | no | |
REQUEST_DB_RELOAD | 73 | Reload database. | no | 1.0rc4 |
REQUEST_PUSH_RECORD | 79 | no | 1.0rc6 | |
REQUEST_PUSH_DISTRIB_CONFIG | 80 | no | 1.0rc6 | |
REQUEST_DB_COPY | 90 | no | 1.0rc8 | |
REQUEST_REPLICATION | 91 | no | 1.0 | |
REQUEST_CLUSTER | 92 | no | 1.0 | |
REQUEST_DB_TRANSFER | 93 | no | 1.0.2 | |
REQUEST_DB_FREEZE | 94 | no | 1.1.0 | |
REQUEST_DB_RELEASE | 95 | no | 1.1.0 | |
REQUEST_DATACLUSTER_FREEZE | 96 | no | ||
REQUEST_DATACLUSTER_RELEASE | 97 | no | ||
REQUEST_CREATE_SBTREE_BONSAI | 110 | Creates an sb-tree bonsai on the remote server | no | 1.7rc1 |
REQUEST_SBTREE_BONSAI_GET | 111 | Get value by key from sb-tree bonsai | no | 1.7rc1 |
REQUEST_SBTREE_BONSAI_FIRST_KEY | 112 | Get first key from sb-tree bonsai | no | 1.7rc1 |
REQUEST_SBTREE_BONSAI_GET_ENTRIES_MAJOR | 113 | Gets the portion of entries major than specified one. If returns 0 entries than the specified entrie is the largest | no | 1.7rc1 |
REQUEST_RIDBAG_GET_SIZE | 114 | Rid-bag specific operation. Send but does not save changes of rid bag. Retrieves computed size of rid bag. | no | 1.7rc1 |
Every request has a response unless the command supports the asynchronous mode (look at the table above).
Every time the client sends a request, and the command is not in asynchronous mode (look at the table above), client must read the one-byte response status that indicates OK or ERROR. The rest of response bytes depends on this first byte.
* OK = 0;
* ERROR = 1;
OK response bytes are depends for every request type. ERROR response bytes sequence described below.
The format is: [(1)(exception-class:string)(exception-message:string)]*(0)(serialized-exception:bytes)
The pairs exception-class and exception-message continue while the following byte is 1. A 0 in this position indicates that no more data follows.
E.g. (parentheses are used here just to separate fields to make this easier to read: they are not present in the server response):
(1)(com.orientechnologies.orient.core.exception.OStorageException)(Can't open the storage 'demo')(0)
Example of 2 depth-levels exception:
(1)(com.orientechnologies.orient.core.exception.OStorageException)(Can't open the storage 'demo')(1)(com.orientechnologies.orient.core.exception.OStorageException)(File not found)(0)
Since 1.6.1 we also send serialized version of exception thrown on server side. This allows to preserve full stack trace of server exception on client side but this feature can be used by Java clients only.
This section explains the request and response messages of all suported operations.
Shut down the server. Requires "shutdown" permission to be set in orientdb-server-config.xml file.
Request: (user-name:string)(user-password:string)
Response: empty
Typically the credentials are those of the OrientDB server administrator. This is not the same as the admin user for individual databases.
This is the first operation requested by the client when it needs to work with the server instance. It returns the session id of the client.
Request: (driver-name:string)(driver-version:string)(protocol-version:short)(client-id:string)(serialization-impl:string)(token-session:boolean)(user-name:string)(user-password:string)
Response: (session-id:int)(token:bytes)
Where:
request content:
response content:
This is the first operation the client should call. It opens a database on the remote OrientDB Server. Returns the Session-Id to being reused for all the next calls and the list of configured clusters.
Request: (driver-name:string)(driver-version:string)(protocol-version:short)(client-id:string)(serialization-impl:string)(token-session:boolean)(database-name:string)(database-type:string)(user-name:string)(user-password:string)
Response: (session-id:int)(token:bytes)(num-of-clusters:short)[(cluster-name:string)(cluster-id:short)](cluster-config:bytes.md)(orientdb-release:string)
Where: request detail:
response detail :
Creates a database in the remote OrientDB server instance
Request: (database-name:string)(database-type:string)(storage-type:string)
Response: empty
Where:
NB. It doesn't make sense to use "remote" in this context
Closes the database and the network connection to the OrientDB Server instance. No return is expected. The socket is also closed.
Request: empty
Response: no response, the socket is just closed at server side
Asks if a database exists in the OrientDB Server instance. It returns true (non-zero) or false (zero).
Request: (database-name:string) <-- before 1.0rc1 this was empty (server-storage-type:string - since 1.5-snapshot)
Response: (result:byte)
Where:
Reloads database information. Available since 1.0rc4.
Request: empty
Response:(num-of-clusters:short)[(cluster-name:string)(cluster-id:short)]
Removes a database from the OrientDB Server instance. It returns nothing if the database has been deleted or throws a OStorageException if the database doesn't exists.
Request: (database-name:string)(server-storage-type:string - since 1.5-snapshot)
Response: empty
Where:
Asks for the size of a database in the OrientDB Server instance.
Request: empty
Response: (size:long)
Asks for the number of records in a database in the OrientDB Server instance.
Request: empty
Response: (count:long)
Add a new data cluster.
Request: (name:string)(cluster-id:short - since 1.6 snapshot)
Response: (new-cluster:short)
Where: type is one of "PHYSICAL" or "MEMORY". If cluster-id is -1 (recommended value) new cluster id will be generated.
Remove a cluster.
Request: (cluster-number:short)
Response: (delete-on-clientside:byte)
Where:
Returns the number of records in one or more clusters.
Request: (cluster-count:short)(cluster-number:short)*(count-tombstones:byte)
Response: (records-in-clusters:long)
Where:
Request the record count for clusters 5, 6 and 7. Note the "03" at the beginning to tell you're passing 3 cluster ids (as short each). 1,000 as long (8 bytes) is the answer.
Request: 03050607
Response: 00001000
Returns the range of record ids for a cluster.
Request: (cluster-number:short)
Response: (begin:long)(end:long)
Request the range for cluster 7. The range 0-1,000 is returned in the response as 2 longs (8 bytes each).
Request: 07
Response: 0000000000001000
Load a record by RecordID, according to a fetch plan
Request: (cluster-id:short)(cluster-position:long)(fetch-plan:string)(ignore-cache:byte)(load-tombstones:byte)
Response: [(payload-status:byte)[(record-type:byte)(record-version:int)(record-content:bytes)]*]+
Where:
Create a new record. Returns the position in the cluster of the new record. New records can have version > 0 (since v1.0) in case the RID has been recycled.
Request: (cluster-id:short)(record-content:bytes)(record-type:byte)(mode:byte)
Response: (cluster-id:short)(cluster-position:long)(record-version:int)(count-of-collection-changes)[(uuid-most-sig-bits:long)(uuid-least-sig-bits:long)(updated-file-id:long)(updated-page-index:long)(updated-page-offset:int)]*
Where:
and mode is:
The last part of response is referred to RidBag management. Take a look at the main page for more details.
Update a record. Returns the new record's version.
Request: (cluster-id:short)(cluster-position:long)(update-content:boolean)(record-content:bytes)(record-version:int)(record-type:byte)(mode:byte)
Response: (record-version:int)(count-of-collection-changes)[(uuid-most-sig-bits:long)(uuid-least-sig-bits:long)(updated-file-id:long)(updated-page-index:long)(updated-page-offset:int)]*
Where record-type is:
and record-version policy is:
and mode is:
and update-content is:
The last part of response is referred to RidBag management. Take a look at the main page for more details.
Delete a record by its RecordID. During the optimistic transaction the record will be deleted only if the versions match. Returns true if has been deleted otherwise false.
Request: (cluster-id:short)(cluster-position:long)(record-version:int)(mode:byte)
Response: (payload-status:byte)
Where:
Executes remote commands:
Request: (mode:byte)(command-payload-length:int)(class-name:string)(command-payload)
Response:
- synchronous commands: [(synch-result-type:byte)[(synch-result-content:?)]]+
- asynchronous commands: [(asynch-result-type:byte)[(asynch-result-content:?)]*](pre-fetched-record-size.md)[(pre-fetched-record)]*+
Where the request:
com.orientechnologies.orient.core.sql.query.OSQLSynchQuery
com.orientechnologies.orient.core.sql.OCommandSQL
com.orientechnologies.orient.core.command.script.OCommandScript
. Script commands by using any supported server-side scripting like Javascript command. Since v1.0.fromStream()
method against itResponse is different for synchronous and asynchronous request:
Commits a transaction. This operation flushes all the pending changes to the server side.
Request: (tx-id:int)(using-tx-log:byte)(tx-entry)*(0-byte indicating end-of-records)
tx-entry: (operation-type:byte)(cluster-id:short)(cluster-position:long)(record-type:byte)(entry-content)
entry-content for CREATE: (record-content:bytes)
entry-content for UPDATE: (version:record-version)(content-changed:boolean)(record-content:bytes)
entry-content for DELETE: (version:record-version)
Response: (created-record-count:int)[(client-specified-cluster-id:short)(client-specified-cluster-position:long)(created-cluster-id:short)(created-cluster-position:long)]*(updated-record-count:int)[(updated-cluster-id:short)(updated-cluster-position:long)(new-record-version:int)]*(count-of-collection-changes:int)[(uuid-most-sig-bits:long)(uuid-least-sig-bits:long)(updated-file-id:long)(updated-page-index:long)(updated-page-offset:int)]*
Where:
(original-record-version:int)(record-content:bytes)
(original-record-version:int)
(record-content:bytes)
This response contains two parts: a map of 'temporary' client-generated record ids to 'real' server-provided record ids for each CREATED record, and a map of UPDATED record ids to update record-versions.
Look at Optimistic Transaction to know how temporary RecordIDs are managed.
The last part or response is referred to RidBag management. Take a look at the main page for more details.
Request: (clusterId:int)
Response: (collectionPointer)
See: serialization of collection pointer
Creates an sb-tree bonsai on the remote server.
Request: (collectionPointer)(key:binary)
Response: (valueSerializerId:byte)(value:binary)
See: serialization of collection pointer
Get value by key from sb-tree bonsai.
Key and value are serialized according to format of tree serializer. If the operation is used by RidBag key is always a RID and value can be null or integer.
Request: (collectionPointer)
Response: (keySerializerId:byte)(key:binary)
See: serialization of collection pointer
Get first key from sb-tree bonsai. Null if tree is empty.
Key are serialized according to format of tree serializer. If the operation is used by RidBag key is null or RID.
Request: (collectionPointer)(key:binary)(inclusive:boolean)(pageSize:int)
Response: (count:int)[(key:binary)(value:binary)]*
See: serialization of collection pointer
Gets the portion of entries major than specified one. If returns 0 entries than the specified entry is the largest.
Keys and values are serialized according to format of tree serializer. If the operation is used by RidBag key is always a RID and value is integer.
Default pageSize is 128.
Request: (collectionPointer)(collectionChanges)
Response: (size:int)
See: serialization of collection pointer, serialization of collection changes
Rid-bag specific operation. Send but does not save changes of rid bag. Retrieves computed size of rid bag.
NOTE. Since 1.7rc1 this feature is deprecated. Usage of RidBag is preferable.
Starting from 1.0rc8-SNAPSHOT OrientDB can transform collections of links from the classic mode:
[#10:3,#10:4,#10:5]
to:
(ORIDs@pageSize:16,root:#2:6)
For more information look at the announcement of this new feature: https://groups.google.com/d/topic/orient-database/QF52JEwCuTM/discussion
In practice to optimize cases with many relationships/edges the collection is transformed in a mvrb-tree. This is because the embedded object. In that case the important thing is the link to the root node of the balanced tree.
You can disable this behaviour by setting
mvrbtree.ridBinaryThreshold = -1
Where mvrbtree.ridBinaryThreshold is the threshold where OrientDB will use the tree instead of plain collection (as before). -1 means "hey, never use the new mode but leave all as before".
To improve performance this structure is managed in binary form. Below how is made:
+-----------+-----------+--------+------------+----------+-----------+---------------------+
| TREE SIZE | NODE SIZE | COLOR .| PARENT RID | LEFT RID | RIGHT RID | RID LIST .......... |
+-----------+-----------+--------+------------+----------+-----------+---------------------+
| 4 bytes . | 4 bytes . | 1 byte | 10 bytes ..| 10 bytes | 10 bytes .| 10 * MAX_SIZE bytes |
+-----------+-----------+--------+------------+----------+-----------+---------------------+
= 39 bytes + 10 * PAGE-SIZE bytes
Where:
The size of the tree-node on disk (and memory) is fixed to avoid fragmentation. To compute it: 39 bytes + 10 * PAGE-SIZE bytes. For a page-size = 16 you'll have 39 + 160 = 199 bytes.
Since version 28 the REQUEST_RECORD_LOAD response order is changed from:[(payload-status:byte)[(record-content:bytes)(record-version:int)(record-type:byte)]*]+
to:[(payload-status:byte)[(record-type:byte)(record-version:int)(record-content:bytes)]*]+
Since version 27 is introduced an extension to allow use a token based session, if this modality is enabled a few things change in the modality the protocol works.
protocol methods changed:
REQUEST_DB_OPEN
REQUEST_CONNECT
added cluster-id in the REQUEST_CREATE_RECORD response.
Reviewd serialization of index changes in the REQUEST_TX_COMMIT for detais #2676 Removed double serialization of commands parameters, now the parameters are directly serialized in a document see Network Binary Protocol Commands and #2301
updateContent
flag to UPDATE_RECORD and COMMITCurrent release of OrientDB server supports older client versions.