Trying to fix the Python Driver for 3.2

[odysseaspenta]

I am trying to ensure that the Python driver for 3.2 works fine since there are some warning on the pyorient project site about using it in production with 3.1+. Some of the test cases are failing and I am investigating the issues and found that what the python test case is expecting in response to creating a class is not what the server is now returning.

For example, one test case creates a class
response = client.command("create class my_class extends V")

and then attempts to create a record in this class. In creating the record, it specifies the cluster_id for the record which it is expecting as a response to the create class command. The create class command though returns the number of classes and not the clusters of the newly created class.

In OCommandExecutorSQLCreateClass it returns size instead of cluster_ids
` public Object execute(final Map<Object, Object> iArgs) {
if (className == null)
throw new OCommandExecutionException(
"Cannot execute the command because it has not been parsed yet");

final ODatabaseDocument database = getDatabase();

boolean alreadyExists = database.getMetadata().getSchema().existsClass(className);
if (!alreadyExists || !ifNotExists) {
  if (clusters != null)
    database
        .getMetadata()
        .getSchema()
        .createClass(className, clusters, superClasses.toArray(new OClass[0]));
  else
    database
        .getMetadata()
        .getSchema()
        .createClass(className, clusterIds, superClasses.toArray(new OClass[0]));
}
return database.getMetadata().getSchema().getClasses().size();

}`

Any suggestions on how this should be fixed and why it broke at some point?

[tglman]

Hi,

so yes i think this was a regression, also the executor you are pointing to is the legacy executor(and legacy query engine), that may be completely removed in future, I would say if you want to upgrade the driver to the new version, don't be afraid to break the compatibility with the old version, better a not compatible working version than no version.

Regards

[odysseaspenta]

Can you point me to the old and new executors in the source code and I can investigate further. I looked around to try and find them on my own but I couldn't find how the binary protocol determines the executor that is in effect. It would definitely be better for me to take some time to find the right way to move the Python driver forward rather than to hack a fix that would only be temporary. Given the adoption of Python as the primary language for data science and machine learning, it is imperative that Orientdb has a Python driver that is up to date.

[odysseaspenta]

I had some more time to investigate this a bit and it looks like the binary protocol doesn't use the new executor and we would need to use the http protocol in the python driver to switch to the new executor. Is that correct? I am going to need a bit of help to get this started because I don't want to go down the wrong path.

[tglman]

Hi,
Sorry for the late reply the new query engine is also integrated in the binary protocol, it just use a different message[s] and is a bit more complex because it allow to return a limited set of results and then fetch more later, let me point you to the java code related to this parts:

the query request : https://github.com/orientechnologies/orientdb/blob/develop/client/src/main/java/com/orientechnologies/orient/client/remote/message/OQueryRequest.java
the query response: https://github.com/orientechnologies/orientdb/blob/develop/client/src/main/java/com/orientechnologies/orient/client/remote/message/OQueryResponse.java

the logic server side that execute the query: https://github.com/orientechnologies/orientdb/blob/develop/server/src/main/java/com/orientechnologies/orient/server/OConnectionBinaryExecutor.java#L1321

the java client side use of the query request in simple idempotent queries: https://github.com/orientechnologies/orientdb/blob/develop/client/src/main/java/com/orientechnologies/orient/client/remote/OStorageRemote.java#L1109 commads (insert,update,delete, ddls): https://github.com/orientechnologies/orientdb/blob/develop/client/src/main/java/com/orientechnologies/orient/client/remote/OStorageRemote.java#L1160 and server side scripts request: https://github.com/orientechnologies/orientdb/blob/develop/client/src/main/java/com/orientechnologies/orient/client/remote/OStorageRemote.java#L1274

the request for fetch more results from a running query: https://github.com/orientechnologies/orientdb/blob/develop/client/src/main/java/com/orientechnologies/orient/client/remote/message/OQueryNextPageRequest.java that will use the same message OQueryResponse for return the next values.

relative request code client side: https://github.com/orientechnologies/orientdb/blob/develop/client/src/main/java/com/orientechnologies/orient/client/remote/OStorageRemote.java#L1336

the execution of next page : https://github.com/orientechnologies/orientdb/blob/develop/server/src/main/java/com/orientechnologies/orient/server/OConnectionBinaryExecutor.java#L1393

than if your query is been terminated before all result are fetched it need to be closed with : https://github.com/orientechnologies/orientdb/blob/develop/client/src/main/java/com/orientechnologies/orient/client/remote/message/OCloseQueryRequest.java here the execution of the close https://github.com/orientechnologies/orientdb/blob/develop/server/src/main/java/com/orientechnologies/orient/server/OConnectionBinaryExecutor.java#L1382 and a bit empty response message https://github.com/orientechnologies/orientdb/blob/develop/client/src/main/java/com/orientechnologies/orient/client/remote/message/OCloseQueryResponse.java

here is the client side logic for close: https://github.com/orientechnologies/orientdb/blob/develop/client/src/main/java/com/orientechnologies/orient/client/remote/OStorageRemote.java#L1330

I Hope this help.

Regards