XQuery .xqm files transferred to eXist-db using WebDAV are not searchable in "Edit -> Find in files"

[nsincaglia]

What is the problem

I installed a new version of eXist-db v6.0.1 on an AWS Linux 2 instance. I transferred some XQuery files from another server to my new server using WebDAV. Some XQuery files are named with a .xq extension and other XQuery files are named with the .xqm extension. After moving the files to the new server using WebDAV, I tried to search for a variable name that was contained in both the .xq and .xqm XQuery files using "Edit -> Find in files" in eXide. The search returned the variable name that I was looking for in the .xq XQuery files but the variable name was not able to be found in the .xqm XQuery files.

What did you expect

I would expect that both .xq and .xqm files would both be recognized as XQuery files and would be searchable using "Edit -> Find in files".

Describe how to reproduce or add a test

Transfer an .xq XQuery file using WebDAV to a eXist-db v6.0.1 server and Transfer an .xqm XQuery file using WebDAV. Then navigate to "Edit -> Find in files" and search for a variable or function in both the .xq and .xqm file.

Context information

Please always add the following information

• eXist-db v6.0.1
  openjdk version "1.8.0_312"
  OpenJDK Runtime Environment (build 1.8.0_312-b07)
  OpenJDK 64-Bit Server VM (build 25.312-b07, mixed mode)
  AWS Linux 2
  64 bit
  JAR installer

[joewiz]

@nsincaglia The module in eXide that receives the requests from the "Edit > Find in files" command is https://github.com/eXist-db/eXide/blob/develop/modules/search.xq. This module doesn't examine files' extensions; rather, it checks the files' mime types to identify XQuery, Javascript, CSS, LESS, and plain text files—the 5 types of files that the module will search within.

If there is an XQuery module that eXide fails to find expected hits in, could you check its mime type? To do so for the module linked above, for example, run this:

xmldb:get-mime-type(xs:anyURI("/db/apps/eXide/modules/search.xq"))

If it returns the expected application/xquery (which eXist automatically maps to all of the XQuery file extensions listed in https://github.com/eXist-db/exist/blob/develop/exist-core/src/main/resources/org/exist/util/mime-types.xml#L105-L108), then please double-check the module to ensure it really contains the text you're searching for. Other troubleshooting tips: do you get the same unexpected results on both servers, or only the one where you copied the files to? Could you post a screenshot of the "Edit > Find in files" showing all of the input fields? Could you check to make sure the user you logged into with eXide has read privileges on the modules in question?

If it says that your .xqm flie has a mime type other than application/xquery, then we'll need to dig into how that could possibly have come to pass.

[nsincaglia]

Before I submitted the ticket for this issue, I copied two XQuery files from my old server to my new server using WebDAV. When I run xmldb:get-mime-type() function on both files on the old server they both return application/xquery. When I run the xmldb:get-mime-type() function on the new server for both files it also returns application/xquery for both files. Yet, when I use "Edit -> Search in files" for text in the .xq file I can find what I am searching for but I can't find my search term when searching the .xqm file.

I created two brand new files on my old server and moved them to the new server using WebDAV and I was able to search and find the variables in both the .xq and .xqm files.

Maybe the file extension has nothing to do with this. I just know I can consistently reproduce this issue by selecting .xq and .xqm files on my old server and moving them using WebDAV over to the new server. The .xqm files contain function and variable definitions, whereas the .xq file are just simple XQuery scripts. Maybe there is something about the .xqm files that is causing this issue. I am not sure. I would be happy to demonstrate the issue for anyone who would like to see it for themselves.

[joewiz]

@nsincaglia Seeing it isn't nearly as useful as having steps I can reproduce on a stock eXist system running locally. Can you craft a set of steps to reproduce this on a local system?

[joewiz]

Also, recall I provided several questions to answer in the case that the mime type of the module in question was the expected value of application/xquery. These questions:

... then please double-check the module to ensure it really contains the text you're searching for. Other troubleshooting tips: do you get the same unexpected results on both servers, or only the one where you copied the files to? Could you post a screenshot of the "Edit > Find in files" showing all of the input fields? Could you check to make sure the user you logged into with eXide has read privileges on the modules in question?

[nsincaglia]

Yes, I have verified that the text I am looking for is in the file. This issue is only happening my new eXist-db v6.0.1 system I setup. I set up this system twice and in both cases I experienced this issue. After the second time, I was convinced that it was not something I did during installation so I decided to report the issue. I have done some more testing today and I am finding that this issue still exists but the issue does not happen consistently. Sometimes I move a file over and I can perform a search and it is found. Other times, I move a file and perform a search and it is not found. The screen shot image I have attached show that I have a variable $nick-sincaglia-2 in a file I transferred using WebDav. The search does not find it.

[joewiz]

On today's Community Call, @nsincaglia added details:

using Transmit to copy from one server to another

@dizzzz added that WebDAV can support copying from one server to another, but eXist doesn't support this. (User and permission info would not be transmitted.) He suggested trying to copy the file from the source server to the PC, and from the PC to the destination server. Nick said he'll try that.

[dizzzz]

I have seen in 6.x similar issues where the find function does to search all xquery code; As a result I removed too many functions resulting in a broken app :-)