Applicability to SparkSQL/PySpark/Parquet file types #362
-
|
Hello, I'm exploring the use of OMOP and this DQ segment on spark clusters and was curious to know if it is compatible with SparkSQL/PySpark/Parquet file types? Any pointers to resources would be warmly welcomed!! Kind Regards, |
Beta Was this translation helpful? Give feedback.
Answered by
katy-sadowski
Jun 13, 2023
Replies: 1 comment
-
|
I just saw this, apologies. This package uses DatabaseConnector to connect to a database, and SqlRender to execute SQL against that database; both do support Spark. I'm not extremely familiar with the Spark universe so I'd recommend checking the linked docs to see if these tools will work with your specific setup. |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
katy-sadowski
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I just saw this, apologies. This package uses DatabaseConnector to connect to a database, and SqlRender to execute SQL against that database; both do support Spark. I'm not extremely familiar with the Spark universe so I'd recommend checking the linked docs to see if these tools will work with your specific setup.