Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #499

Merged
merged 30 commits into from
Oct 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
ae0a1ac
update remote DB schema version
katy-sadowski Aug 18, 2023
ffa161c
remove unneeded setup function
katy-sadowski Aug 18, 2023
d035db1
update GH Actions env vars
katy-sadowski Aug 18, 2023
cb090bc
TEMPORARY - test env vars in GH
katy-sadowski Aug 18, 2023
2d2a460
revert temp test change
katy-sadowski Aug 18, 2023
44830e3
Merge pull request #479 from katy-sadowski/katy__cdm_v54
katy-sadowski Aug 18, 2023
aa89be9
Developers Read Me file added
Sep 11, 2023
e853118
add cdm54 schema to weekly, add redshift
katy-sadowski Sep 14, 2023
1061801
Merge pull request #485 from katy-sadowski/katy__more_cdm54
katy-sadowski Sep 14, 2023
cdf773c
add redshift driver DL
katy-sadowski Sep 14, 2023
8f6c61e
Merge pull request #486 from katy-sadowski/katy__redshift_driver
katy-sadowski Sep 14, 2023
52b56fd
added missing params in DQD docs code to run
katy-sadowski Sep 15, 2023
983ea7d
Merge pull request #487 from katy-sadowski/katy__docs_update
katy-sadowski Sep 15, 2023
c22f1ff
add DB support info to README
katy-sadowski Sep 19, 2023
b4162ca
update executeDqChecks description
katy-sadowski Sep 22, 2023
7a2b489
Merge pull request #489 from katy-sadowski/katy__docs_update
katy-sadowski Sep 22, 2023
ab21d33
Developers Read Me file added
Oct 2, 2023
d53f2e2
add instruction for excluding checks
katy-sadowski Oct 4, 2023
f107255
test fix dependency
katy-sadowski Oct 4, 2023
da3539a
rollback dependency change
katy-sadowski Oct 4, 2023
b6448d2
Developers Read Me file added
Oct 4, 2023
bc93170
Developers Read Me file added
Oct 4, 2023
e49eaea
Developers Read Me file added
Oct 6, 2023
abd194d
Merge pull request #488 from niteshbalakrishnan/main
katy-sadowski Oct 6, 2023
b1eb194
Merge pull request #492 from katy-sadowski/katy__exclude_checks
katy-sadowski Oct 11, 2023
ceebb3e
add readr local edition downgrade
katy-sadowski Oct 16, 2023
ca1a941
fix naming conflict, add imports
katy-sadowski Oct 16, 2023
cd5bd33
Merge pull request #495 from katy-sadowski/katy__vroom_fix
katy-sadowski Oct 18, 2023
dec1ea5
release prep
katy-sadowski Oct 19, 2023
7845a00
Merge pull request #498 from katy-sadowski/katy__2.4.1_release
katy-sadowski Oct 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/R_CMD_check_Hades.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,21 +29,25 @@ jobs:
R_REMOTES_NO_ERRORS_FROM_WARNINGS: true
RSPM: ${{ matrix.config.rspm }}
CDM5_ORACLE_CDM_SCHEMA: ${{ secrets.CDM5_ORACLE_CDM_SCHEMA }}
CDM5_ORACLE_CDM54_SCHEMA: ${{ secrets.CDM5_ORACLE_CDM54_SCHEMA }}
CDM5_ORACLE_OHDSI_SCHEMA: ${{ secrets.CDM5_ORACLE_OHDSI_SCHEMA }}
CDM5_ORACLE_PASSWORD: ${{ secrets.CDM5_ORACLE_PASSWORD }}
CDM5_ORACLE_SERVER: ${{ secrets.CDM5_ORACLE_SERVER }}
CDM5_ORACLE_USER: ${{ secrets.CDM5_ORACLE_USER }}
CDM5_POSTGRESQL_CDM_SCHEMA: ${{ secrets.CDM5_POSTGRESQL_CDM_SCHEMA }}
CDM5_POSTGRESQL_CDM54_SCHEMA: ${{ secrets.CDM5_POSTGRESQL_CDM54_SCHEMA }}
CDM5_POSTGRESQL_OHDSI_SCHEMA: ${{ secrets.CDM5_POSTGRESQL_OHDSI_SCHEMA }}
CDM5_POSTGRESQL_PASSWORD: ${{ secrets.CDM5_POSTGRESQL_PASSWORD }}
CDM5_POSTGRESQL_SERVER: ${{ secrets.CDM5_POSTGRESQL_SERVER }}
CDM5_POSTGRESQL_USER: ${{ secrets.CDM5_POSTGRESQL_USER }}
CDM5_SQL_SERVER_CDM_SCHEMA: ${{ secrets.CDM5_SQL_SERVER_CDM_SCHEMA }}
CDM5_SQL_SERVER_CDM54_SCHEMA: ${{ secrets.CDM5_SQL_SERVER_CDM54_SCHEMA }}
CDM5_SQL_SERVER_OHDSI_SCHEMA: ${{ secrets.CDM5_SQL_SERVER_OHDSI_SCHEMA }}
CDM5_SQL_SERVER_PASSWORD: ${{ secrets.CDM5_SQL_SERVER_PASSWORD }}
CDM5_SQL_SERVER_SERVER: ${{ secrets.CDM5_SQL_SERVER_SERVER }}
CDM5_SQL_SERVER_USER: ${{ secrets.CDM5_SQL_SERVER_USER }}
CDM5_REDSHIFT_CDM_SCHEMA: ${{ secrets.CDM5_REDSHIFT_CDM_SCHEMA }}
CDM5_REDSHIFT_CDM54_SCHEMA: ${{ secrets.CDM5_REDSHIFT_CDM54_SCHEMA }}
CDM5_REDSHIFT_OHDSI_SCHEMA: ${{ secrets.CDM5_REDSHIFT_OHDSI_SCHEMA }}
CDM5_REDSHIFT_PASSWORD: ${{ secrets.CDM5_REDSHIFT_PASSWORD }}
CDM5_REDSHIFT_SERVER: ${{ secrets.CDM5_REDSHIFT_SERVER }}
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/R_CMD_check_main_weekly.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,21 +21,25 @@ jobs:
R_REMOTES_NO_ERRORS_FROM_WARNINGS: true
RSPM: ${{ matrix.config.rspm }}
CDM5_ORACLE_CDM_SCHEMA: ${{ secrets.CDM5_ORACLE_CDM_SCHEMA }}
CDM5_ORACLE_CDM54_SCHEMA: ${{ secrets.CDM5_ORACLE_CDM54_SCHEMA }}
CDM5_ORACLE_OHDSI_SCHEMA: ${{ secrets.CDM5_ORACLE_OHDSI_SCHEMA }}
CDM5_ORACLE_PASSWORD: ${{ secrets.CDM5_ORACLE_PASSWORD }}
CDM5_ORACLE_SERVER: ${{ secrets.CDM5_ORACLE_SERVER }}
CDM5_ORACLE_USER: ${{ secrets.CDM5_ORACLE_USER }}
CDM5_POSTGRESQL_CDM_SCHEMA: ${{ secrets.CDM5_POSTGRESQL_CDM_SCHEMA }}
CDM5_POSTGRESQL_CDM54_SCHEMA: ${{ secrets.CDM5_POSTGRESQL_CDM54_SCHEMA }}
CDM5_POSTGRESQL_OHDSI_SCHEMA: ${{ secrets.CDM5_POSTGRESQL_OHDSI_SCHEMA }}
CDM5_POSTGRESQL_PASSWORD: ${{ secrets.CDM5_POSTGRESQL_PASSWORD }}
CDM5_POSTGRESQL_SERVER: ${{ secrets.CDM5_POSTGRESQL_SERVER }}
CDM5_POSTGRESQL_USER: ${{ secrets.CDM5_POSTGRESQL_USER }}
CDM5_SQL_SERVER_CDM_SCHEMA: ${{ secrets.CDM5_SQL_SERVER_CDM_SCHEMA }}
CDM5_SQL_SERVER_CDM54_SCHEMA: ${{ secrets.CDM5_SQL_SERVER_CDM54_SCHEMA }}
CDM5_SQL_SERVER_OHDSI_SCHEMA: ${{ secrets.CDM5_SQL_SERVER_OHDSI_SCHEMA }}
CDM5_SQL_SERVER_PASSWORD: ${{ secrets.CDM5_SQL_SERVER_PASSWORD }}
CDM5_SQL_SERVER_SERVER: ${{ secrets.CDM5_SQL_SERVER_SERVER }}
CDM5_SQL_SERVER_USER: ${{ secrets.CDM5_SQL_SERVER_USER }}
CDM5_REDSHIFT_CDM_SCHEMA: ${{ secrets.CDM5_REDSHIFT_CDM_SCHEMA }}
CDM5_REDSHIFT_CDM54_SCHEMA: ${{ secrets.CDM5_REDSHIFT_CDM54_SCHEMA }}
CDM5_REDSHIFT_OHDSI_SCHEMA: ${{ secrets.CDM5_REDSHIFT_OHDSI_SCHEMA }}
CDM5_REDSHIFT_PASSWORD: ${{ secrets.CDM5_REDSHIFT_PASSWORD }}
CDM5_REDSHIFT_SERVER: ${{ secrets.CDM5_REDSHIFT_SERVER }}
Expand Down
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Package: DataQualityDashboard
Type: Package
Title: Execute and View Data Quality Checks on OMOP CDM Database
Version: 2.4.0
Date: 2023-07-26
Version: 2.4.1
Date: 2023-10-18
Authors@R: c(
person("Katy", "Sadowski", email = "[email protected]", role = c("aut", "cre")),
person("Clair", "Blacketer", role = c("aut")),
Expand Down
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ importFrom(jsonlite,fromJSON)
importFrom(jsonlite,parse_json)
importFrom(jsonlite,toJSON)
importFrom(magrittr,"%>%")
importFrom(readr,local_edition)
importFrom(readr,read_csv)
importFrom(rlang,.data)
importFrom(stats,na.omit)
Expand Down
8 changes: 8 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
DataQualityDashboard 2.4.1
==========================
This release includes:

- Minor documentation updates
- A patch for an issue in one of DQD's transitive dependencies, `vroom`
- Test suite upgrades to run remote DB tests against OMOP v5.4, and to add Redshift to remote DB tests

DataQualityDashboard 2.4.0
==========================
This release includes:
Expand Down
4 changes: 4 additions & 0 deletions R/convertResultsCase.R
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
#' @importFrom SqlRender snakeCaseToCamelCase camelCaseToSnakeCase
#' @importFrom dplyr rename_with
#' @importFrom tools file_path_sans_ext
#' @importFrom readr local_edition
#'
#' @export

Expand All @@ -47,6 +48,9 @@ convertJsonResultsFileCase <- function(
stop("You must specify an output folder if writing to file.")
}

# temporary patch to work around vroom 1.6.4 bug
readr::local_edition(1)

results <- jsonlite::fromJSON(jsonFilePath)

if ("numViolatedRows" %in% names(results$CheckResults) && targetCase == "camel") {
Expand Down
7 changes: 5 additions & 2 deletions R/executeDqChecks.R
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

#' @title Execute DQ checks
#'
#' @description This function will connect to the database, generate the sql scripts, and run the data quality checks against the database.
#' @description This function will connect to the database, generate the sql scripts, and run the data quality checks against the database. By default, results will be written to a json file as well as a database table.
#'
#' @param connectionDetails A connectionDetails object for connecting to the CDM database
#' @param cdmDatabaseSchema The fully qualified database name of the CDM schema
Expand Down Expand Up @@ -54,7 +54,7 @@
#' @importFrom utils packageVersion write.table
#' @importFrom rlang .data
#' @importFrom tidyselect all_of
#' @importFrom readr read_csv
#' @importFrom readr read_csv local_edition
#' @importFrom dplyr mutate case_when
#'
#' @export
Expand Down Expand Up @@ -119,6 +119,9 @@ executeDqChecks <- function(connectionDetails,
}
}

# temporary patch to work around vroom 1.6.4 bug
readr::local_edition(1)

# capture metadata -----------------------------------------------------------------------
if (!sqlOnly) {
connection <- DatabaseConnector::connect(connectionDetails = connectionDetails)
Expand Down
5 changes: 4 additions & 1 deletion R/listChecks.R
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,13 @@
#' @param fieldCheckThresholdLoc The location of the threshold file for evaluating the field checks. If not specified the default thresholds will be applied.
#' @param conceptCheckThresholdLoc The location of the threshold file for evaluating the concept checks. If not specified the default thresholds will be applied.
#'
#' @importFrom readr read_csv
#' @importFrom readr read_csv local_edition
#'
#' @export
listDqChecks <- function(cdmVersion = "5.3", tableCheckThresholdLoc = "default", fieldCheckThresholdLoc = "default", conceptCheckThresholdLoc = "default") {
# temporary patch to work around vroom 1.6.4 bug
readr::local_edition(1)

dqChecks <- {}
dqChecks$checkDescriptions <-
read_csv(system.file(
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@ System Requirements
===================
Requires R (version 3.2.2 or higher). Requires [DatabaseConnector](https://github.com/OHDSI/DatabaseConnector) (version 2.0.2 or higher).

A variety of database platforms are supported, as documented [here](https://ohdsi.github.io/Hades/supportedPlatforms.html).

Note that while data quality check threshold files are provided for OMOP CDM versions 5.2, 5.3, and 5.4, the package is currently only tested against versions 5.3 and 5.4.

Installation
=============
1. See the instructions [here](https://ohdsi.github.io/Hades/rSetup.html) for configuring your R environment, including RTools and Java.
Expand Down
2 changes: 1 addition & 1 deletion docs/404.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/LICENSE-text.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions docs/articles/AddNewCheck.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions docs/articles/CheckStatusDefinitions.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions docs/articles/CheckTypeDescriptions.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 12 additions & 2 deletions docs/articles/DataQualityDashboard.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions docs/articles/DqdForCohorts.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions docs/articles/SqlOnly.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions docs/articles/Thresholds.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/articles/index.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/authors.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading