Intermittent failures on update task since 3.0.1

[T00fy]

This is directly related to #154

While ./gradlew --no-daemon and ./gradlew --stop do get around this isusue, it still occurs everytime gradle has cached a deamon, and requiring to kill the daemons everytime is cumbersome.

Can this be investigated?

[gdonoso94]

+1

[max-norin]

Hi!

I have the same problem. The problem occurs after few minutes of gradle deamon inactivity.

I tried running the following code in the LiquibasePlugin.applyTasks method.

def commandDefinition = CommandFactory.getInstance().getCommandDefinition('update')
println("liquibase-plugin: commandDefinition.name=${commandDefinition.name}")
println("liquibase-plugin: commandDefinition.shortDescription=${commandDefinition.shortDescription}")
println("liquibase-plugin: commandDefinition.longDescription=${commandDefinition.longDescription}")
println("liquibase-plugin: commandDefinition.arguments.keySet()=${commandDefinition.arguments.keySet()}")

The first run results are as follows.

liquibase-plugin: commandDefinition.name=[update]
liquibase-plugin: commandDefinition.shortDescription=Deploy any changes in the changelog file that have not been deployed
liquibase-plugin: commandDefinition.longDescription=null
liquibase-plugin: commandDefinition.arguments.keySet()=[changeExecListener, changeExecListenerClass, changeExecListenerPropertiesFile, changelogFile, changelogParameters, contextFilter, database, databaseChangelog, defaultCatalogName, defaultSchemaName, driver, driverPropertiesFile, labelFilter, password, showSummary, showSummaryOutput, skipDatabaseStep, updateNullChecksums, url, username]

After 5 minutes, the results are as follows.

liquibase-plugin: commandDefinition.name=[update]
liquibase-plugin: commandDefinition.shortDescription=Deploy any changes in the changelog file that have not been deployed
liquibase-plugin: commandDefinition.longDescription=null
liquibase-plugin: commandDefinition.arguments.keySet()=[]

For some reason, commandDefinition.arguments becomes empty after a few minutes of inactivity.

I also run the following code in the LiquibasePlugin.applyTasks method.

def commandDefinition = CommandFactory.getInstance().getCommandDefinition('update')
println("liquibase-plugin: commandDefinition.name=${commandDefinition.name}")
println("liquibase-plugin: commandDefinition.shortDescription=${commandDefinition.shortDescription}")
println("liquibase-plugin: commandDefinition.longDescription=${commandDefinition.longDescription}")
println("liquibase-plugin: commandDefinition.arguments.keySet()=${commandDefinition.arguments.keySet()}")

Scope.exit(Scope.getCurrentScope().scopeId)

commandDefinition = CommandFactory.getInstance().getCommandDefinition('update')
println("liquibase-plugin: commandDefinition.name=${commandDefinition.name}")
println("liquibase-plugin: commandDefinition.shortDescription=${commandDefinition.shortDescription}")
println("liquibase-plugin: commandDefinition.longDescription=${commandDefinition.longDescription}")
println("liquibase-plugin: commandDefinition.arguments.keySet()=${commandDefinition.arguments.keySet()}")

Result:

liquibase-plugin: commandDefinition.name=[update]
liquibase-plugin: commandDefinition.shortDescription=Deploy any changes in the changelog file that have not been deployed
liquibase-plugin: commandDefinition.longDescription=null
liquibase-plugin: commandDefinition.arguments.keySet()=[changeExecListener, changeExecListenerClass, changeExecListenerPropertiesFile, changelogFile, changelogParameters, contextFilter, database, databaseChangelog, defaultCatalogName, defaultSchemaName, driver, driverPropertiesFile, labelFilter, password, showSummary, showSummaryOutput, skipDatabaseStep, updateNullChecksums, url, username]
liquibase-plugin: commandDefinition.name=[update]
liquibase-plugin: commandDefinition.shortDescription=Deploy any changes in the changelog file that have not been deployed
liquibase-plugin: commandDefinition.longDescription=null
liquibase-plugin: commandDefinition.arguments.keySet()=[]

[max-norin]

In a future version of liquibase there will be a CommandFactory.resetCommandDefinitions() method. I think using it will solve the problem that arguments is empty.

https://github.com/liquibase/liquibase/blob/master/liquibase-standard/src/main/java/liquibase/command/CommandFactory.java#L246

    public void resetCommandDefinitions() {
        COMMAND_DEFINITIONS.clear();
    }

[chschu]

I don't really understand what Liquibase does with its Scope. However, it seems a bit strange that CommandFactory. commandArgumentDefinitions is populated by static initializers (e. g. the .build() calls in UpdateToTagCommandStep), while the CommandFactory itself is bound to the Scope.

[jandrade1]

Has anyone found a workaround for this issue, other than not using a Gradle daemon (or stopping it every time) or rolling back to version 2.x of the plugin? I'm having to stop the daemon constantly while doing local builds which is really annoying. Fortunately, we don't use a daemon in our CI/CD pipelines so it's not a problem there.

[marques-work]

@jandrade1 I think the reliable workaround is to use --no-daemon unfortunately. The dominating suspicion is that this is related to threads/shared memory and the usual workaround to these issues is to use a single thread. In the case of gradle, this is accomplished via --no-daemon. Until this is fixed, there's always v2. I know you were looking for a workaround that was outside of these two options, but I'm not aware of any.

FWIW, I'm not sure there was anything wrong with v2, since this plugin is really just an interface to invoking the liquibase CLI. It doesn't stop you from using the latest liquibase version (at least until liquibase changes its CLI interface) so if this is a major nuissance, that might be the way to go until this gets fixed.

[stevesaliman]

The problem with v2 is that it doesn't work with newer versions of Liquibase. To use anything newer than (I think) 4.16, we need to update to version 3 of the plugin, which fixes the way the argument list gets built for the CLI in a Liquibase 4.24+ compatible way.

I'm not sure why multiple threads would cause different behavior though, but my hunch is that it is a classpath kind of thing. I'm not too familiar with how the daemon works though.

[jandrade1]

@stevesaliman thanks for the clarification. We're running Liquibase 4.28 so it sounds like we must use v3.x of the plugin. I really wish I was more familiar with Gradle plugin development to contribute with a PR here, but I'm not. I imagine most people are running Gradle with a daemon and would be running into this issue consistently.

[marques-work]

@stevesaliman thanks for the clarification. We're running Liquibase 4.28 so it sounds like we must use v3.x of the plugin. I really wish I was more familiar with Gradle plugin development to contribute with a PR here, but I'm not. I imagine most people are running Gradle with a daemon and would be running into this issue consistently.

@jandrade1 FWIW I was using liquibase 4.2X with the v2 plugin just fine before v3 plugin was released. Then again, the only thing I ever ran was the update task and nothing else, so I don't know if other things were broken.

Basically my point is if your use case is very simple (like my team's), v2 may still be an option in the short term, but YMMV.

You could also just write gradle tasks to shell out to the liquibase CLI directly if that's feasible for you as a temporary workaround.

[jandrade1]

@marques-work really appreciate your help here. I'm trying v2.2.2 of the plugin, with a Gradle daemon, and so far so good. I didn't have to make any changes besides just the plugin version number in my build scripts. I also have a simple use case like you where I just invoke the update task. I'll try this a bit more and if it continues to work I'll just stick with v2 until the issue with v3 is fixed.

We're recent adopters of Liquibase at my company, and v3 was the first version of the plugin we had tried, so I had not even tested v2 until now. I'm in charge of creating application templates for my company so I need to make sure the build scripts are solid, and it looks like v2 is the answer for now.