Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update the lib to 6.5.0 and add gradle-check-flaky-test-issue-creation.jenkinsfile. #4777

Merged
merged 1 commit into from
Jun 13, 2024
Merged

Update the lib to 6.5.0 and add gradle-check-flaky-test-issue-creation.jenkinsfile. #4777

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions jenkins/gradle/gradle-check-flaky-test-issue-creation.jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

lib = library(identifier: '[email protected]', retriever: modernSCM([
$class: 'GitSCMSource',
remote: 'https://github.com/opensearch-build-libraries/opensearch-build-libraries.git',
]))


pipeline {
agent { label 'Jenkins-Agent-Ubuntu2004-X64-M58xlarge-Single-Host' }
options {
timeout(time: 1, unit: 'HOURS')
buildDiscarder(logRotator(daysToKeepStr: '180'))
}
triggers {
parameterizedCron '''
H */8 * * *
'''
}
stages {
stage('Detect Gradle Check Flaky Tests') {
steps {
script {
gradleCheckFlakyTestChecker()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to add tests for this jenkinsFile specifically?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this limitation currently its not possible opensearch-project/opensearch-build-libraries#436 (comment).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So even the regression txt file wont be generated?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wont Sayali initially I have tried it, but the line it reaches gradleCheckFlakyTestChecker it starts failing for this line here https://github.com/opensearch-project/opensearch-build-libraries/blob/main/src/gradlecheck/FetchPostMergeFailedTestClass.groovy#L77 as it's not able to find the class OpenSearchMetricsQuery.

Copy link
Member Author

@prudhvigodithi prudhvigodithi Jun 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried the following way but fails at OpenSearchMetricsQuery, its not able to mock external classes part of the used library.

/*
 * Copyright OpenSearch Contributors
 * SPDX-License-Identifier: Apache-2.0
 */

import jenkins.tests.BuildPipelineTest
import org.junit.Before
import org.junit.Test
import org.yaml.snakeyaml.Yaml
import static com.lesfurets.jenkins.unit.global.lib.LibraryConfiguration.library
import static com.lesfurets.jenkins.unit.global.lib.GitSource.gitSource
import static com.lesfurets.jenkins.unit.MethodCall.callArgsToString
import static org.hamcrest.CoreMatchers.hasItem
import static org.hamcrest.MatcherAssert.assertThat
import static org.junit.jupiter.api.Assertions.assertThrows

class TestGradleCheckFlakyTestIssueCreation extends BuildPipelineTest { 
    def mockedResponseHere = '''
    {
        "aggregations": {
            "test_class_keyword_agg": {
                "buckets": [
                    {
                        "key": "TestClass1",
                        "doc_count": 10
                    },
                    {
                        "key": "TestClass2",
                        "doc_count": 5
                    }
                ]
            }
        }
    }
    '''

    @Override
    @Before
    void setUp() {
        helper.registerSharedLibrary(
            library().name('jenkins')
                .defaultVersion('main')
                .allowOverride(true)
                .implicit(true)
                .targetPath('vars')
                .retriever(gitSource('https://github.com/prudhvigodithi/opensearch-build-libraries.git'))
                .build()
        )

        super.setUp()
        addParam('METRICS_HOST_ACCOUNT', '1.0.0')
        helper.registerAllowedMethod('withAWS', [Map, Closure], null)

        def openSearchMetricsQueryMock = new OpenSearchMetricsQuery('', '', '', '', null)
        openSearchMetricsQueryMock.metaClass.fetchMetrics = { String query ->
            return new groovy.json.JsonSlurper().parseText(mockedResponseHere)
        }
    }

    @Test
    void jenkinsFileExecution() {
        super.testPipeline('jenkins/gradle/gradle-check-flaky-test-issue-creation.jenkinsfile',
                'tests/jenkins/jenkinsjob-regression-files/promotion/gradle-check-flaky-test-issue-creation.jenkinsfile')
    }
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird! @dblock Maybe you have some idea here?

}
}
}
}
post() {
always {
script {
postCleanup()
}
}
}
}
2 changes: 1 addition & 1 deletion jenkins/gradle/gradle-check.jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
* compatible open source license.
*/

lib = library(identifier: 'jenkins@6.4.8', retriever: modernSCM([
lib = library(identifier: 'jenkins@6.5.0', retriever: modernSCM([
$class: 'GitSCMSource',
remote: 'https://github.com/opensearch-project/opensearch-build-libraries.git',
]))
Expand Down
2 changes: 1 addition & 1 deletion jenkins/opensearch-dashboards/distribution-build.jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
* compatible open source license.
*/

lib = library(identifier: 'jenkins@6.4.1', retriever: modernSCM([
lib = library(identifier: 'jenkins@6.5.0', retriever: modernSCM([
$class: 'GitSCMSource',
remote: 'https://github.com/opensearch-project/opensearch-build-libraries.git',
]))
Expand Down
2 changes: 1 addition & 1 deletion jenkins/opensearch-dashboards/integ-test.jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
* compatible open source license.
*/

lib = library(identifier: 'jenkins@6.3.2', retriever: modernSCM([
lib = library(identifier: 'jenkins@6.5.0', retriever: modernSCM([

$class: 'GitSCMSource',
remote: 'https://github.com/opensearch-project/opensearch-build-libraries.git',
Expand Down
2 changes: 1 addition & 1 deletion jenkins/opensearch/distribution-build.jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
* compatible open source license.
*/

lib = library(identifier: 'jenkins@6.4.1', retriever: modernSCM([
lib = library(identifier: 'jenkins@6.5.0', retriever: modernSCM([
$class: 'GitSCMSource',
remote: 'https://github.com/opensearch-project/opensearch-build-libraries.git',
]))
Expand Down
2 changes: 1 addition & 1 deletion jenkins/opensearch/integ-test.jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
* compatible open source license.
*/

lib = library(identifier: 'jenkins@6.3.2', retriever: modernSCM([
lib = library(identifier: 'jenkins@6.5.0', retriever: modernSCM([

$class: 'GitSCMSource',
remote: 'https://github.com/opensearch-project/opensearch-build-libraries.git',
Expand Down
2 changes: 1 addition & 1 deletion tests/jenkins/TestOpenSearchDashboardsIntegTest.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ class TestOpenSearchDashboardsIntegTest extends BuildPipelineTest {

helper.registerSharedLibrary(
library().name('jenkins')
.defaultVersion('6.3.2')
.defaultVersion('6.5.0')
.allowOverride(true)
.implicit(true)
.targetPath('vars')
Expand Down
8 changes: 4 additions & 4 deletions tests/jenkins/TestOpenSearchIntegTest.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ class TestOpenSearchIntegTest extends BuildPipelineTest {

helper.registerSharedLibrary(
library().name('jenkins')
.defaultVersion('6.3.2')
.defaultVersion('6.5.0')
.allowOverride(true)
.implicit(true)
.targetPath('vars')
Expand Down Expand Up @@ -125,17 +125,17 @@ class TestOpenSearchIntegTest extends BuildPipelineTest {
}
helper.addShMock("""env PATH=\$PATH JAVA_HOME=/opt/java/openjdk-17 ./test.sh integ-test manifests/tests/jenkins/data/opensearch-3.0.0-test.yml --component k-NN --test-run-id 234 --paths opensearch=/tmp/workspace/tar --base-path DUMMY_PUBLIC_ARTIFACT_URL/dummy_job/3.0.0/9010/linux/x64/tar """) { script ->
return [stdout: "Error running integtest for component k-NN, creating Github issue", exitValue: 1]}
helper.addShMock("""gh issue list --repo https://github.com/opensearch-project/k-NN.git -S "[AUTOCUT] Integration Test failed for k-NN: 3.0.0 in:title" --label autocut,v3.0.0 --json number --jq '.[0].number'""") { script ->
helper.addShMock("""gh issue list --repo https://github.com/opensearch-project/k-NN.git -S "[AUTOCUT] Integration Test failed for k-NN: 3.0.0 in:title" --json number --jq '.[0].number'""") { script ->
return [stdout: "", exitValue: 0]
}
helper.addShMock("""gh issue list --repo https://github.com/opensearch-project/k-NN.git -S "[AUTOCUT] Integration Test failed for k-NN: 3.0.0 in:title is:closed closed:>=2023-10-24" --label autocut,v3.0.0 --json number --jq '.[0].number'""") { script ->
helper.addShMock("""gh issue list --repo https://github.com/opensearch-project/k-NN.git -S "[AUTOCUT] Integration Test failed for k-NN: 3.0.0 in:title is:closed closed:>=2023-10-24" --json number --jq '.[0].number'""") { script ->
return [stdout: "", exitValue: 0]
}
assertThrows(Exception) {
runScript('jenkins/opensearch/integ-test.jenkinsfile')
}
assertJobStatusFailure()
assertThat(getCommandExecutions('sh', 'script'), hasItem('{script=gh issue create --title \"[AUTOCUT] Integration Test failed for k-NN: 3.0.0\" --body \"The integration test failed at distribution level for component k-NN<br>Version: 3.0.0<br>Distribution: tar<br>Architecture: x64<br>Platform: linux<br><br>Please check the logs: https://some/url/redirect<br><br> * Test-report manifest:*<br> - https://ci.opensearch.org/ci/dbc/dummy_job/3.0.0/9010/linux/x64/tar/test-results/234/integ-test/test-report.yml <br><br> _Note: Steps to reproduce, additional logs and other files can be found within the above test-report manifest. <br>Instructions of this test-report manifest can be found [here](https://github.com/opensearch-project/opensearch-build/tree/main/src/report_workflow#guide-on-test-report-manifest-from-ci)._\" --label autocut,v3.0.0 --label \"untriaged\" --repo https://github.com/opensearch-project/k-NN.git, returnStdout=true}'))
assertThat(getCommandExecutions('sh', 'script'), hasItem("""{script=gh issue create --title \"[AUTOCUT] Integration Test failed for k-NN: 3.0.0\" --body \"The integration test failed at distribution level for component k-NN<br>Version: 3.0.0<br>Distribution: tar<br>Architecture: x64<br>Platform: linux<br><br>Please check the logs: https://some/url/redirect<br><br> * Test-report manifest:*<br> - https://ci.opensearch.org/ci/dbc/dummy_job/3.0.0/9010/linux/x64/tar/test-results/234/integ-test/test-report.yml <br><br> _Note: Steps to reproduce, additional logs and other files can be found within the above test-report manifest. <br>Instructions of this test-report manifest can be found [here](https://github.com/opensearch-project/opensearch-build/tree/main/src/report_workflow#guide-on-test-report-manifest-from-ci)._\" --label \"autocut,v3.0.0\" --label \"untriaged\" --repo https://github.com/opensearch-project/k-NN.git, returnStdout=true}"""))
}

@Test
Expand Down
Loading
Loading