Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Try adding windows build #496

Closed
wants to merge 2 commits into from
Closed

[WIP] Try adding windows build #496

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8abc31f
Try adding windows build
slide Dec 4, 2024
e9b544c
Add prep script to download WAR
slide Dec 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 71 additions & 52 deletions Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,63 +9,82 @@ if (env.BRANCH_IS_PRIMARY) {

properties(jobProperties)

podTemplate(
inheritFrom: 'jnlp-maven-17',
workingDir: '/home/jenkins/agent',
containers: [
containerTemplate(name: 'jnlp', image: 'jenkinsciinfra/packaging:latest')
],
envVars: [
envVar(key: 'HOME', value: '/home/jenkins/agent/workspace'),
],
) {
nodeWithTimeout(POD_LABEL) {
withEnv([
"BUILDENV=${WORKSPACE}/env/test.mk",
"BRANDING_DIR=${WORKSPACE}/branding",
"BRAND=${WORKSPACE}/branding/jenkins.mk",
"GPG_FILE=${WORKSPACE}/credentials/sandbox.gpg",
"GPG_KEYNAME=Bogus Test",
"GPG_PASSPHRASE=s3cr3t",
"GPG_PASSPHRASE_FILE=${WORKSPACE}/credentials/test.gpg.password.txt",
"HOME=/home/jenkins/agent/workspace",
"WAR=${WORKSPACE}/jenkins.war",
"MSI=${WORKSPACE}/jenkins.msi",
"RELEASELINE=-experimental",
]) {
stage('Preparation') {
checkout scm
sh './prep.sh'
stage('Build Packages') {
parallel Linux: {
podTemplate(
inheritFrom: 'jnlp-maven-17',
workingDir: '/home/jenkins/agent',
containers: [
containerTemplate(name: 'jnlp', image: 'jenkinsciinfra/packaging:latest')
],
envVars: [
envVar(key: 'HOME', value: '/home/jenkins/agent/workspace'),
],
) {
nodeWithTimeout(POD_LABEL) {
withEnv([
"BUILDENV=${WORKSPACE}/env/test.mk",
"BRANDING_DIR=${WORKSPACE}/branding",
"BRAND=${WORKSPACE}/branding/jenkins.mk",
"GPG_FILE=${WORKSPACE}/credentials/sandbox.gpg",
"GPG_KEYNAME=Bogus Test",
"GPG_PASSPHRASE=s3cr3t",
"GPG_PASSPHRASE_FILE=${WORKSPACE}/credentials/test.gpg.password.txt",
"HOME=/home/jenkins/agent/workspace",
"WAR=${WORKSPACE}/jenkins.war",
"MSI=${WORKSPACE}/jenkins.msi",
"RELEASELINE=-experimental",
]) {
stage('Preparation') {
checkout scm
sh './prep.sh'
}

stage('Build') {
sh 'make package && python3 -m pytest bin --junitxml target/junit.xml'
junit 'target/junit.xml'
def results = '*.war, target/debian/*.deb, target/rpm/*.rpm, target/suse/*.rpm'
stash includes: results, name: 'results'
archiveArtifacts results
}
}
}
}

stage('Build') {
sh 'make package && python3 -m pytest bin --junitxml target/junit.xml'
junit 'target/junit.xml'
def results = '*.war, target/debian/*.deb, target/rpm/*.rpm, target/suse/*.rpm'
stash includes: results, name: 'results'
archiveArtifacts results
nodeWithTimeout('docker') {
stage('Test') {
checkout scm
unstash 'results'
infra.withDockerCredentials {
ansiColor('xterm') {
sh '''
cat /proc/cpuinfo
cat /proc/meminfo
python3 -m venv venv
. venv/bin/activate
pip install -U pip wheel
pip install -r requirements.txt
ANSIBLE_FORCE_COLOR=true molecule test
ANSIBLE_FORCE_COLOR=true molecule test -s servlet
deactivate
'''.stripIndent()
}
}
}
}
}
}
}, Windows: {
nodeWithTimeout('docker-windows') {
stage('Preparation') {
checkout scm
powershell '.\\prep.ps1'
}

nodeWithTimeout('docker') {
stage('Test') {
checkout scm
unstash 'results'
infra.withDockerCredentials {
ansiColor('xterm') {
sh '''
cat /proc/cpuinfo
cat /proc/meminfo
python3 -m venv venv
. venv/bin/activate
pip install -U pip wheel
pip install -r requirements.txt
ANSIBLE_FORCE_COLOR=true molecule test
ANSIBLE_FORCE_COLOR=true molecule test -s servlet
deactivate
'''.stripIndent()
stage('Build') {
withEnv([
"WAR=${WORKSPACE}/jenkins.war"
]) {
powershell ".\\make.ps1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So while this works its not the same image as run in the windows build currently.

https://github.com/jenkins-infra/release/blob/master/Jenkinsfile.d/core/package#L259

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to use the same pod template from the release repo? Is the pod template for the Linux build the same as used in release?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It inherits something similar I think but not quite the same, but cc @dduportal

Container image is passed here:

packaging/Jenkinsfile

Line 16 in 5100d16

containerTemplate(name: 'jnlp', image: 'jenkinsciinfra/packaging:latest')

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good grief @timja . However, given how slow the Windows pod are to spin up (the dotnet SDK container takes 12 to 17 min to pull and AKS does not allow nodes to come with container images preloaded AFAIK), I would not mind using a Windows (2022?) VM with the same template as in ci.jio.

@slide does the current docker-windows work? if yes, we can totally set up release to use the same.

The only important part is the timing: we need to sync the PR change here with the release agent definition change to avoid breaking next release

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker-windows is using the packer image setup, is that correct? Also, I am not sure if it works, I don't have access to test my Jenkinsfile changes on CI as I am not part of the packaging repo team.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(failed: https://ci.jenkins.io/job/Packaging/job/packaging/job/PR-496/2/pipeline-console/?selected-node=51)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I just request access via helpdesk?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I just request access via helpdesk?

Ideally yes: it is easier to audit afterwards. We tend to also ask in jenkins dev mailing list to have +1 (to gather community feedbacks) as an unspoken but usual rule, to get a quorum.

That should be a quick one

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sent an email to the developers list and got some +1's: https://groups.google.com/g/jenkinsci-dev/c/U5WoCd77wuo

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added permission, I think you will need to create a new PR for the Jenkinsfile change to work

}
}
}
}
Expand Down
9 changes: 9 additions & 0 deletions prep.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@

Set-Location $PSScriptRoot

if(-not (Test-Path -Path $env:WAR)) {
Invoke-WebRequest -Uri "https://github.com/jenkins-infra/jenkins-version/releases/download/latest/jenkins-version-windows-amd64.zip" -OutFile (Join-Path $PSScriptRoot 'jenkins-version-windows-amd64.zip') -UseBasicParsing
[IO.Compression.ZipFile]::ExtractToDirectory('jenkins-version-windows-amd64.zip', $PSScriptRoot)
& .\jv.exe download
Remove-Item -Force -Path 'jv.exe','README.md','LICENSE','jenkins-version-windows-amd64.zip'
}