Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

October build run status #689

Open
4 tasks
petefoth opened this issue Sep 30, 2024 · 12 comments
Open
4 tasks

October build run status #689

petefoth opened this issue Sep 30, 2024 · 12 comments

Comments

@petefoth
Copy link
Contributor

petefoth commented Sep 30, 2024

  • started:
    • 21.0 devices missed last month
      • started:
      • unexpected failures: renoir,rhode,river,rtwo,sabrina,sagit,sailfish,salami,sargo,shark,shiba,sif
      • completed:
    • 20.0 branch
      • started:
      • unexpected failures: vayu
      • completed:
    • Remaining 21.0 devices
      • started:
      • unexpected failures:
      • completed:
    • newly promoted devices: NB1
  • completed:

Builds for the following devices did not get made in September, so they go to the top of the queue this month:

  1. Didn't get built in time: radxa0,radxa0_tab,raven,redfin,renoir,rhode,river,rtwo,sabrina,sagit,sailfish,salami,sargo,shark,shiba,sif
  2. New and promoted devices: cmi,onclite
  3. Failed builds: billie,haydn,pioneer,voyager,xz2c

1 & 2 will be first in line this month.. 3 will be done first in the list of remaining 21.0 builds. So the full target list for October will be

  • Missed & failed 21.0 builds: radxa0_tab,raven,redfin,renoir,rhode,river,rtwo,sabrina,sagit,sailfish,salami,sargo,shark,shiba,sif
  • 20.0 builds: for all 20.0 devices that are in the build target list at 202410-00:01
  • Remaining 21.0 builds: cmi,onclite,billie,haydn,pioneer,voyager,xz2c,a52q,a52sxq,a71,a72q,a73xq,akari,akatsuki,akita,alioth,aura,aurora,axolotl,barbet,beckham,berlin,berlna,beryllium,bluejay,blueline,bonito,borneo,bramble,beyond0lte,beyond1lte,beyond2lte,beyondx,caprip,cebu,channel,cheeseburger,cheetah,cheryl,chiron,coral,crosshatch,d1,d1x,d2s,d2x,davinci,deadpool,devon,dipper,discovery,dopinder,dre,dubai,dumpling,earth,enchilada,equuleus,evert,f62,fajita,felix,flame,foster,foster_tab,FP3,FP4,FP5,g710n,g710ulm,gemini,grus,gta4l,gta4lwifi,gta4xl,gta4xlwifi,gtowifi,gts4lv,gts4lvwifi,guacamole,guacamoleb,guam,guamp,hawao,heart,hotdog,hotdogb,husky,ingot,instantnoodle,instantnoodlep,joan,judyln,judyp,judypn,kane,kebab,kiev,kirin,lake,lemonade,lemonadep,lemonades,lisa,lmi,luigi,lynx,m5,m5_tab,m52xq,marlin,mars,martini,mata,mermaid,messi,Mi439,Mi8917,Mi8937,miatoll,munch,nairo,nash,natrium,nio,nx,nx_tab,ocean,odroidc4,odroidc4_tab,oriole,oscar,panther,payton,pdx203,pdx206,pdx214,pdx215,pdx234,perseus,PL2,quill,quill_pad,socrates,sunfish,surya,sweet,taimen,tangorpro,TP1803,troika,umi,ursa,vela,wade,walleye,xpeng,Z01R,zippo

Originally posted by @petefoth in #683 (comment)

@petefoth
Copy link
Contributor Author

Mon 30 Sep 24

  • Add quill` to the 'Remaining 21.0' builds as it just failed in September build run

@petefoth
Copy link
Contributor Author

petefoth commented Oct 1, 2024

Tue 01 Oct 24

  • builds up to redfin OK, then started failing with missing namespace error - killed
  • cleaned up and started with cmi,onclite,billie,haydn,pioneer,voyager,xz2c, then the 20.0 bulids

@videoman614
Copy link

renoir,rhode,river,rtwo,sabrina,sagit,sailfish,salami,sargo,shark,shiba,sif builds fine for official, why is it failing for lineageos for microg?

@petefoth
Copy link
Contributor Author

petefoth commented Oct 1, 2024

renoir,rhode,river,rtwo,sabrina,sagit,sailfish,salami,sargo,shark,shiba,sif builds fine for official, why is it failing for lineageos for microg?

Probably the same reason as the September 21.0 builds failed.I fixed that problem for most devices, but it seems the fix didn;t work for these devices. I will look into it further when I get some free time

@petefoth
Copy link
Contributor Author

petefoth commented Oct 3, 2024

Thu 03 Oct 24

  • builds died during build of apollon >> [Wed Oct 2 21:08:07 UTC 2024]
  • restart using build_list.sh: apollon,bardock,bardockpro,betalm,crownlte,G,gauguin,lancelot,liber,merlinx,mh2lm,nx659j,sake,star2lte,starlte,tissot,vayu,violet,X00TD,xmsirius,z2_plus,zangya,zangyapro

@petefoth
Copy link
Contributor Author

petefoth commented Oct 4, 2024

Fri 04 Oct 24

  • liber buld failed.
[ 62% 541/864] including kernel/motorola/sm6150/drivers/staging/greybus/tools/Android.mk ...
FAILED:
In file included from kernel/google/gs201/private/google-modules/wlan/wcn6740/wlan/qcacld-3.0/Android.mk:29:
In file included from kernel/motorola/sm6150/drivers/staging/greybus/tools/Android.mk:9:
In file included from build/make/core/executable.mk:53:
In file included from build/make/core/executable_internal.mk:29:
In file included from build/make/core/dynamic_binary.mk:34:
In file included from build/make/core/binary.mk:9:
build/make/core/base_rules.mk:338: error: kernel/motorola/sm6150/drivers/staging/greybus/tools: MODULE`
  • Alll subsequent builds failed merlinx,mh2lm,nx659j,sake,star2lte,starlte,tissot,vayu,violet,X00TD,xmsirius,z2_plus,zangya,zangyapro
  • last successful build was lancelot
  • It looks like the same problem as with the 21.0 builds, but with kernel rather than vendor files.

@petefoth
Copy link
Contributor Author

petefoth commented Oct 5, 2024

Sat 05 Oct 24

  • vayu failed - retry later
FAILED: out/target/product/vayu/obj/vendor/qcom/opensource/audio-kernel/include/linux/avtimer.h
/bin/bash -c "\$(cd out/target/product/vayu/obj/KERNEL_OBJ; ../../../../../../../kernel/msm-4.14/scripts/headers_install.sh ../../../../../../../out/target/product/vayu/obj/vendor/qcom/opensource/audio-kernel/include/linux/ ../../../../../../../kernel/motorola/sm6150/techpack/audio/include/uapi/linux/ avtimer.h)"
/bin/bash: line 1: cd: out/target/product/vayu/obj/KERNEL_OBJ: No such file or directory
/bin/bash: line 1: ../../../../../../../kernel/msm-4.14/scripts/headers_install.sh: No such file or directory

@petefoth
Copy link
Contributor Author

petefoth commented Oct 6, 2024

Sun 06 Oct 24

  • 20.0 builds completed
  • 21.0 Started
  • a71 failed - retry later
Error message
[ 99% 186690/186866] build out/target/product/a71/obj/PACKAGING/check_vintf_all_intermediates/check_vintf_compatible.log (priority: 4)
FAILED:
...

List 'out/target/product/a71/product/etc/vintf/': NAME_NOT_FOUND
The following HALs in device manifest are not declared in FCM <= level 5:
  android.hardware.biometrics.fingerprint.IFingerprint/default (@1)
  android.hardware.bluetooth.audio.IBluetoothAudioProviderFactory/default (@4)
  android.hardware.camera.provider.ICameraProvider/internal/0 (@1)
  android.hardware.cas.IMediaCasService/default (@1)
  android.hardware.drm.IDrmFactory/clearkey (@1)
  android.hardware.health.IHealth/default (@1)
  android.hardware.sensors.ISensors/default (@1)
  android.hardware.usb.IUsb/default (@1)
  [email protected]::IUsb/default
  android.hardware.wifi.IWifi/default (@2)
  android.hardware.wifi.hostapd.IHostapd/default (@2)
  android.hardware.wifi.supplicant.ISupplicant/default (@3)
  vendor.samsung.hardware.keymint.ISehKeyMintExtension/default (@1)
  vendor.samsung.hardware.keymint.ISehKeyMintFactory/default (@1)
ERROR: files are incompatible: The following instances are in the device manifest but not specified in framework compatibility matrix:
    vendor.samsung.hardware.keymint.ISehKeyMintExtension/default (@1)
    vendor.samsung.hardware.keymint.ISehKeyMintFactory/default (@1)
Suggested fix:
1. Update deprecated HALs to the latest version.
2. Check for any typos in device manifest or framework compatibility matrices with FCM version >= 5.
3. For new platform HALs, add them to any framework compatibility matrix with FCM version >= 5 where applicable.
4. For device-specific HALs, add to DEVICE_FRAMEWORK_COMPATIBILITY_MATRIX_FILE or DEVICE_PRODUCT_COMPATIBILITY_MATRIX_FILE.: Success
INCOMPATIBLE

@petefoth
Copy link
Contributor Author

Fri 11 Oct 24

  • coral and all subsequent builds. It's because vendor is not being cleaned up
``` [100% 1167/1167] analyzing Android.bp files and generating ninja file at out/soong/build.lineage_coral.ninja FAILED: out/soong/build.lineage_coral.ninja cd "$(dirname "out/host/linux-x86/bin/soong_build")" && BUILDER="$PWD/$(basename "out/host/linux-x86/bin/soong_build")" && cd / && env -i "$BUILDER" --top "$TOP" --soong_out "out/soong" --out "out" --soong_variables out/soong/soong.lineage_coral.variables -o out/soong/build.lineage_coral.ninja --globListDir lineage_coral --globFile out/soong/globs-lineage_coral.ninja -l out/.module_paths/Android.bp.list --available_env out/soong/soong.environment.available --used_env out/soong/soong.environment.used.lineage_coral.build Android.bp error: vendor/sony/sm8250-common/Andqroid.bp:14767:1: "[email protected]" depends on undefined module "[email protected]". Or did you mean ["[email protected]" "[email protected]" "[email protected]" "[email protected]_hal" "[email protected]_hal" "[email protected]_hal" "[email protected]_hal" "[email protected]_hal" "vendor.qti.hardware.qspa-V1-java" "vendor.qti.hardware.qspa-V1-rust" "vendor.qti.hardware.qspa-V2-java" "vendor.qti.hardware.qspa-V2-rust" "vendor.qti.hardware.radio.am.xml" "[email protected]" "vendor.qti.hardware.systemhelper" "[email protected]" "[email protected]_hal" "[email protected]"]? ```
  • put in some debug info
  • restart builds from coral

@petefoth
Copy link
Contributor Author

petefoth commented Oct 15, 2024

Tue 15 Oct 24

  • Several builds, starting with grus have failed. All the failures so far have been missing namespace errors
    and the missing namespaces have all been in vendor/xiaomi repos.
  • the problem is that changes were made in the last couple of hours in TheMuppest git for sweet, davinci, & sm6150-common repos, and matching groups fields added for those repos in the manifest rpo. But we are using a proprietary.xml that doesn't have those changes, because it is generated once, outside the build loop. Should be enough to just restart the build (from grus). Longer term we could generate the proprietary.xml inside the main loop

Wed 16 Oct 24

  • restarting the build seems to have fixed it

Thu 17 Oct 24

  • hawao failed, same problem: upstream changes to manifest,. Kill & restart from hawao
  • moved the code that generates proprietary.xml inside the main loop, so it is done for each device, minimising (with luck) the chances of this happening again.

@gwstorm
Copy link

gwstorm commented Oct 21, 2024

Is this the same issue here?

docker-lineage-cicd-1  | Set cache size limit to 50.0 GB
docker-lineage-cicd-1  | >> [Mon Oct 21 04:57:26 UTC 2024] Branch:  lineage-21.0
docker-lineage-cicd-1  | >> [Mon Oct 21 04:57:26 UTC 2024] Devices: lemonadep
docker-lineage-cicd-1  | >> [Mon Oct 21 04:57:26 UTC 2024] Copying '/srv/local_manifests/*.xml' to '.repo/local_manifests/'
docker-lineage-cicd-1  | >> [Mon Oct 21 04:57:26 UTC 2024] (Re)initializing branch repository
docker-lineage-cicd-1  | sed: can't read vendor/lineage/config/common.mk: No such file or directory
docker-lineage-cicd-1 exited with code 2

@petefoth
Copy link
Contributor Author

docker-lineage-cicd-1 | >> [Mon Oct 21 04:57:26 UTC 2024] (Re)initializing branch repository
docker-lineage-cicd-1 | sed: can't read vendor/lineage/config/common.mk: No such file or directory
docker-lineage-cicd-1 exited with code 2

That's not the error that we were seeing in the build failures. And it's not one I recall seeing in any of our earlier build failures. So it looks like you are having a different problem. I don't want to clutter this issue, so if you are seeing this problem repeatedly and consistently it may be worth creating a new issue where it ca be dealt with.

From that error message,if file vendor/lineage/config/common.mk does not exist in your source tree, that tree is in a very strange state. Try removing ``vendor/lineage/orcd vendor/lineage && git reset --hard` then re-running your build.

If that doesn't work, please open a new issue, with as much detail as possible. Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants