Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot build unsigned release APK #174

Open
7 tasks done
IzzySoft opened this issue Oct 25, 2024 · 23 comments · Fixed by #175
Open
7 tasks done

cannot build unsigned release APK #174

IzzySoft opened this issue Oct 25, 2024 · 23 comments · Fixed by #175
Labels
bug Something isn't working

Comments

@IzzySoft
Copy link

Steps to reproduce

Since you switched to build.gradle.kts, I can no longer build an unsigned release APK as you enforce signing. That means, no more reproducible builds at IzzyOnDroid. Could you please tell me how to create an *unsigned.apk?

Expected behavior

./gradlew assembleRelease produces unsigned APK if no keystore exists

Actual behavior

it creates an app-release.apk (with debug signing?)

version of the program

0.5.3

Android version

n/a

Device

n/a

Other details

same most likely applies to thumbkey, which had it's RB failing today as well.

Acknowledgements

  • I have written a short but informative title.
  • I have updated the app to the latest version.
  • I have searched the existing issues and this is a new one, NOT a duplicate or related to another open issue.
  • This is not a question or a discussion, in which case I should have gone to lemmy.ml/c/rankmyfavs
  • This is a single bug report, in case of multiple bugs I will open a separate issue for each one (they can always link to each other if related)
  • I have admitted that I am a clown by having checked this box, as I have not read these acknowledgements. 🤡
  • I have filled out all of the requested information in this form.
@IzzySoft IzzySoft added the bug Something isn't working label Oct 25, 2024
@IzzySoft
Copy link
Author

OK, found the trick:

sed -r 's/signingConfigs.getByName\("(release|debug)"\)/signingConfigs.findByName("none")/' -i app/build.gradle.kts

It's RB again. Apologies for the noise. Will try the same trick on thumbkey now as well…

@dessalines
Copy link
Owner

No probs. I'm confused as to why that's necessary for these two, but not jerboa, because jerboa has the exact same signing configuration: https://github.com/LemmyNet/jerboa/blob/main/app/build.gradle.kts#L70

But I see that my blocks are different from what they currently recommend, so I'll try those out.

@dessalines
Copy link
Owner

I think I found the issue. I'll fix it in a PR rn.

@dessalines dessalines reopened this Oct 25, 2024
dessalines added a commit that referenced this issue Oct 25, 2024
@IzzySoft
Copy link
Author

OK, worked for thumbkey as well.

Note to self: if the resulting APK is signed by a debug key, it doesn't help to disable just the release signing 🙈

signingConfig = if (project.hasProperty("RELEASE_STORE_FILE")) {

No such file, so it must be the else part 💡

I'm confused as to why that's necessary for these two, but not jerboa

I cannot tell, Jerboa is not in my builder. Hm, wild guess: maybe you're speaking of F-Droid's building? Different tooling. They trust a lot in "magic", removing all signing related stuff automatically. At IzzyOnroid, we don't – so when setting up recipes, we deal with all changes applied to the app's code explicitly. For example, here's the complete block for 0.5.3 of rank-my-favs:

  - tag: 0.5.3
    apks:
      - apk_pattern: app-release\.apk
        apk_url: https://github.com/dessalines/rank-my-favs/releases/download/0.5.3/app-release.apk
        build:
          - sed -r 's/signingConfigs.getByName\("(release|debug)"\)/signingConfigs.findByName("none")/' -i app/build.gradle.kts
          - chmod +x gradlew
          - ./gradlew assembleRelease
          - mv app/build/outputs/apk/release/*unsigned.apk /outputs/unsigned.apk
        build_home_dir: /build
        build_repo_dir: /build/repo
        build_user: build
        provisioning:
          android_home: /opt/sdk
          build_tools:
          cmake:
          cmdline_tools:
            version: '12.0'
            url: https://dl.google.com/android/repository/commandlinetools-linux-11076708_latest.zip
            sha256: 2d2d50857e4eb553af5a6dc3ad507a17adf43d115264b1afc116f95c92e5e258
          extra_packages: []
          image: debian:bookworm-slim
          jdk: openjdk-17-jdk-headless
          ndk:
          platform:
          platform_tools:
          tools:
          verify_gradle_wrapper: true

@IzzySoft
Copy link
Author

I think I found the issue. I'll fix it in a PR rn.

Oh, nice! Yeah, that should do the trick. Whoever wants a debug signed APK, can use assembleDebug 😉

Pointer: if you wonder why I've used findByName, it's because when it fails to find something it simply skips signing, without "erroring out". Learned that only recently. So I can leave above recipe as-is, and it should still work after your change has been applied 😉 Though of course I could drop that with the next release… Thanks a lot!

@dessalines
Copy link
Owner

No probs! Thx for bumping these up also.

I see jerboa in the izzyondroid index, but I'm guessing it has a hybrid configuration then rather than its own in your builder.

@IzzySoft
Copy link
Author

Ah, now. Yes, other than at F-Droid, the app repo and the builders (plural) are separate things. The app repo (which you've linked to) provides the F-Droid compatible repository, with the APKs taken directly from the developers. The builders have their own setup, and build from source. They are not necessarily run by us (though the two you currently can see are, there are also independent builders which we hope to be able to make visible soon; it's a question of "design" to make them clearly distinguishable as "independent builders" – which means no "ratings" from our side but is rather something great to see: others confirm our work, independently, that's great! But folks must be aware of that, so they can put their trust accordingly).

For details and background, see our article Reproducible Builds, special client support and more at IzzyOnDroid. As for visualization (apart from the "RB shields" in the IoD repo browser and the pendant in the mentioned clients hopefully arriving soon, one of the independents is already working on "something nice" (as usual, we don't want to announce things before we know they are real and available; but what I hear from Ben sounds great already).

That said, let me take a look at Jerboa. I wonder why it's not in my builder… Ah, it's in Fays. And yeah, it's currently NOT RB:

com.jerboa: last versions not RB (lots of additional stuff in dex)

Which is very strange; I've just gave it a try, and there seem to be entire classes in my build which are not present in theirs oops, yours… According to the logs of Fay's builder, Jerboa fails since 0.0.68; 0.0.67 was the last release with "reproducible": true.

Example section of the diff from classes.dex, maybe it gives you a clue:

+  Class descriptor  : 'Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;'
+  Access flags      : 0x0011 (PUBLIC FINAL)
+  Superclass        : 'Lkotlin/coroutines/jvm/internal/SuspendLambda;'
+  Interfaces        -
+    #0              : 'Lkotlin/jvm/functions/Function3;'
+  Static fields     -
+  Instance fields   -
+    #0              : (in Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;)
+      name          : '$r8$classId'
+      type          : 'I'
+      access        : 0x1011 (PUBLIC FINAL SYNTHETIC)
+  Direct methods    -
+    #0              : (in Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;)
+      name          : '<init>'
+      type          : '(ILkotlin/coroutines/Continuation;I)V'
+      access        : 0x11001 (PUBLIC SYNTHETIC CONSTRUCTOR)
+      code          -
+      registers     : 4
+      ins           : 4
+      outs          : 3
+      insns size    : 6 16-bit code units
+| androidx.compose.foundation.gestures.DraggableKt$NoOpOnDragStarted$1.<init>:(ILkotlin/coroutines/Continuation;I)V
+|: iput v3, v0, Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;.$r8$classId:I
+|: invoke-direct {v0, v1, v2}, Lkotlin/coroutines/jvm/internal/SuspendLambda;.<init>:(ILkotlin/coroutines/Continuation;)V
+|: return-void
+      catches       : (none)
+      positions     :
+      locals        :
+
+  Virtual methods   -
+    #0              : (in Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;)
+      name          : 'invoke'
+      type          : '(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;'
+      access        : 0x0011 (PUBLIC FINAL)
+      code          -
+      registers     : 5
+      ins           : 4
+      outs          : 4
+      insns size    : 78 16-bit code units
+| androidx.compose.foundation.gestures.DraggableKt$NoOpOnDragStarted$1.invoke:(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
+|: iget v0, v1, Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;.$r8$classId:I
+|: packed-switch v0, 00000046 // +00000044
+|: check-cast v2, Landroidx/compose/foundation/gestures/PressGestureScopeImpl;
+|: check-cast v3, Landroidx/compose/ui/geometry/Offset;
+|: iget-wide v2, v3, Landroidx/compose/ui/geometry/Offset;.packedValue:J
+|: check-cast v4, Lkotlin/coroutines/Continuation;
+|: new-instance v2, Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;
+|: const/4 v3, #int 3 // #3
+|: const/4 v0, #int 2 // #2
+|: invoke-direct {v2, v3, v4, v0}, Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;.<init>:(ILkotlin/coroutines/Continuation;I)V
+|: sget-object v3, Lkotlin/Unit;.INSTANCE:Lkotlin/Unit;
+|: invoke-virtual {v2, v3}, Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;.invokeSuspend:(Ljava/lang/Object;)Ljava/lang/Object;
+|: return-object v3
+|: check-cast v2, Lkotlinx/coroutines/CoroutineScope;
+|: check-cast v3, Ljava/lang/Number;
+|: invoke-virtual {v3}, Ljava/lang/Number;.floatValue:()F
+|: check-cast v4, Lkotlin/coroutines/Continuation;
+|: new-instance v2, Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;
+|: const/4 v3, #int 3 // #3
+|: const/4 v0, #int 1 // #1
+|: invoke-direct {v2, v3, v4, v0}, Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;.<init>:(ILkotlin/coroutines/Continuation;I)V
+|: sget-object v3, Lkotlin/Unit;.INSTANCE:Lkotlin/Unit;
+|: invoke-virtual {v2, v3}, Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;.invokeSuspend:(Ljava/lang/Object;)Ljava/lang/Object;
+|: return-object v3
+|: check-cast v2, Lkotlinx/coroutines/CoroutineScope;
+|: check-cast v3, Landroidx/compose/ui/geometry/Offset;
+|: iget-wide v2, v3, Landroidx/compose/ui/geometry/Offset;.packedValue:J
+|: check-cast v4, Lkotlin/coroutines/Continuation;
+|: new-instance v2, Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;
+|: const/4 v3, #int 3 // #3
+|: const/4 v0, #int 0 // #0
+|: invoke-direct {v2, v3, v4, v0}, Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;.<init>:(ILkotlin/coroutines/Continuation;I)V
+|: sget-object v3, Lkotlin/Unit;.INSTANCE:Lkotlin/Unit;
+|: invoke-virtual {v2, v3}, Landroidx/compose/foundation/gestures/DraggableKt$NoOpOnDragStarted$1;.invokeSuspend:(Ljava/lang/Object;)Ljava/lang/Object;
+|: return-object v3
+|: nop // spacer

No counter-part to that in your classes.dex.

Build was on Debian bookworm with JDK-17, invoked with ./gradlew assembleRelease.

@dessalines
Copy link
Owner

We've had some issues with reproducible builds on jerboa, probably due to android dep updates, but none of us were able to get very far with discovering what causes it. I think the current f-droid builds have been working for the last few releases luckily.

@IzzySoft
Copy link
Author

I think the current f-droid builds have been working for the last few releases luckily.

Unsurprising. It builds fine, it's just not RB – and you did not establish RB at F-Droid.

Wild thought: do you have pinned all dependencies to their specific versions – or could it be some of them are calculated at build time, so our builds (happening later than yours) could pick up newer versions of some dependency?

@dessalines
Copy link
Owner

Looks like they're all pinned, and renovate seems to enforce that. We've tried to discover why it happens occasionally, but never get very far.

@IzzySoft
Copy link
Author

Hm, k – thanks for checking nevertheless! Be welcome to ping me should you think you've found a culprit and want me to verify.

@IzzySoft
Copy link
Author

0.6.3 fails RB (apologies for not opening a separate issue, but there's no matching template). Here's the diff of the APKs:

  -rw-r--r--  0.0 unx      120 b-      118 defN 1981-01-01 01:01:02 288957a8 META-INF/version-control-info.textproto
- -rw-r--r--  0.0 unx     6675 b-     6675 stor 1981-01-01 01:01:02 56b011bb assets/dexopt/baseline.prof
- -rw-r--r--  0.0 unx      775 b-      775 stor 1981-01-01 01:01:02 8702d299 assets/dexopt/baseline.profm
+ -rw-r--r--  0.0 unx     6682 b-     6682 stor 1981-01-01 01:01:02 121a76ee assets/dexopt/baseline.prof
+ -rw-r--r--  0.0 unx      776 b-      776 stor 1981-01-01 01:01:02 6dae9261 assets/dexopt/baseline.profm
  -rw-r--r--  0.0 unx  4532556 b-  2185888 defN 1981-01-01 01:01:02 eaee59cd classes.dex

So it's only the baselines. Looking at the baseline diff:

   profile_key='classes.dex'
-  num_type_ids=2427
-  hot_method_region_size=34352
+  num_type_ids=2428
+  hot_method_region_size=34356
   dex_checksum=0xeaee59cd
   num_method_ids=30607
 dex_data 0
-  num_hot_method_ids=8588
-  num_type_ids=2427
+  num_hot_method_ids=8589
+  num_type_ids=2428
   method_id=0
   num_inline_caches=0
   method_id=1
@@ -3736,6 +3736,8 @@
   num_inline_caches=0
   method_id=3925
   num_inline_caches=0
+  method_id=3970
+  num_inline_caches=0
   method_id=4055
   num_inline_caches=0
   method_id=4057
@@ -17672,6 +17674,7 @@
   type_id=874
   type_id=876
   type_id=878
+  type_id=900
   type_id=901

This looks a bit like a "flaky" (non-deterministic) build, so I've tried our approach for that. Indeed it's non-deterministic: with 10 builds I get 2 different variants, just not the one your build produced. As 0.6.2 was still RB, maybe you have a clue what of the changes since then might cause this, @dessalines?

@dessalines
Copy link
Owner

Here's the diff: 0.6.2...0.6.3

There's a gradle upgrade, an AGP upgrade, and a few tablet dependencies added. Besides those, I don't see why my code changes would cause an issue.

@dessalines dessalines reopened this Dec 29, 2024
@IzzySoft
Copy link
Author

AGP update? Rings a bell, some newer AGP seems to have introduced some issues I've heard. Can you please tell me from which AGP you upgraded to which newer AGP? The case I heard of was 8.3.2 => 8.5.2.

@dessalines
Copy link
Owner

Looks like that one was from 8.7.2 -> 8.7.3

@IzzySoft
Copy link
Author

Hm, so not the same then (could be similar, though). As we already established "flakiness", maybe you could take a look at Differing assets/dexopt/baseline.profm and see if the offered "upstream fix" would work out?

@dessalines
Copy link
Owner

dessalines commented Jan 1, 2025

I'd probably rather wait for this to get fixed by google upstream, its an issue with their ART compiler, and it'd affect all apps equally. Not to mention almost every time this magically gets fixed by doing the next release, even tho I changed nothing code-related.

@IzzySoft
Copy link
Author

IzzySoft commented Jan 1, 2025

That can take years, @dessalines – and until then, our builders needlessly try to verify your app and are doomed to fail. Wouldn't it be better to find a "work around" until Google fixed it? You wouldn't believe how many of those we've already established while waiting for our issues to be processed by "big G". I need to check how many open issues are pending there already for RB problems, but IIRC at least more than 20, some of them for much more than "just half a year".

@dessalines
Copy link
Owner

I installed the utilities and was able to compare my .apk's prof and profm, with the one that was in my build folder, and they're identical. Makes sense since that fix said its been fixed for gradle versions > 8.1.0 , and this is on 8.12

I'm not able to compare it to f-droids unsigned builder tho because I don't have access to their builder.

@IzzySoft
Copy link
Author

IzzySoft commented Jan 2, 2025

Not talking about f-droid's builders (we don't use those at IzzyOnDroid). We use rbtlog. The setup is described in our wiki under Verification Builder – and you can even use that to "just build without comparing".

@dessalines
Copy link
Owner

It'd be helpful if you could link a document for how to do these things. Also I'm not on a debian-based machine, so installing some of these utilities (the non-python and non-android ones) might be difficult.

@dessalines
Copy link
Owner

Seems to have resolved itself again I think, I see 0.6.5 on izzyondroid.

@IzzySoft
Copy link
Author

IzzySoft commented Jan 2, 2025

Seems to have resolved itself again I think, I see 0.6.5 on izzyondroid.

But not RB, the green shield is not up (other than on F-Droid, a failed RB at IzzyOnDroid doesn't delay/disable an update, just the shield won't be up; it's planned that you can define in Droid-ify/NeoStore to wait with updates until the shield is up, or ignore it). The last build there was 0.6.2. Trying 0.6.5 now.

It'd be helpful if you could link a document for how to do these things.

There's documentation on the linked wiki, the Readme of rbtlog – and some more with rbhelper. But let's go step-by-step:

Also I'm not on a debian-based machine, so installing some of these utilities (the non-python and non-android ones) might be difficult.

If you could setup a Debian VM, I could provide you scripts that automate the setup of rbtlog + rbhelper. With that in place, I could give you a few hints on working with it; I'd just need to know what you want to do then. If you finally want to use that for building in general, that can be done (i.e. just build, not compare). If you want to cross-check your builds, that can be done too.

OK, run for 0.6.5 just finished saying it's not RB. Still the same issue as above, with the profiles not matching. Still looks like being flaky/non-deterministic, so the very same problem remains. I'll add our "shaker" again to see if one out of 10 builds might get your variant. But I'll now let the results through, whether RB succeeds or not. Ah, looks like we've been lucky, the second run succeeded. But that's a lottery; next time all 10 runs might fail, or the timeout hit before that…

Have you meanwhile been able to see if the "upstream fix" suggested with Differing assets/dexopt/baseline.profm would apply? That's something I cannot verify without you applying it. ah, you answered that already: should no longer be needed with the gradle version you use. Well, then I don't know any other solution apart from using the shaker – or both of us building with rbtlog…

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants