Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[XABT] Create separate assembly preparation tasks. #9637

Merged
merged 2 commits into from
Jan 22, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

This file was deleted.

108 changes: 108 additions & 0 deletions src/Xamarin.Android.Build.Tasks/Tasks/CompressAssemblies.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
#nullable enable

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Microsoft.Android.Build.Tasks;
using Microsoft.Build.Framework;
using Xamarin.Android.Tools;

namespace Xamarin.Android.Tasks;

/// <summary>
/// Compresses assemblies using LZ4 compression before placing them in the APK.
/// Note this is independent of whether they are stored compressed with ZIP in the APK.
/// Our runtime bits will LZ4 decompress them at assembly load time.
/// </summary>
public class CompressAssemblies : AndroidTask
{
public override string TaskPrefix => "CAS";

[Required]
public string ApkOutputPath { get; set; } = "";

public bool EmbedAssemblies { get; set; }

[Required]
public bool EnableCompression { get; set; }

public bool IncludeDebugSymbols { get; set; }

[Required]
public string ProjectFullPath { get; set; } = "";

[Required]
public ITaskItem [] ResolvedFrameworkAssemblies { get; set; } = [];

[Required]
public ITaskItem [] ResolvedUserAssemblies { get; set; } = [];

[Required]
public string [] SupportedAbis { get; set; } = [];

[Output]
public ITaskItem [] ResolvedFrameworkAssembliesOutput { get; set; } = [];

[Output]
public ITaskItem [] ResolvedUserAssembliesOutput { get; set; } = [];

public override bool RunTask ()
{
if (IncludeDebugSymbols || !EnableCompression || !EmbedAssemblies) {
ResolvedFrameworkAssembliesOutput = ResolvedFrameworkAssemblies;
ResolvedUserAssembliesOutput = ResolvedUserAssemblies;
return true;
}

var compressed_assemblies_info = GetCompressedAssemblyInfo ();

// Get all the user and framework assemblies we may need to compresss
var assemblies = ResolvedFrameworkAssemblies.Concat (ResolvedUserAssemblies).Where (asm => !(ShouldSkipAssembly (asm))).ToArray ();
var per_arch_assemblies = MonoAndroidHelper.GetPerArchAssemblies (assemblies, SupportedAbis, true);
var compressed_output_dir = Path.GetFullPath (Path.Combine (Path.GetDirectoryName (ApkOutputPath), "..", "lz4"));

foreach (var kvp in per_arch_assemblies) {
Log.LogDebugMessage ($"Compressing assemblies for architecture '{kvp.Key}'");

foreach (var asm in kvp.Value.Values) {
MonoAndroidHelper.LogIfReferenceAssembly (asm, Log);

var compressed_assembly = AssemblyCompression.Compress (Log, asm, compressed_assemblies_info, compressed_output_dir);

if (compressed_assembly.HasValue ()) {
Log.LogDebugMessage ($"Compressed '{asm.ItemSpec}' to '{compressed_assembly}'.");
asm.SetMetadata ("CompressedAssembly", compressed_assembly);
}
}
}

ResolvedFrameworkAssembliesOutput = ResolvedFrameworkAssemblies;
ResolvedUserAssembliesOutput = ResolvedUserAssemblies;

return !Log.HasLoggedErrors;
}

IDictionary<AndroidTargetArch, Dictionary<string, CompressedAssemblyInfo>> GetCompressedAssemblyInfo ()
{
var key = CompressedAssemblyInfo.GetKey (ProjectFullPath);
Log.LogDebugMessage ($"Retrieving assembly compression info with key '{key}'");

var compressedAssembliesInfo = BuildEngine4.UnregisterTaskObjectAssemblyLocal<IDictionary<AndroidTargetArch, Dictionary<string, CompressedAssemblyInfo>>> (key, RegisteredTaskObjectLifetime.Build);

if (compressedAssembliesInfo is null)
throw new InvalidOperationException ($"Assembly compression info not found for key '{key}'. Compression will not be performed.");
Comment on lines +88 to +94
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old code was like this, but...

What saves this RegisterTaskObject value? If we can avoid it, I wouldn't recommend using RegisterTaskObject here as a project could have:

<TargetFrameworks>net9.0-android;net10.0-android</TargetFrameworks>

The two would overwrite each other with this value for the key. The CompressedAssemblyInfo type will be incompatible between the two assemblies under .NET framework because their versions do not match.

Can we calculate the value within this task and not use RegisterTaskObject?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's generated here:

string key = CompressedAssemblyInfo.GetKey (ProjectFullPath);

The problem is that we have to generate the LLVM IR assembly sources way, way before we package the assemblies and the info gathered while generating the native assembly sources is crucial to the correct construction of the assembly store. We allocate buffers of specific size for each of the assemblies, so that decompressed data for each assembly gets a buffer of the correct size. This saves memory. The way it works is that each assembly has an index, allocated while generating the native assembler sources, and that index is used at run time to find the correct buffer.

Maybe the assemblies could be compressed just before generation step for the native assembler sources, but I don't know if it isn't too early.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I could tell, the only thing that gets used from this is the "DescriptorIndex":

AssemblyData compressedAssembly = new AssemblyData (assembly.ItemSpec, info.DescriptorIndex);

It feels like this could be a piece of metadata added to the @(Item) instead, but I certainly haven't decoded the full picture.

Any change here should likely be done in its own PR.


return compressedAssembliesInfo;
}

bool ShouldSkipAssembly (ITaskItem asm)
{
var should_skip = asm.GetMetadataOrDefault ("AndroidSkipAddToPackage", false);

if (should_skip)
Log.LogDebugMessage ($"Skipping {asm.ItemSpec} due to 'AndroidSkipAddToPackage' == 'true' ");

return should_skip;
}
}
Loading
Loading