Create and maintain official Flatpak packages on Flathub

[nekohayo]

In order to:

• Ensure timely updates "directly from the app maintainers" to our users, hence potentially faster development/release/feedback cycles
• Simplify the testing & support matrix for maintainers/contributors (i.e. to report a bug, users must report it in our official upstream bug tracker here, NOT in a distro bug tracker unless it concerns a distro package, and they must test with our flatpak packages, otherwise they're on their own)
• Facilitate testing for users of all distros
• Help with the revival of the project (Resurrecting this project #210) and interest from users

...it would be nice if the GTG project provided two (or three) sets of flatpak packages:

1. "Getting Things Fixed" (GTG nightly/dev snapshots; should this be on Flathub, or separately and only mentioned in our contributors/bug-reporter docs?)
2. Getting Things GNOME "current stable" (ex: 0.4), also published on Flathub
3. Getting Things GNOME "old stable" (0.3.1), just for migration/compatibility purposes to give a chance for users to have a "migration path" (i.e. if they want to be absolutely sure "current stable" is rock solid before moving to it).

The basis for these exists in the commits that @bilelmoussaoui made in https://github.com/bilelmoussaoui/gtg-flatpak

Bilal mentioned to me that he would prefer, from a policy perspective, that the upstream project (that is us) maintains the flatpak, rather than flathub folks maintaining a gazillion things.

So whoever is taking and spearheading this task here (not me, I'm afraid), would need to:

• Figure out what the steps are for a "clean" integration, and if there are any things missing below:
• Study Bilal's previous work, and adapt it to have two (or three!) sets of packages under the GTG project's umbrella
  • The latest (upcoming) stable release
  • The git "daily" version (nice to have)
  • The 0.3.1 legacy release (if possible, just to facilitate people's migration path)
• Clean up the manifest file a little bit (as per Bilal's suggestion), and update it for GNOME 3.34+
• Ideally patch/integrate the AppData file in GTG proper, to add the tags required by Flathub

  I (Jeff) can do that (I've already been updating that file), but I need someone to tell me exactly what I need to put in there.

• Ideally fix the little issues in https://github.com/bilelmoussaoui/gtg-flatpak/issues and report/migrate remaining issues here (with the "packaging" issue tag)
• Get at least the "stable" package version published on Flathub (and promise to update it once in a while whenever we make a release :)
• Document (for package maintainers and for users) this stuff somewhere on our eventual website/wiki and/or README, as part of Update websites and contributors documentation #200

[bilelmoussaoui]

I can help cleaning the manifest if needed. Regarding the dev/snapshot, there's currently no way to pass any argument to the docker container running on a github action, otherwise, we could have had nightlies with flatpak easily.
Most of the GNOME apps have a flatpak manifest in the main repository that builds against the master runtime (the next release) to ensure at least the app builds and runs for the next release. It also allows building and running it with GNOME Builder easily, which could help potential contributors :)

[diegogangl]

I tried getting gtg to work with builder a while back but apparently it doesn't support the "simple" build command, we would need a build system like meson to work out of the box with builder. We could take a look at how Timetrack does it: https://gitlab.gnome.org/danigm/timetrack/tree/master/flatpak

[diegogangl]

Here's the manifesto I've been using for GTG master, @bilelmoussaoui could you give it a quick check? If it looks good we can include it in the repo and make the 0.4-RC flatpak from this one

https://gist.github.com/diegogangl/973b0ffade9976993580c1c4401ba858

[bilelmoussaoui]

Here's the manifesto I've been using for GTG master, @bilelmoussaoui could you give it a quick check? If it looks good we can include it in the repo and make the 0.4-RC flatpak from this one

https://gist.github.com/diegogangl/973b0ffade9976993580c1c4401ba858

• https://gist.github.com/diegogangl/973b0ffade9976993580c1c4401ba858#file-gtg_master-flatpak-L6-L7 It would be nice to rename the icon to use the app-id and remove the rename-icon & copy-icon from the manifest
• https://gist.github.com/diegogangl/973b0ffade9976993580c1c4401ba858#file-gtg_master-flatpak-L4 bump the runtime to 3.36
• https://gist.github.com/diegogangl/973b0ffade9976993580c1c4401ba858#file-gtg_master-flatpak-L18-L19 you shouldn't need dconf access anymore
• The system & session bus access are a big no and should be limited to only the required bus's https://gist.github.com/diegogangl/973b0ffade9976993580c1c4401ba858#file-gtg_master-flatpak-L16-L17. You can run the flatpak with --log-session-bus or --log-system-bus to see which bus names are needed
• As it's a GTK3 application, you should use fallback-x11 instead of x11 https://gist.github.com/diegogangl/973b0ffade9976993580c1c4401ba858#file-gtg_master-flatpak-L11
• https://gist.github.com/diegogangl/973b0ffade9976993580c1c4401ba858#file-gtg_master-flatpak-L14 host filesystem access seems to be way too much for a ToDo app, what can't u achieve with the documents portal?

[diegogangl]

https://gist.github.com/diegogangl/973b0ffade9976993580c1c4401ba858#file-gtg_master-flatpak-L14 host filesystem access seems to be way too much for a ToDo app, what can't u achieve with the documents portal?

Well, how else are we going to rm -rf / when users don't complete their tasks before midnight?
We need to open some XMLs and INIs, but these are in XDG data and config which I understand are inside the sandbox. We do have an export plugin that needs to let users pick a file to export to. Does the document portal cover that?

The system & session bus access are a big no and should be limited to only the required bus's https://gist.github.com/diegogangl/973b0ffade9976993580c1c4401ba858#file-gtg_master-flatpak-L16-L17. You can run the flatpak with --log-session-bus or --log-system-bus to see which bus names are needed

Fixed. I added own-name for GTG* because we need to own GTG and GTGTasks (it's a long story: #277), is that acceptable?

Thanks for taking a look! I've updated the gist

[diegogangl]

BTW, is the location of our icon right or should we move it?
https://github.com/getting-things-gnome/gtg/tree/master/data/icons/hicolor/scalable/apps

[bilelmoussaoui]

The location looks right, but not the icon name. Flatpak only exports the files that are exported with the app-id; so the icons should be renamed to follow the app-id as well https://github.com/getting-things-gnome/gtg/blob/master/data/org.gnome.GTG.desktop#L10

and you don't need to own org.gnome.GTG* because you already own your app-id by default (otherwise you won't even be able to run your gtk app in a flatpak sandbox)

The documents portal (Gtk.NativeFileChooser) allows you to get access to a file/directory temporally
which is perfect for the export use case.

As the tasks are saved under xdg-config (hopefully), you shouldn't need any permissions to edit/remove them. No permissions needed for that

[diegogangl]

Alright, fixed the icon and used Gtk.NativeFileChooser for the export plugin, once we tag an RC we should be ready to publish

[diegogangl]

@bilelmoussaoui I'm trying to build the flatpak and I'm hitting a problem now:

Failed to register: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: org.freedesktop.DBus.Error.ServiceUnknown

I removed the old dbus wrapper but I'm still getting this message (and gtg refuses to start). Are we missing some permissions in the file?

[bilelmoussaoui]

Per https://github.com/getting-things-gnome/gtg/blob/master/GTG/gtk/application.py#L85, this might not work if you're running debug release. The devel version requires a different manifest in general, with a different app-id :)

[diegogangl]

There's no debug release yet, that depends on the Meson port. The app id is only changing if you run gtg with the debug parameter (-d) at the moment. I think the problem is that the dbus wrapper has some decorators that trigger the error even if you only import it. I could just get rid of it for now, but then the signal callback here https://github.com/getting-things-gnome/gtg/blob/master/GTG/core/timer.py#L45 throws a different error about needing a main loop. Also it seems like we are interacting directly with the system bus which is a no-no right?

[bilelmoussaoui]

Most be dbus lib that does some weird things, you shouldn't require the whole system bus just to talk to org.freedesktop.login1

[diegogangl]

I think that's the problem, we're not talking to org.freedesktop.login1, we're listening for a signal.

        bus = dbus.SystemBus()
        bus.add_signal_receiver(self.on_prepare_for_sleep,
                                'PrepareForSleep',
                                'org.freedesktop.login1.Manager',
                                'org.freedesktop.login1')

Is there any documentation on how to do this inside flatpak? I can't find anything.

[bilelmoussaoui]

I think that's the problem, we're not talking to org.freedesktop.login1, we're listening for a signal.

        bus = dbus.SystemBus()
        bus.add_signal_receiver(self.on_prepare_for_sleep,
                                'PrepareForSleep',
                                'org.freedesktop.login1.Manager',
                                'org.freedesktop.login1')

Is there any documentation on how to do this inside flatpak? I can't find anything.

Doesn't really matter if you listen to a signal or not;

[diegogangl]

I ended up removing the old dbus wrapper, a function in the plugin api and fixing the timer. Now it's all working correctly under the restrictions. 👍

[nekohayo]

So my understanding is that:

• @diegogangl's "dev version" flatpak is going to be ready when @leio's meson-improvements branch is merged soon...
• what about the "stable release" flatpak version, is it just the exact same thing as the dev version but tied to a particular commit, or is there more to it (for the version number to show up etc.)?
• what would remain afterwards in terms of publishing this whole thing? Where/how to host it, script it, etc.?

And do we have to worry right now about the org.gnome.GTG vs io.gnome.GTG namespace thingy as per https://discourse.gnome.org/t/official-proposal-how-we-define-gnome-software/3371/16 or do we consider we have enough yaks in the tundra and that we should just release as org.gnome.GTG for the time being and "if it breaks people's flatpaks later on, we just blog to tell them how to uninstall and reinstall and migrate their data from one folder to another"?

[diegogangl]

I pushed a flatpak manifest into master yesterday. It's already using meson, the only thing missing is to point to a specific commit and voila, we have a stable flatpak.
For the dev/nightly version we need meson to provide a "profile" option, so we can change the appid to org.gnome.GTGDevel (or something like that).

And do we have to worry right now about the org.gnome.GTG vs io.gnome.GTG namespace thingy

Remember that to use io.gnome we would have to be accepted in Gnome's Circle. And we still can't tick all the boxes for that. So I suggest we launch as we are for now. We are not the only ones who will have to migrate anyways.

what would remain afterwards in terms of publishing this whole thing? Where/how to host it, script it, etc.?

Flathub? We have to look into the publishing workflow there. Once we have pretty screenshots of course :)

[nekohayo]

Flathub? We have to look into the publishing workflow there. Once we have pretty screenshots of course :)

I'll try to get something running on my Fedora machines to take new "marketing" screenshots.

As for Flathub submissions, it seems from https://github.com/flathub/flathub/wiki/App-Submission that it's mostly just a matter of cloning the flathub repo under our own project, making a branch in it, and then doing a pull request from that to flathub itself; but how does one handle "stable" vs "dev" versions in there, two branch names I suppose? "gtg" and "gtg-nightly" or something?

[diegogangl]

Seems like we need to have a branch called "beta" in our flathub repo:
https://blogs.gnome.org/alexl/2019/02/19/changes-in-flathub-land/

[bilelmoussaoui]

You can't publish a nightly release in Flathub, at least not yet. Most of nightly apps hosted on GNOME uses a different app-id to allow both versions to co-exists. Once you publish your application on Flathub it must have one unique app-id. You can still use the nightly design for the beta version but it must use the same app-id.

[diegogangl]

Ah that's a shame. Having the same app-id means users would test the beta with their actual data from the stable version.

[nekohayo]

Ah that's a shame. Having the same app-id means users would test the beta with their actual data from the stable version.

I actually consider it a (safety) feature that the development version doesn't touch the user's regular data, so I don't think this should be blocking it.

If we can somehow have the nightly (so to speak, with its different app-id) flatpak published, that would be very relevant to include in the pre-release announcement blog post that I've drafted...

[nekohayo]

A few observations on the existing flatpak package attempt by Diego:

• It depends on the whole GNOME "SDK" (which weighs 670 MB) instead of just the GNOME platform; would it be possible for it to only depend on the platform? Not sure why @bilelmoussaoui had it use the SDK back then, maybe it's not needed anymore?
• It doesn't seem to package or properly associate the app icon, at least it doesn't show up in GNOME Shell or Ubuntu's version of dash-to-dock in my Ubuntu LTS testing machine
• There's the open question of if Flathub would allow us to just have the git/beta version published at first before the 0.4 stable version is also made available: https://discourse.flathub.org/t/beta-without-a-stable-release/508

[bilelmoussaoui]

A few observations on the existing flatpak package attempt by Diego:

* It depends on the whole GNOME "SDK" (which weighs 670 MB) instead of just the GNOME platform; would it be possible for it to only depend on the platform? Not sure why @bilelmoussaoui had it use the SDK back then, maybe it's not needed anymore?

that was probably a typo

* It doesn't seem to package or properly associate the app icon, at least it doesn't show up in GNOME Shell or Ubuntu's version of dash-to-dock in my Ubuntu LTS testing machine

probably an issue on the app itself

* There's the open question of if Flathub would allow us to just have the git/beta version published at first before the 0.4 stable version is also made available: https://discourse.flathub.org/t/beta-without-a-stable-release/508

sure, submit the app and mention you need to have it on beta first. Beta != git btw :) it has to be at least usable/ can be run

[mparker17]

Hello! Thanks for the excellent blog post on Planet GNOME, @nekohayo !

bilelmoussaoui: You can't publish a nightly release in Flathub, at least not yet. Most of nightly apps hosted on GNOME uses a different app-id to allow both versions to co-exists. Once you publish your application on Flathub it must have one unique app-id. You can still use the nightly design for the beta version but it must use the same app-id.

diegogangl: Ah that's a shame. Having the same app-id means users would test the beta with their actual data from the stable version.

I don't know if this is useful (maybe you already know about this), but there are flathub-beta and gnome-nightly flatpak remotes...

flatpak remote-add --if-not-exists flathub-beta https://flathub.org/beta-repo/flathub-beta.flatpakrepo
flatpak remote-add --if-not-exists gnome-nightly https://nightly.gnome.org/gnome-nightly.flatpakrepo

... I think (citation definitely needed: I'm very very new to the whole linux application development / flatpak stuff) that you can use the same app-id and publish to these channels (EDIT: re-reading the docs for gnome-nightly, looks like they encourage a different app-id so you can "install ... [apps] ... alongside the regular stable versions"). They aren't well-documented, though.

I first learned about the Flathub Beta Repo because I was tracking when Firefox was going to be available in Flathub but I see there's also more info about beta releases on Alexander Larsson's blog. More info about the GNOME nightly flatpak repo is on the Apps/Nightly wiki.

[nekohayo]

Glad you liked the blog post, and thanks for the comment here, hopefully there's some insights that Diego & others can use here :)

Meanwhile I encountered a whole other issue where the package really doesn't like playing nicely with a system install (maybe Bilal would know what causes this): #367

[nekohayo]

Diego has made a request here: flathub/flathub#1618

[mparker17]

For whatever it's worth, I can now confirm that I can install GTG 0.4.0 by running...

flatpak remote-add --if-not-exists flathub-beta https://flathub.org/beta-repo/flathub-beta.flatpakrepo
flatpak install flathub-beta org.gnome.GTG

🙌Thanks! 😁

[nekohayo]

Cool! For what it's worth https://flathub.org/apps/details/org.gnome.GTG does not yet seem to be showing online (it gives a 404), I don't know how long it takes for that to happen...

[bilelmoussaoui]

Cool! For what it's worth https://flathub.org/apps/details/org.gnome.GTG does not yet seem to be showing online (it gives a 404), I don't know how long it takes for that to happen...

The frontend doesn't support beta apps yet.

[nekohayo]

The stable version now shows up on the flathub website and overall it seems to "Just Work™", so Mission Accomplished.