Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Low disk space on download server (again) #505

Open
petefoth opened this issue Nov 16, 2023 · 11 comments
Open

Low disk space on download server (again) #505

petefoth opened this issue Nov 16, 2023 · 11 comments

Comments

@petefoth
Copy link
Contributor

petefoth commented Nov 16, 2023

While investigating problems with zip files not being rynced to the download server (see #503 ), a manual rsync failed with
rsync: [receiver] write failed on "/mnt/archive/zips/berlna/lineage-20.0-20231102-microG-berlna.zip": No space left on device (28)

Checking free space on the server showed we have nearly no space left

df -h
    Filesystem      Size  Used Avail Use% Mounted on
    <snip>
    /dev/vdb1       590G  559G  920M 100% /mnt/archive

The lack of space is party due to the fact we now publish loads of .img files in addition to the main ROM zip files and recovery images - see #405 & #467

Zipping up these images (see #483) caused the 'Update recovery` function of the Updater to fail #487, so was reverted - See PR #490

Unless we can find a way to zip up the shipped image files without breaking anything else we will probably have to keep only two recent builds instead of three

We may be able to free up enough space to avoid this by trimming some of the 17.1 builds that are still hanging around for the following devices:
kiwi, harpia, I001D, crackling, a7xelte, mdarcy_tab, merlin, s2, surnia, m8, I01WD, j7elte, m8d, osprey, RMX1801, mdarcy, s5neolte, quill_tab, Amber, quill, porg_tab, a3xelte, a5xelte, lux,wt88047,

@st3rox
Copy link

st3rox commented Nov 17, 2023

590GB total disk space? Is it not possible/practical to add more storage? I'm sure we could community-fund a new hard drive

On that note, the "Donate" button on the website goes straight to the microg project. I would like to donate directly to this project if possible

@petefoth
Copy link
Contributor Author

590GB total disk space? Is it not possible/practical to add more storage? I'm sure we could community-fund a new hard drive

That's a possibility, though it poses some logistical challenges :) And it may not be necessary: once the current build run is complete, there are a number of 17.1 builds that can be removed (even leaving the most recent 17.1 build for each device), I'll see how much space that frees up.

On that note, the "Donate" button on the website goes straight to the microg project. I would like to donate directly to this project if possible

Thanks. I will consult with the other maintainers and, if we decide we do need more disk space, we'll put in place a way of funding it. Apart from that, the project has very few expenses, and none of the project maintainers are doing it for the money ;)

@st3rox
Copy link

st3rox commented Nov 17, 2023

Thanks. I will consult with the other maintainers and, if we decide we do need more disk space, we'll put in place a way of funding it. Apart from that, the project has very few expenses, and none of the project maintainers are doing it for the money ;)

Idk what the physical system specs are like but donations could also go to upgrades, maybe even getting back up to 2 builds per month ;)

I personally have 4 devices that rely on this project and will gladly donate to keep it going smoothly

@petefoth
Copy link
Contributor Author

petefoth commented Nov 20, 2023

Idk what the physical system specs are like but donations could also go to upgrades, maybe even getting back up to 2 builds per month ;)
I personally have 4 devices that rely on this project and will gladly donate to keep it going smoothly

Thanks for the offer. I will discuss the disk space issue with the other maintainers, once I have tidied up and know how much spare we have. We will explore the possibility of increasing capacity, although upgrading our servers is not entirely straightforward.

PS Maybe not wise to rely too much on a project with very few resources and which offers no guarantees :)

@st3rox
Copy link

st3rox commented Nov 20, 2023

PS Maybe not wise to rely too much on a project with very few resources and which offers no guarantees :)

Hahaha yeah i know

My PC can build if I should ever need to, I have run it a few times to verify it works

I also could switch to vanilla lineageOS or something else out there, but I prefer microg to work out of the box , and I've been around since Cyanogenmod 7 so I'd like to stick with LineageOS

@petefoth
Copy link
Contributor Author

After the December build run completed the disk is less than 60% full.
But, some devices now only have a single build stored instead of two. That should be rectified in the January build run, so the used space will increase. I will wait and see how much free space we have then before deciding whether or not we need to do anthong more about this issue.

Filesystem      Size  Used Avail Use% Mounted on
....
/dev/vdb1       590G  329G  231G  59% /mnt/archive

@petefoth
Copy link
Contributor Author

petefoth commented Jan 2, 2024

After 30 hours and 12 builds in the January build run, we have used 4GB more disk space i.e. roughly 3 builds per GB. If that rate is typical, we should end up using just over 400GB out of 590GB :)

@petefoth
Copy link
Contributor Author

petefoth commented Jan 20, 2024

January build run complete

df -h
Filesystem      Size  Used Avail Use% Mounted on
...
/dev/vdb1       590G  331G  229G  60% /mnt/archive

But for many devices we have only a single 200.build. Check again after February build run, when we should have two.

@petefoth
Copy link
Contributor Author

Check again after February build run, when we should have two.

The free disk space numbers after the February build run, with two builds stored per device

Filesystem      Size  Used Avail Use% Mounted on
/dev/vdb1       590G  491G   70G  88% /mnt/archive

The figures include the new archive folder containing the final builds for devices which are no longer supported by LOS (see Build retention strategy #573 ), which takes up a total of 24G. Even if we removed the archive, we would still not have space to keep three builds per device rather than two. This is mainly due to the fact that we now keep several img files for each device build

@petefoth
Copy link
Contributor Author

petefoth commented Apr 7, 2024

At the end of the April build run, we are using 85% (473G) of the available disk space (590G). This consists of

  • 392 builds for 201 current devices: max 2 builds per device, 70 x 21.0 and 322 x 20.0 builds
  • 87 builds in archive for discontinued devices: 1 build per device, 84 x 18.1 and 3 x 20.0 builds

@petefoth
Copy link
Contributor Author

As of today (Mon 29 Jul 24) we are down to only 11GB free space: /dev/vdb1 590G 550G 11G 99%

We may have to start throwing stuff away

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants