Build fails on mupen64plus ui-console

[kattjevfel]

Trying to compile this project on Arch Linux as discussed on email, using the dependencies and commands in the readme.
Compiled in a clean chroot using extra-x86_64-build.

ar: creating mupen64plus-video-glide64mk2.a
make[4]: Leaving directory '/build/marley-git/src/marley/mupen64plus/Source/video-glide64mk2'
ar rs libmupen64plus.a  _obj/api/callbacks.o _obj/api/common.o _obj/api/config.o _obj/api/debugger.o _obj/api/frontend.o _obj/api/vidext.o _obj/backends/api/video_capture_backend.o _obj/backends/plugins_compat/audio_plugin_compat.o _obj/backends/plugins_compat/input_plugin_compat.o _obj/backends/clock_ctime_plus_delta.o _obj/backends/dummy_video_capture.o _obj/backends/file_storage.o _obj/device/cart/cart.o _obj/device/cart/af_rtc.o _obj/device/cart/cart_rom.o _obj/device/cart/eeprom.o _obj/device/cart/flashram.o _obj/device/cart/sram.o _obj/device/controllers/game_controller.o _obj/device/controllers/paks/biopak.o _obj/device/controllers/paks/mempak.o _obj/device/controllers/paks/rumblepak.o _obj/device/controllers/paks/transferpak.o _obj/device/dd/dd_controller.o _obj/device/device.o _obj/device/gb/gb_cart.o _obj/device/gb/mbc3_rtc.o _obj/device/gb/m64282fp.o _obj/device/memory/memory.o _obj/device/pif/bootrom_hle.o _obj/device/pif/cic.o _obj/device/pif/n64_cic_nus_6105.o _obj/device/pif/pif.o _obj/device/r4300/cached_interp.o _obj/device/r4300/cp0.o _obj/device/r4300/cp1.o _obj/device/r4300/idec.o _obj/device/r4300/interrupt.o _obj/device/r4300/pure_interp.o _obj/device/r4300/r4300_core.o _obj/device/r4300/tlb.o _obj/device/rcp/ai/ai_controller.o _obj/device/rcp/mi/mi_controller.o _obj/device/rcp/pi/pi_controller.o _obj/device/rcp/rdp/fb.o _obj/device/rcp/rdp/rdp_core.o _obj/device/rcp/ri/ri_controller.o _obj/device/rcp/rsp/rsp_core.o _obj/device/rcp/si/si_controller.o _obj/device/rcp/vi/vi_controller.o _obj/device/rdram/rdram.o _obj/main/main.o _obj/main/util.o _obj/main/cheat.o _obj/main/eventloop.o _obj/main/rom.o _obj/main/savestates.o _obj/main/screenshot.o _obj/main/sdl_key_converter.o _obj/main/workqueue.o _obj/plugin/plugin.o _obj/plugin/dummy_video.o _obj/plugin/dummy_audio.o _obj/plugin/dummy_input.o _obj/plugin/dummy_rsp.o _obj/minizip/ioapi.o _obj/minizip/zip.o _obj/minizip/unzip.o _obj/md5/md5.o _obj/osal/dynamiclib_unix.o _obj/osal/files_unix.o _obj/osd/osd.o _obj/device/r4300/recomp.o _obj/device/r4300/x86_64/assemble.o _obj/device/r4300/x86_64/dynarec.o _obj/device/r4300/x86_64/regcache.o _obj/asm_defines/asm_defines.o _obj/osd/oglft_c.o _obj/oglft/OGLFT.o _obj/device/r4300/x86_64/dyna_start.o 
ar: creating libmupen64plus.a
make[4]: Leaving directory '/build/marley-git/src/marley/mupen64plus/Source/core'
make[3]: Leaving directory '/build/marley-git/src/marley/mupen64plus/Source'
make[2]: *** [Makefile:8: Source] Error 2
make[2]: Leaving directory '/build/marley-git/src/marley/mupen64plus'
make[1]: *** [Makefile:1094: all-recursive] Error 1
make[1]: Leaving directory '/build/marley-git/src/marley'
make: *** [Makefile:939: all] Error 2
==> ERROR: A failure occurred in build().
    Aborting...
==> ERROR: Build failed, check /var/lib/archbuild/extra-x86_64/katt/build

Full log for that segment: https://gist.github.com/kattjevfel/56e9394aae8bd6eda8f02da42eb7f66d

[beaumanvienna]

Good idea to compile in a clean chroot. Did you follow this wiki?

It fails because of make[3]: *** No rule to make target 'input-sdl/_obj/config.o', needed by 'mupen.a'. Stop.

I did yesterday a sudo pacman -Syu under Arch and it compiled normally afterwards.

I get

~/dev/marley_release$ find . -name "config.o"
./mupen64plus/Source/core/_obj/api/config.o
./mupen64plus/Source/input-sdl/_obj/config.o

So normally the file input-sdl/_obj/config.o should be there. The target that should create this file is *************** mupen64plus input-sdl ***************. The makefile for input-sdl is here.

[beaumanvienna]

When I follow the classic way described in the wiki, it works in a chroot:

mkdir ~/chroot
CHROOT=$HOME/chroot
mkarchroot $CHROOT/root base-devel
arch-nspawn $CHROOT/root pacman -Syu
arch-nspawn $CHROOT/root pacman -S libaio libjpeg-turbo libpcap libpulse portaudio sdl2 soundtouch gnu-free-fonts boost-libs alsa-lib bluez-libs enet libevdev libpulse libx11 libxi libxrandr lzo mbedtls libsndfile mesa libudev.so libusb-1.0.so libgl glew glibc zlib glu cmake git libglvnd python qt5-tools freetype2 qt5-base sfml libavcodec.so python libavformat.so libavutil.so libcurl.so libminiupnpc.so libswscale.so sdl2_image sdl2_ttf nasm boost libpng libsamplerate wxgtk2 libzip sndio aom zip bluez bluez-plugins bluez-utils
arch-nspawn $CHROOT/root git clone https://github.com/beaumanvienna/marley 
arch-nspawn $CHROOT/root bash
cd marley
aclocal && autoconf && automake --add-missing --foreign && ./configure && make

Here is the output. Search for *************** mupen64plus input-sdl ***************. _obj/config.o comes after plugin.o, autoconfig.o, and sdl_key_converter.o.

[kattjevfel]

Yes, I used the "Convenience way", using the extra-x86_64-build script, it does all the work in one go.
Could you try with an actual PKGBUILD? I decided to publish my WIP one here: https://github.com/kattjevfel/pkgbuilds/blob/master/WIP/marley-git/PKGBUILD

[beaumanvienna]

A slightly modified version of your PKGBUILD script is compiling. I changed the dependencies to how they are described in Marley's Readme. Here is the output.

However, the top-level configure script does not have an install target. extra-x86_64-build PKGBUILD fails for me after the build stage.

[beaumanvienna]

Here is a comparison of the two builds.

It looks like the environments have slightly different environment. If you look at page 5 where the actual configure run starts, different settings are detected. That's weird.

[kattjevfel]

What I see is that you also have multilib in your chroot, and I do not (since this doesn't list any need for multilib).
You also seem to be doing the entire build manually? This very much defeats the purpose of makechrootpkg. The point of it is to test a pkgbuild on a clean system to make sure it'll work on other systems.

The way you're meant to use it is updating the PKGBUILD and re-run either extra-x86_64-build or makechrootpkg -c -r /your/chroot until everything works, and then follow up with namcap (actually done with the first option) on the built package to check for un-needed or un-listed dependencies. This may help you greatly if building outside a chroot and it can tell you what package you're missing.

[beaumanvienna]

Can you check if this script compiles ok for you?
https://github.com/beaumanvienna/marley/blob/master/arch/PKGBUILD
It will fail in the installation stage, but you should get the binary built. I need to add an installation target to configure.ac.

Let me know if you intend on making a pull request. Since the script originally came from you, I feel that contribution should be recorded properly. You can remove bluez bluez-plugins bluez-utils. Check with namcap if there's anything else to be removed.

The multilib in the manually chroot is because I have it enabled on my system. It is not needed. I don't think this explains why the "convenient way" and the "classical way" have different libs. The x86_64 libs shouldn't be affected.

Testing with extra-x86_64-build makes sense to see if the PKGBUILD works. The manual chroot way is to test if folks can simply clone the repo and build it per the readme.

[kattjevfel]

https://gist.github.com/kattjevfel/7e0e6b2ac3819417f76b455c7b440a15 slightly different error with that pkgbuild

also namcap can only look at pacman packages, so it won't be of any use until package() actually does anything,

[beaumanvienna]

Your original PKGBUILD script actually works for me: console output

There is something in your settings different. We should make a comparison of the above console output and yours. Can you post your entire console output, please? I configured in gnome-terminal CTRL+SHIFT+A to select all. Simply selecting it with the mouse takes long. I'm using an old git repository to dump the output text files because none of the paste-bin websites accepts 5MB text input.

Did you alter any config files on your machine to change the behavior of the AUR tools?

[beaumanvienna]

The error that your getting is because you invoke make with -j16. I will fix it.

In the meantime, if you could quickly confirm my theory and set the CPU proc number in your AUR build environment to 1 and let me know if it compiles.

[beaumanvienna]

(hopefully) fixed with 99f76b4

[kattjevfel]

While I do have my main makepkg.conf set to MAKEFLAGS="-j$(nproc)", the chroot should be using the default settings, which is unset, which at least with other projects typically defaults to 1 core. Not quite sure how it "slipped through" or if that really is the case at all since I can't find where you'd possibly have seen that.

If I'm reading that commit right you're now forcing to use all cores? That might not be the greatest idea since not everyone want their computer (potentially a laptop with a bad cooling solution) to be maxed out for a long time. My opinion is that it should default to 1 core and then the user can do whatever they choose.

I'll have to try to force it to run with 1 core later since It's getting way to late into the night as I'm writing this, but I'm fairly certain it was already compiling with only 1 core since it'd been a lot faster otherwise..

[beaumanvienna]

I fixed it already upstream, so you can just try with your normal setting MAKEFLAGS="-j$(nproc)".

No, it didn't slip through. I actually saw the same error message about config.o missing today when I closed out issue #30. Mednafen and mupen64 were indeed running only on one core up until today, but that is fixed now. Mupen64plus was even choking on the -j16 as you noticed.

The default I would like to keep at using all cores for a simple make command. If folks wanted to override that with make -j1 they can. Or so I believe. I had to test if it really worked. The command line MAKEFLAGS options should prevail over the top-level setting. Only if you use override such as in the command, the makefile setting will win. I timed the compile run today. With all cores 14 minutes on the new AMD machine and 17 on my other big tower. The CPU time, however, on the old machine was ~5000s. I guess it is accumulating what all cores spent combined. That would be 1 1/2 hour. Having the default at 17 minutes is long enough...

[beaumanvienna]

Thinking about what you said regarding the number of CPU used for make, you might have a point. The Ryzen 7 Threadripper reports 128 CPU cores. The memory usage for make -j128 would go through the roof. Compiling with 12 cores, the memory usage on my machine goes up by about 2GB. Multiply that number by 128/12 and you're at ~22GB. If the machine has not enough memory, it would freeze up.

And I realized what I said above about the make flags from the command line superceeding the settings in Makefile.am, this is only true for the front end files and mednafen. Mupen64plus, ppsspp, dolphin, and pcsx2 are hardcoded to the available CPU cores:

• See mupen64plus
• See ppsspp
• See dolphin
• See pcsx2

[beaumanvienna]

I addressed your feedback and changed the default CPU cores to be used for compiling to one core. Marley now supports the number of CPU cores specified via export MAKEFLAGS=-j$(nproc). To take full advantage of this feature, the compile instructions are

export MAKEFLAGS=-j$(nproc)
aclocal && autoconf && automake --add-missing --foreign && ./configure --prefix=/usr MAKEFLAGS=$MAKEFLAGS
make

This is because the configure run already compiles the library ffmpeg for ppsspp, so that the following sub-configure run for ppsspp can find it. ppsspp requires a modified version of ffmpeg. The configure script is for some unknown reason blocking the shell variable MAKEFLAGS, unless it set as described in the above instructions. You can say echo $HOME in the script and it works, however, if you said echo $MAKEFLAGS, it only works if passed as an argument to the script, but not from the shell variable directly. With ffmpeg now also being compiled on all cores, I shaved off another three minutes from the compile-time:

real	10m51.476s
user	84m51.135s
sys	5m3.330s

I recommended you add MAKEFLAGS=$MAKEFLAGS to your PKGBUILD script for Marley.

Refer to commit e3880b9

I'd say issue #33 is fully addressed and can be closed. There is more to review for the Arch package build script, please refer to #5.

[kattjevfel]

Correct, I finally got around to testing it in my chroot and all is well.