Image sorting crashes app when i want to force certain images to be drawn behind others

[RikNorakomi]

I followed your RPG tutorial on Youtube and am currently working on a 2d platform game where I have tombs/zombie spawn sites whose image I always want to have drawn behind other entities.

I've implemented a simple way of seting a flag on the ImageComponent when I want the Image to always be drawn behind to other as followed. It works for a while and the zombie spawn tomb is now always drawn behind other Entity images, but after a little time the app always crashes with a StackOverflow (see stacktarce below).
Question: Is this caused by a bug in Fleks? Or is my implementation wrong and how should I fix it?

class ImageComponent : Comparable<ImageComponent> {
    lateinit var image: FlipImage // Image Wrapper that allows for flipping an image on the x axis
    var zAxisAlignment: ZAlignment = IMAGE_XY_POSITION_COMPARISON

    override fun compareTo(other: ImageComponent): Int {
        if (other.zAxisAlignment == IMAGE_ALWAYS_IN_BACK) return 1
        if (zAxisAlignment == IMAGE_ALWAYS_IN_BACK) return -1

        val yDiff = other.image.y.compareTo(image.y)
        return if (yDiff != 0){
            yDiff
        } else {
            other.image.x.compareTo(image.x)
        }
    }

    companion object {
        private val log = logger<ImageComponent>()
        class ImageComponentListener(
            private val stage: Stage,
        ) : ComponentListener<ImageComponent> {
            override fun onComponentAdded(entity: Entity, component: ImageComponent) {
                stage.addActor(component.image)
            }

            override fun onComponentRemoved(entity: Entity, component: ImageComponent) {
                stage.root.removeActor(component.image)
            }
        }
    }
}

enum class ZAlignment {
    IMAGE_ALWAYS_IN_FRONT,
    IMAGE_ALWAYS_IN_BACK,
    IMAGE_XY_POSITION_COMPARISON
}

Stacktrace:
Exception in thread "main" com.badlogic.gdx.utils.GdxRuntimeException: java.lang.StackOverflowError at com.badlogic.gdx.backends.lwjgl3.Lwjgl3Application.<init>(Lwjgl3Application.java:172) at com.xxx.lwjgl3.Lwjgl3Launcher.main(Lwjgl3Launcher.kt:15) at comxxx.lwjgl3.Lwjgl3Launcher.main(Lwjgl3Launcher.kt) Caused by: java.lang.StackOverflowError at com.github.quillraven.fleks.collection.BagKt.med3(bag.kt:195) at com.github.quillraven.fleks.collection.BagKt.quickSort(bag.kt:250) at com.github.quillraven.fleks.collection.BagKt.quickSort(bag.kt:293) ...

[RikNorakomi]

Looks like I found a way to prevent the SO crash as followed:

override fun compareTo(other: ImageComponent): Int {

        if (other.zAxisAlignment == IMAGE_ALWAYS_IN_BACK && zAxisAlignment != IMAGE_ALWAYS_IN_BACK ) return 1
        if (zAxisAlignment == IMAGE_ALWAYS_IN_BACK && other.zAxisAlignment != IMAGE_ALWAYS_IN_BACK ) return -1

        val yDiff = other.image.y.compareTo(image.y)
        return if (yDiff != 0){
            yDiff
        } else {
            other.image.x.compareTo(image.x)
        }
    }

I assume this prevents an endless loop in the BagKt.quickSort function.

[Quillraven]

Hi,

I think the problem is in your comparator which causes the sorting to fail. From what I remember you must guarantee that it does not matter in which order your elements are passed to compareTo. The result must be the same.

And I think in your logic this is not the case because of the way you compare your Z value.

Here is an example how I did it recently.

override fun compareTo(other: Boundary): Int = when {
        z < other.z -> -1
        z > other.z -> 1
        y < other.y -> -1
        y > other.y -> 1
        x < other.x -> -1
        x > other.x -> 1
        else -> 0
    }

Here I first compare the Z values. If there is no difference I compare Y. If there is still no difference then X. And otherwise they are at the exact same position so it will be 0 and they are treated as equal for the sorting.

Can you try this version?

[RikNorakomi]

Yes this is it. I like it. Very clean solution! Thanks for the help.

[Quillraven]

Ah I was to slow but seems like my assumption was correct. Sorting did not work in your first version and caused an endless loop because you did not provide a unique sorting order per element.

Anyway, you can also take over my example which allows you to use any Z value for your entities.

E.g. -1 would be background, 1 is foreground and 0 is in between. But you are not limited to that and can use an arbitrary amount of z values (=layers).