From c3fd4bcdaf6011c5ed41177b560e5c895055db9a Mon Sep 17 00:00:00 2001 From: Zicklag Date: Sat, 25 Nov 2023 20:10:33 -0600 Subject: [PATCH] feat: add post: Jumpy Migration to the Bones Framework --- .../index.md | 292 ++++++++++++++++++ 1 file changed, 292 insertions(+) create mode 100644 content/blog/jumpy-migration-to-bones-framework/index.md diff --git a/content/blog/jumpy-migration-to-bones-framework/index.md b/content/blog/jumpy-migration-to-bones-framework/index.md new file mode 100644 index 0000000..f170ff1 --- /dev/null +++ b/content/blog/jumpy-migration-to-bones-framework/index.md @@ -0,0 +1,292 @@ +--- +title: Jumpy's Migration to the Bones Framework +description: > + We recently migrated Jumpy to our new Bones Framework. + Here's all about what we did and what it means for Jumpy. +template: blog/page.html +date: 2023-11-25 + +taxonomies: + authors: + - Zicklag +--- + +Recently we migrated Jumpy from Bevy to our new [Bones Framework][bones]. +Here we're going to go over what that means, and how it helps us. + +## What is the Bones Framework? + +Bones Framework is our new **game engine**, built on the Bones ECS. Before now, Bones was mostly just an ECS, +and some other related components, not it's own game engine. It was used to give Jumpy a deterministic game +core that supported snapshot and restore. The latest refactor, though, change **a lot**. + +### Asset System + +The thing that started this journey was the asset system. Before the refactor, Bones had a very clunky +integration with Bevy's asset system that did _just enough_ to make it usable in Jumpy. It was not optimal +and it was only meant to be temporary, until bones got it's own asset system. + +As I started working on the asset system, I was keeping in mind some important things that we wanted to +support: + +- **Asset Packs:** We wanted to have asset packs be a first-class feature, so that it would be trivial to + make games that allows users to add their own content. +- **Network Synchronization:** If you have mods, and you have network games, then we want a way to for you + to synchronize your mods over the network, without having to do a super-annoying manual transfer. +- **Convenient Metadata Assets:** Jumpy was already making heavy use of what we called metadata assets. + These are inter-connected YAML files that make it easy to organize all of our game data, such as items, + maps, player skins, etc. This pattern worked really well for making the game, but wasn't super convenient + to accomplish in Bevy, so we wanted to make sure that this worked super well out-of-the box in Bones. +- **Modding:** We wanted to be able to have mods create their _own_ kind of metadata assets, so that if a + mod added a "clam" element, it could load the information for it from `.clam.yaml` files. + +The trickiest part was related to metadata assets and modding. We needed a way for YAML files to be loaded +into Rust structs, kind of like the Rust [`serde`] library does, but we also needed to be able to _describe_ +custom asset types that might not come from Rust. + +At some point, I finally realized that this _describability_ was actually the _same_ thing that we needed +for first-class scripting support in the Bones ECS. + +### Bones Schema + +This kicked off the creation of [`bones_schema`], which, for those who know about it, is similar in concept +to [`bevy_reflect`]. The idea is that we needed a way to describe data with schemas, that could be used in +multiple crucial places: + +- **The ECS:** The schema gives us memory layout information needed by the ECS to allocate memory for the + data. +- **The Asset System:** The schema tells us what fields and types the data has, so that we can load the data + from YAML or JSON files. +- **The Scripting System:** The schema tells us about the fields and types the data has so that scripts can + read and write the data. + +The new schema system opened the door _all_ of these systems to cleanly interact with each-other, and it +helped simplify some of the existing code in the Bones ECS, too. + +### A Unified Framework + +While the schema system is fairly simple in concept and implementation, there was still just a _lot_ to do. +Even if something is simple, a lot of simple stuff is still a lot of stuff. While migrating all of Bones +to using the new schema system, _and_ creating a new asset system, it seemed to make sense that we might +as well start checking off the next big ticket item that we needed to get done: creating a **unified bones +framework**. + +As it currently stood, Jumpy was made out of a strange combination of Bones and Bevy. There was `jumpy_core`, +which was built on Bones and it's clunky Bevy asset integration, and then there was `jumpy` proper, which was +a Bevy game that used `jumpy_core` for the gameplay section of the game. + +Inside `jumpy` you had to deal with the Bones ECS _and_ the Bevy ECS and dealing with that juggle was not +super pretty. It was confusing, and it made it probably twice as confusing for contributors who, depending on +what part of the game they want to contribute to, have to learn two different game engines that are eerily +similar, but still very different. + +On top of all of this, there were tons of things that we were doing in Jumpy that we would rather be done in +bones, so that other games could take advantage of it, and so that Jumpy could stay focused on what makes +Jumpy unique. + +This called for the Bones Framework, so that we could make Jumpy a Bones game, not a "Bones + lots of glue + +Bevy" game. + +So, while we migrated to the new schema design, and made a new asset system, we _also_ started moving parts +of Jumpy out of Jumpy and into Bones. We migrated things like the localization system and, importantly, our +egui components. Migrating egui to Bones was a big win because, previously, there was no way to do UI from +inside of the Bones world. All of the UI was trapped outside in the Bevy part of Jumpy. Now though, Bones +has access to the full power of egui. + +### Becoming Renderer Agnostic + +As we shaped the Bones Framework we also made another useful accomplishment by making Bones **renderer +agnostic**. After we had our own asset system, Bones was no longer tied to Bevy. Today we still use Bevy +for rendering, but there's nothing that forces us to. Bones Framework games can be moved to any renderer +that can render sprites, tilemaps, egui, and debug lines. + +This is a great freedom for us to have, not that there is anything wrong with Bevy, but we can also see +that Bevy is putting a great deal of effort into 3D rendering and other things that we don't necessarily +need. Being agnostic means that we take our project into our own hands a bit more, and allow us to focus +on what is important to us when necessary. + +Also, by focusing rendering on _simple_ primtives, such as sprites, tilemaps, lines, and UI, we actually +keep the game more scriptable and moddable, because we are not depending on the ability to do arbitrarily +complex rendering directly from the ECS. + +### Maintaining Rendering Power and Flexibility + +Limiting our rendering primitives, though, does mean that the Bones Framework might not do all the +different rendering things that some games might want or need. That's totally fine, though, because we +allow you to make custom rendering integrations. + +For example, say you are making a game that uses our official `bones_bevy_renderer`. This is the easiest +way to get started with Bones, and it uses Bevy to render your game, without you having to think about it +or do anything other than write your game for Bones. + +But then you see that there's a really cool [Bevy Magic Light 2D][bml2d] plugin that can make 2D +lighting effects, and you want to use that in your bones game. Bones allows you to create your own +integration between it and the Bevy Magic Light plugin, so that you can control the lighting plugin using +Bones ECS components. + +Once you create your integration you get the ability to get cool lighting effects in your Bones game, _and_ +it will be compatible with Bones's asset pack and scripting system! + +Or, in another scenario, immagine you wanted to make a "falling sand", such as [Sandspiel]. This is +actually a use-case [@Zac8668] is exploring with their [Astratomic] project! For a game like this, rendering +is pretty much 100% custom. You're trying to put all your little sand particles on the screen in a way +that is most efficient for that kind of game. Bevy isn't going to help you much at all with this +kind of rendering, so in this case, a custom rendering backend for bones would be appropriate. This lets +you get rendering efficiency while still hooking into Bones's modding features! + +While Bones emphasises simplicity over the powerhouse rendering features that Bevy offers, it will still let +you stretch your rendering powers where necessary, with a little extra work, and we'll also be trying to +come up with more simple interfaces for other common rendering tasks that aren't supported out-of-the-box +in Bones yet. + +### Multi-World Paradigm + +As we migrated Bones to be a fully fledged framework, and not just a core piece of Jumpy, it became clear +that we were going to need a way to organize the "outside" part of Jumpy that was currently done in Bevy. +This part of the game was responsible for reading raw game inputs and handling the networking pieces +for online play. It "orchestrated" the core game, which was isolated and deterministic, and had no +knowledge of the networking. + +This clear division between the core match, and the larger game and networking was something that +still made sense, but now we needed a way for Bones to be both on the "inside" and the "outside" of this +division. The solution was to make **multi-world** a first-class feature. + +When using almost any ECS there is a concept of the "World" that stores all your data. All your game systems +interact with the data in this world. With our new multi-world setup, you are able to keep a useful isolation +between different parts of your game by putting them in different worlds. In Bones, we called these worlds, +along with their related systems, "Sessions". + +This new pattern of game sessions helped unlock a few really useful patterns. + +#### Game Start and Reset + +It's a common task in a game to show a main menu of sorts, and have a button that +starts the game when clicked. This usually triggers a completely different set of systems and logic than +what is present on the main menu. And when you need to exit back to the main menu, you need to delete all +of the stuff that was used for the gameplay and re-setup the menu. This is actually a pretty annoying and +tricky thing to do when all of your data is in one world, because you can't just delete it all without +deleting the menu, too. + +With sessions, though, this becomes trivial. You simply make one session for the menu, and another session +for the game. When the game starts, you hide the menu session, and create a new game session. When the game +is over, you delete the game session, and show the menu session again. Simple! + +#### Session Runners + +Along with sessions, we have a concept of "session runners". These runners are responsible for running the +session game loop. Having custom runners allows us to create runners for things like fixed updates, to make +the game run with a fixed simulation rate. We can also use runners to encapsulate networking or interpolation +logic, so that you could have one session runner for rollback networking, or maybe another session runner +for client-server networking, or other use-cases. + +#### Game States + +Sessions also turned out useful where we used to use Bevy states to enable and disable different systems. + +For instance, for the pause menu, we used to use game states to control whether or not the game systems +should run and the game should play, or the pause menu systems should play. In this case, we can just +use sessions. Once session for the game and one for the pause menu, and the pause menu can stop and start +the game session based on interaction with the UI it renders. + +Since we added sessions, we haven't found a need for game states again, and the pattern feels nice so far. + +## Finishing the First Pass Jumpy Migration + +Finally we had migrated the core Jumpy gameplay to the Bones framework. It was a _ton_ of work, but I'm +convinced that it was the right direction, and we've gained a lot with the migration. That said, we +haven't finished migrating the whole game. Notably the map editor and network play hasn't been ported +to the latest version of Jumpy yet, but we'll get there! + +First, there were some developments in the wider Rust community that we wanted to take advantage of. 😉 + +## Implementing Scripting + +Recently, deveopment was picked back up on the [Piccolo] project. Piccolo is a [Lua] implementation +written in Rust, and that was something we were interested in. + +In order to get seamless web support, it was important that whatever scripting language we integrated +with Bones be implemented in pure Rust. While it's absolutely possible to us langauges not written +in Rust on native platforms, using those languages the web becomes much more difficult. Lua is also +a very common and popular language for game modding, so being able to support Lua was a big deal for us. + +While it's still under development, we are happy to be some of the first users of Piccolo and [@kyren], +the author, has been very helpful to us as we've gotten started with it. + +Since all of the recent updates to Bones, the Bones ECS was, as far as I could tell, had all of the +features it needed to be able to be accessed from a scripting language, and now it was time to test it +out! + +In reality, there were lots of fixes and updates I needed to make to get bones _actually_ ready, but the +foundation that we had set was holding. Step by step we resolved all of the blockers until finally +we were able to actually implement a Jumpy item with a Lua plugin! See my [blog post][luainjumpy] +from yesterday for more details about that. + +And that catches us up to the present day. + +## Retrospect + +_Phew_, now feels like a good time to catch a breather. Despite taking longer than immagined, so +far things have developed roughly according to plan. Since it's conception, Bones has been planned +to be fundamentally moddable and simple, and it seems that the vision is coming to fruiton. + +I'm going to be glad to get out of the massive refactor and be able to focus on smaller tasks again +that can easily considered "done" and also show some more user-visible improvements. Working on +huge refactors without a lot of visible value-add in the short term can be difficult, but I do +think it was all worth it. I think our attempt at being pragmatic and pursuing our vision has payed off +so far, and will continue to do so. + +While we're setting out to make a game, the Bones engine has grown from the, actual needs of that game, +and it is a means to fulfilling the vision not just of Jumpy, but of the larger ecosystem of Open Source, +moddable games that we want to create. + +I'm personally not one of those developers that just wants to write everything themselves. In fact +I have a history of years of trying to avoid writing a game engine! Bones was not really a part +of my initial plan. But, surprisingly enough, we have needs that only Bones has been able to fill +for us. + +I think that having complete control of our engine is going to be, and already has been, a benefit +for us. We have been able to shape our developer and modder experience into exactly what we want +it to be, and we are able to save ourselves development work by tailoring our engine to our needs. + +It's been a rather enlightening project to work on. This is the first time I've ever written a game +_or_ an engine. I've learned an enormous amount and I know there is still so much more to learn and +I'm excited about that. + +## The Future + +So, what's next? + +The next feature that I want to get done is asset pack loading. We just added scripting, which +is a huge moddability boost, and we already have a tutorial for making Jumpy +skins, but we don't have a way for users to conveniently install custom content! As mentioned above, +this is something that our new asset system was designed for, but we haven't "plugged it in" to +Jumpy yet. This shouldn't be too much work, and the value-add will be great, so I want to get +this done as soon as possible. + +Documentation is the next big TODO, on my List. Jumpy has changed a lot, and Bones has +probably doubled in size at least. Good documentation will help other people contribute and make +Bones more ready for other people to use in their own games. + +Next I want to try to iron out some wrinkles in Jumpy's item system. We don't have events in Bones +yet, which has led to some experimentation with different design patterns that haven't been as clean +or as scripting friendly as I'd like. I'm investigating a simple reactivity system that might help +with this, and might be useful for more UI-like programming patterns. + +Finally, we've got to finish porting some of the things that Jumpy lost in the refactor, like +debugging tools, networking, and the map editor. + +That should get us in a good position to start working on more Jumpy game features. + +[luainjumpy]: @/blog/introducing-lua-scripting-in-jumpy/index.md +[@kyren]: https://github.com/kyren +[Lua]: https://www.lua.org/ +[piccolo]: https://github.com/kyren/piccolo/ +[bones]: https://github.com/fishfolk/bones/ +[`bones_schema`]: https://fishfolk.github.io/bones/rustdoc/bones_schema/index.html +[`bevy_reflect`]: https://docs.rs/bevy_reflect/latest/bevy_reflect/ +[`serde`]: https://serde.rs +[bml2d]: https://github.com/zaycev/bevy-magic-light-2d +[sandspiel]: https://github.com/MaxBittker/sandspiel +[astratomic]: https://github.com/Zac8668/astratomic +[@Zac8668]: https://github.com/Zac8668 +