enu

In my first post I mentioned that I’m the dev-ops guy. But what does that even mean? Most of the playerbase likely won’t be familiar with it or only have a vague idea. So in this post I’d like to give you some insight into what I’m doing for the team. Part of it is, of course, Eevee which I introduced last time. Another part of my work is about automating boring repetitive stuff like releases which need to be done very carefully and are very easy to mess up. Aside from actual game scripts, maps and graphics there are a lot of other parts that need to work correctly. Typically, compilation for all four platforms including all the libraries and binaries the game is using internally. The requirements have increased dramatically because of some of our new features and it would be next to impossible to maintain all of that manually across three games and four platforms. Disclaimer: Some of this will be very technical but if you’re curious what developing these games involves behind the scenes it can be interesting nonetheless. It could also be good for other game authors and modders to see how automation can help them. Patches After the initial version of eevee, the next thing on my hit list was the release process. I wanted to automate the process of detecting all files that were changed, zip it and upload the zip somewhere for the players to download. The first part was this Git command: git diff --name-only --diff-filter=ACMRTUX <commit>..HEAD Git knows the entire history of the codebase so you can get some interesting data from it. In this case the command above gives you a list of all the files that were changed since the given commit. Normally I’d feed the result to another command that would zip all those files– however, with eevee involved it’s not as easy because Git would give you the yaml / ruby files instead of the rxdata files that the players need. So, I wrote the logic to generate a patch into eevee directly. It uses this command internally but then detects which files need to be replaced by rxdata files in the patch. GitHub Actions Even when you have the patch zip you still need to upload it somewhere. And while we’re at it why would you even make the zip yourself even when it’s faster using the command? Software developers automate all of it these days. GitHub Actions can do exactly that so I wrote a simple workflow that would trigger any time Ame tagged a new release in Git, use eevee to generate a patch and immediately upload it to her server where the players can get it. With that making a new patch was just a few clicks, Ame could focus more on e19 development and easily release several patches per hour if she felt like it. I think we first used this for Reborn’s e19 public release. Since then the workflow has evolved a lot. Desolation e6 and Rejuvenation v13.5 were also using some newer versions of this and I made even more changes for the upcoming version of Reborn. Of course, now it’s mostly me and the new team actually triggering the workflow. mkxp-z With eevee and my release workflow being adopted by Rejuv and Deso I suddenly had Git access to not just Reborn but all three games. So… time for more fun, right? The workflow I mentioned above could only handle the small incremental patches; the main releases of Reborn e19 and Desolation e6 were still handled manually. Changing that would be a much bigger fish to catch– more like a whale, in fact. The main complication is that the games are not Windows-only but also have macOS and Linux releases. It wasn’t enough to just remove the dev-specific files and zip the rest because the repository didn’t contain the runtimes for the other platforms. The engine the games are using for runtime is called mkxp-z. It can be built for all three platforms and compiling it is not an easy process. Particularly because to compile it for a particular platform you need to compile it on that platform. And I don’t have a mac. There are pre-compiled versions of it available but to complicate things further the games are using a modified version of mkxp-z so that they could have a custom game icon and increased frame rate cap. If only there was a way to compile this automatically with our modifications, right? Well, guess what, there is. The maintainers of mkxp-z have their own GitHub Actions workflows to compile that thing for all platforms, so I didn’t need my own mac. I just forked it, applied the tweaks we needed and soon I was able to get a compiled version for Rejuv. It took several tries and each compilation takes over an hour because mkxp-z is a beast, but it worked. Afterwards I could set up another workflow to combine the compiled mkxp-z with the game files. A couple test versions later and we could fully automate Rejuvenation’s main release 13.5.0 for all platforms; no need for Jan, Cass or anyone else to painstakingly prepare it on their own. I won’t share the workflows here as I’m still working on some details for the community release. But there is no harm in sharing these so in case you want to set up some automation like this for your own mods or games and you feel getting your hand on our workflows would help, hit me on Discord. Engine tweaks This ended up opening several more possibilities in the future. For instance, I was able to adjust the default controls of the game which are baked into mkxp-z’s source code itself. Next I was able to make it so that the engine contained some extra libraries which we needed for some other new features– more on those in the next post! Then there was the very controversial topic of input repeat– how often an action is repeated while holding down a key. This has a big impact on how fast you can scroll through Bag and debug menus. The mkxp-z defaults (which were used in Rejuvenation 13.5) are too slow, while Reborn e19 had it adjusted for debug convenience which was slightly too fast. In our mkxp-z fork I was able to change these settings to be configurable in the mkxp.json file. So, while the default input repeat is ultimately set somewhere in between the normal defaults and Reborn e19 speed, you can now adjust it to your liking. And finally it means we can fix some bugs in the engine. It’s written in C++ which I’m not familiar with so my options are limited but I can tweak a line or two when we find a problem. Segmentation Fault hunt First I should clarify what a segmentation fault even is. I’m no expert on this topic, but from my understanding a segfault happens when a program tries to access an area of memory it doesn't have access to, causing the operating system to kill the program. In general, there is next to no way to recover from these errors because the system just kills the program with no chance to react. In general, these happen only when incorrectly using some low-level things such as pointers in C. Higher level programming languages such as Ruby, Javascript, PHP, Go and so on generally won’t let you mess up like that, and if it does happen anyway it usually means there is a bug in the language itself. Well guess what? We had a segmentation fault in Reborn during 19.5 alpha. You know how the game sometimes shows a popup with an error message? Those are our errors in the Ruby code and generally not a big issue. Segfaults instead just end the game: no popup, no message in the log, nothing. This signifies a bug in the game engine itself - mkxp-z. To make things worse, we didn’t have a reliable way to reproduce the error, it just sometimes happened during or after a battle with no consistency. Sometimes you could play for a week without getting a crash, other times you could get it three times in a day. (Like I did! -editor Orsan) There are, of course, some ways to debug these cases. I consulted the mkxp-z devs about this issue and was able to make a mkxp-z build with debug symbols. Debug symbols are basically a mapping between the compiled program and the original source code, so if a segfault happens the GNU Debugger (gdb) can be used to process the core dump and find the lines in the original code where the problem occurred. It took a few more hours but I did manage to get a gdb backtrace of the issue. With that additional info we eventually discovered that someone else from the mkxp-z community already had a fix for the problem (which had to do with certain move animations accessing invalid files). Without the backtrace, there was no way to know what to look for, so we wouldn’t have found the connection between the issue and the available patch. The game has been stable ever since, so it was well worth it. And, because I already had all the infrastructure around our own mkxp-z fork already developed, I was able to simply merge the patch into the fork and use the fixed mkxp-z in the 19.5 beta release for Ame’s patrons. That’s all for today! Sorry if my posts are more technically oriented. Next time I’ll finally get into some improvements for the players! If you have some questions feel free to ask in the comments or Discord DMs.

“The dev blog was a bit desolate, but now we've rejuvenated it to let y'all know about reborn Reborn.” ~ Stardust Time for another blog post! Today I’d like to introduce a development tool which we have been using internally for almost two years. Reborn was my guinea pig here but both Rejuvenation and Desolation are using it as well these days. Big thanks to Ame that she was willing to try this out on her game even though I was just an alpha tester at the time and the e19 release period was very busy! This one is meant especially for those of you who maintain large mods that include map changes and creators of their own games. Several features we’ll be introducing in the 19.5 update and more features that we’re planning for later would not have been possible without this tool. The problem In my previous post I mentioned that the team is using a version control system called Git. It’s an essential tool for any kind of software development. It lets you keep track of every change that was ever done to the game with the possibility to go back and see the changes, merge together the work of several teammates, and so on. One notable thing about Git is that it works well with text files but not with binary files. Such files are basically computer gibberish which can mean anything and therefore is unreadable for Git. Think images, audio, video, executable and so on. The issue with RPG Maker XP (the editor which is used to create these games) is that it stores maps and some other things in a binary format. You know, the .rxdata files in the Data directory of the game? Yeah, those. I didn’t like that so after some research I found a super old tool capable of decoding them. So I forked that, made it more useful for actual development and called it the Easy Essentials VErsioning Engine, aka Eevee. Eevee basics Now what can this little fella do for you? Well it can transform .rxdata files to a text format and vice-versa. Git is very good at versioning text: it can track when each line was changed, who did it, let you merge changes from two people even in a single file and so on. So for Git to be able to work with maps it was very important to have the maps in a text format. For the team this is very important as it allows us to have several long-term projects that need a lot of map changes in separate git branches and we can easily sync them with whatever changes we make in the main branch. Synchronizing that without Eevee would be a nightmare and very prone to accidentally undoing someone’s work. But this way we can easily keep everything up to date internally as a part of our routine and those long-term projects can spend as much time in the oven as they need without causing any problems. Another no less important advantage is that text files are human-readable and search-able. Wanna know all the places where you can obtain a particular item? Find all events where a mon is removed because all of those were bugged? (Yes they are all bugged in Reborn 19.0.16.) We can do that! The format Now originally the format eevee produced was simply YAML. Which was okay but I didn’t have full control over the output so it still contained a lot of extra gibberish. However, near the end of last year I finally sat down and implemented a custom format for Eevee which makes it even more useful. It allowed me to make many custom improvements to the format for easier usage - making sure that important data is there in a readable form while useless and redundant data from the old yamls is simply cut. For example all the options that can have one of a limited number of options in RPG maker are transformed from their internal number representation into text here for much better readability. To give you an idea what a map looks like in this format, here is Lapis Gym in the new format (scroll down a bit, the events at the top aren’t very interesting). You can see all the dialogue and all other data the map contains. We can also easily find all the places where a particular switch or variable is used or where a particular audio track is played. And finally, it allows doing some bulk changes by simply string replacing across the entire project. For instance, some typos were in many places all throughout the game. There are three other things worth pointing out in the example file. Firstly, the comment at the top of the file shows you where to find the map in question in RPG Maker. Secondly, unlike the rxdata files, the filename also contains the name of the map. Other features Eevee is also able to run in the background while you’re working in RPG Maker and output your changes to ruby files on the fly. There are more features which you can read about in Eevee’s documentation if the tool looks useful to you! We use it to generate patches and validate our maps so that they don’t contain common errors. There is also a guide to help update your modded maps to the new version of the game. It’s not an easy process but for large mods it should be a lot faster and cause less issues than redoing all the work manually. The source-code is entirely open on GitHub so you can see how it works internally and help improve it if you find a problem.

Hi, Community! This might come as a bit of a surprise. Almost two years since e19, is it? Can't blame you if you thought the game was done. Admittedly there was a hiatus for a while and we have been a bit quiet about the recent revival, but the game is very much alive now. In fact, we're planning a new release in the not too distant future. Wait, who is this random person? Right, I never wrote here before... Well, hi! Enu here. I'm the dev-ops guy. While my main focus is Reborn, most of my work benefits Rejuvenation and Desolation as well. I created some tools that make the development and release process easier for the teams and later I also worked on improving some aspects of the player experience. I first found Reborn around episode 14 and it quickly became my favorite Poké game. Occasionally I tried to look into some bugs that were really annoying - the veterans among you might remember them. The long text lines where the first part disappeared before you could read it? Or funky NPC follower movement, especially around ledges? I helped out with those. Also I know Git quite well (it’s a version control system, a great thing for software development) so when Ame's team started to use it I gave her an introduction about what it can do. Later when she started talking about episode 19 alpha on Patreon I half jokingly asked where to apply. I didn’t really expect to get in but thanks to those past involvements Ame actually did invite me into the alpha testing group. What happened afterwards was that I stayed active even after alpha testing concluded, wrote some tools for the team and kept working on fixing bugs even after the e19 release cycle finished. In the meantime most of the original Reborn team moved on to other projects though a few of them still make a change or two occasionally. Desolation and Rejuvenation teams were working hard on their respective updates. I helped with updating the games to take advantage of the new tools we had available. And eventually after these updates were released I ended up building a new team for Reborn together with Ame. So what is this update about? First to avoid too-high expectations, this is not a story update. There are only a few dialogue changes overall - such as new NPCs mentioning new passwords. There is no big graphical overhaul, no new gen or something either. The game still looks and plays the same. Our main focus was the engine. Modders might have noticed that latest versions of Desolation and Rejuvenation have a very different code base than Reborn e19 had. It was mostly Cass and the Reju / Deso teams who did the engine overhaul. This update gets Reborn running on this new engine as well. The main benefit is that the engine is shared between the three games which means that future bug fixes and even some features can easily be applied to all three games. In fact many of the new features coming to Reborn will appear in the other games as well in their next update! This way we'll all have a good base for future updates and benefit from the hard work of the other teams. The new engine should also make the creation of new mods easier than ever. For players, this update brings many long-requested QoL features, a couple of new passwords, improved controller support, accessibility improvements and an official (if experimental) JoiPlay version to be played on Android among other things. More about these in upcoming blog posts! We also fixed hundreds of bugs - both in overworld and in battle. Small clarification about version numbers. Reborn e19 had versions 19.00 to 19.16. However, they should have been 19.0.0 to 19.0.16. The convention was just incorrect back then. The new update will be version 19.5.0. Introducing new team Amethyst No introduction needed! As you all know Amethyst mostly moved on to work on Starlight. So while setting up the new team my main goal was to do it in a way that the team is capable of working without Amethyst keeping an eye on all the details. She makes some changes herself every now and then and we make sure to ask her about priorities, what features she wants included and other decision making. This way we can keep making the game better but still very much in line with her vision and wishes. Other than that her hands are free to work on what she wants and not lose time on minor details that don't necessarily need her attention. Crim A veteran from the original team! Many things from e19 and before are their handiwork and we’re very lucky to still have them on the team. Crim has the skills to do just about anything the game needs - graphics, animations, mapping, story writing and scripting. They’re looking into tweaking many of these aspects in the future and we’re very excited to see it. stardust A very experienced Reborn challenge runner! She’ll often take note of random battle bugs either from her own runs or community conversations, then go and fix them before we knew the bug even existed! Other than that she's also the author of the NG++ mod for e19. This mod will not be updated for 19.5.0 as it was merged into the main game, along with a number of new skips and improvements. Lucent Flash Lucent is constantly proving his expertise at analyzing... well frankly anything I throw at him. He fixed many random issues and we can rely on him for some larger projects as well. For instance the new engine completely broke many Reborn-specific features such as all Online stuff. Lucent fixed all of that and additionally made it possible for these features to work with the other games and even with large mods that add newer generations or fakemons. Orsan I'm always amazed about the amount of random trivia that Orsan knows about the franchise. It often helps to have him validate how things should work. A large part of the upcoming accessibility improvements are his work, not to mention many other random fixes. Together with Lucent he also added Battle Pavilion bosses for double battles. He’s also been putting a lot of work into ████████████. Haru They're the person behind Randomizer which is also greatly improved in this upcoming release. I don't know all the details but the Randomizer improvements will be revealed when the time comes! Haru also did the hard work of updating the Battle Pavillion and Battle Factory in the Nightclub to the new version of the engine. This alone was a LOT of work. And last but not least, several important engine refactorings are Haru's work as well. Pyrolusite Errr... this is probably a dead give away to what we're planning, isn't it? Can we announce this one early, Ame? Yes, Pyro is the author of the very popular UI mod Pyrolusitium. Yes, he's on the team to improve the UI and we’re very much looking forward to it. Unfortunately, no, it isn't ready just yet and won't be included in this update. It will become part of the game later. ...Oh and myself. I maintain the dev tools, and introduce various initiatives to improve our work. You'll hear more about some of my work in upcoming posts! Some of the old devs also occasionally float in and out to help out with something and we can consult with them when we need to. I don't have an exact date for when this release will go public. There will be a community testing version first because we're still running into random quests being broken because of the engine update every now and then. But overall the game is quite stable now. Thanks for reading and stay tuned for more updates!