Loading…

Compiling XML to IL – How We Made Modding The Peacenet Way Easier

So in case you are unaware, we are in the middle of porting The Peacenet back to MonoGame. We’re doing this to help increase stability and performance by not lugging around a game engine whose features mostly aren’t needed by the game. While Unreal Engine 4 is an amazing engine with a lot of talent put into it, it isn’t suited for a game like The Peacenet – it’s just too heavy.

One of the features of the Greenlight Engine, the engine we are writing for Peacenet, is its ability to pre-load and cache MonoGame Content Pipeline assets for use later in the game’s runtime. In this article, I’m going to write about how we used this pre-loading feature to make writing Peacegate Programs for the game much easier for everyone.

What is a Peacegate Program?

The Peacenet has an in-game operating system which you play in. It’s called Peacegate OS, and has an XFCE4-like graphical user interface. It’s meant to look like Linux. Throughout the game, you get to use various commands and programs in this OS, both in the Terminal and as windows in the GUI. These GUI-based programs are called Peacegate Programs – and you can write your very own for the game.

How we used to do it in Unreal

In the Unreal days, programs were created by first adding a Widget Blueprint of type Program. Then we would add a Program Data Asset to the game defining the name, description, RAM usage and other values used by the game to describe the program. We would design and script the program’s GUI and behaviour in the Widget Blueprint, and the game would automatically seek out the program’s Data Asset when the game loads.

Doing the same in C#

You might think we’d want to do the same in C#, but we don’t. The problem with using the same approach is we don’t have a visual designer for our GUI system like Unreal does – we would have to write our GUI layout in C# which is annoying. We can do better than that!

Introducing… the Content Pipeline and XML!

As Unreal has the Content Browser and Asset Registry, we have the Content Pipeline and Asset Preloader. Content Pipeline being MonoGame, Asset Preloader being one of Greenlight’s main features.

How can we use these in tandem to avoid writing program GUIs in pure C#? I hear you ask…. Well, we came up with two solutions for it over the past week, which we used to make the Terminal. These approaches wouldd be the following:

  • Write the program mostly in XML, using C# only to respond to input events, and use System.Reflection to build the program’s GUI on the fly.
  • Or… do the same as above but instead of using Reflection, generate and compile C# on the fly to MSIL that can be used as a function to build the program GUI on the fly without using virtually any Reflection in the process.

What the approaches have in common

Both of the above solutions have a few things in common, meaning whichever we went with, the process of actually adding programs to the game is the exact same. The only differences are how the game understands the XML you write, how fast it can understand it, and when/how many times the GUI has to be compiled.

The similarities include, however:

  • Program GUIs as well as the program metadata are both written in XML as a MonoGame content asset and built through the Pipeline Tool.
  • The only C# you have to write is your Event Handler Class, used in much the same way as the code-behind of a XAML user interface like WPF or UWP.
  • The game takes care of turning your XML into a program window.

Problems with Solution 1

The problem with Solution 1 is we can’t pre-compile the GUI of your program as we preload the game’s assets. This means that the game has to compile the GUI every time the program launches, using Reflection.

Reflection, however, is a little bit slow. Moreover, because we don’t actually know at compile-time what controls you are using and what properties/events they support as well as whether they support children and how many… we need to do a ton of sanity-checking on your program’s GUI to make sure you’re not doing something the GUI toolkit doesn’t support, resulting in a game crash. This is also slow, because it requires a heavy amount of Reflection.

You likely won’t notice the performance issues for small, simple GUIs, but for larger ones, it might get in the way of gameplay – so… no good!

Solution 2: Writing a Compiler.

If Reflection is slow, and we don’t want to manually write C# code to build program GUIs, …what the fuck do you do!? That’s a question I asked myself this weekend when requesting help from my good friend reflectronic for help optimizing Peacenet’s program-building code.

Turns out there is something you can do – you can actually write an XML-to-MSIL compiler. But how do you do that!?

The fundamental jobs of the compiler

First we need to understand what we need to do with the program’s XML data. What we need to do is simply:

  • Sanity-check the XML for any errors/things the GUI toolkit doesn’t allow, such as binding to non-existent events, using controls that don’t even exist, etc.
  • Generate C# code from the sane XML, the kind of C# that would be seen in an InitializeComponent method of a Windows Form, to build the GUI.
  • Compile this generated C# to MSIL, allowing us to run it as a function when it’s time to open the program.

In laymen terms, we’re writing code to take the XML program data and use it to automatically generate the repetitive C# we don’t want to write by hand that ultimately got us to using Reflection to avoid doing so, and compiling it to be able to run as any other function in the game’s code!

Benefits of this solution

There are many benefits of this compiler solution that are important for helping improve performance in the game in the long run.

  1. We only need to sanity-check the XML once, to generate the C# code.
  2. Generating the C# only has to be done once, and doesn’t actually result in a GUI being built – only the code required to build one is generated – meaning this can be done during Greenlight pre-loading.
  3. If the code can be generated once, it’s there for as long as the game is alive.
  4. Since the whole point of the code generator is to generate C# code to build a GUI, the generator will do all the reflection stuff – meaning the generated C# code will just be raw C# with NO reflection (well, very little reflection).
  5. Since we’re compiling C# on the fly, to build the GUI later on involves just calling a function that the preloader generates and stores in the built program asset. So performance will be the exact same if you’d just manually wrote that function in the game code.

But where do we even start?

Sanity-checking

Sanity-checking is done during pre-loading as the game generates the program’s C# code. Sanity-checking is the same as what Visual Studio does as you write C#, letting you know if you’re doing something that’ll result in a compiler error.

We’re using Reflection for this, doing the same things that the other GUI builder would have to do every time you open a program, but we’re only doing it once so we know that the code we’re generating WILL compile later on. If we hit an error, ideally the program won’t appear in the game and the engine will let the user know there was an error loading the asset.

So if the asset loads, the XML data was DEFINITELY sane, and thus we never have to check it again. Awesome.

Code generation

This is the juicy part. You may be wondering… How do you… generate and compile code at runtime!? Isn’t polymorphism wonderful?

Well the benefit of C# is that it has the System.Linq.Expressions library. This library allows you to generate an Expression Tree in much the same way as the actual C# compiler does, and compile it to a Delegate that you can run as a function.

So, as we sanity-check your XML, we’re building a C# Expression Tree. While you may think we’re just generating raw C# using something like a StringBuilder, we’re not. This Expression Tree stuff is what reflectronic taught me, and it’s how we both generate and compile the program data to a function.

If sanity-checking succeeds, but you get an asset compilation error during code-generation, we likely have a bug in the code and it’s not the XML’s fault. However, if code generation succeeds, we’re done and all we have to do is cache the compiled Delegate in the built program asset – which, luckily, Greenlight’s simple preloader APIs are really flexible and built exactly for this.

In conclusion…

Yep. We wrote an XML to C# to MSIL compiler and built it into our open-source hacking game. While the use of this compiler is very simple, it has many use-cases for the game and will definitely be used for more than just program GUIs.

Leave a Reply