[Answered, not solved] Automated testing in Forge

[GenElectrovise]

Hi all, I've been wondering recently about how one would go about testing mods (I mean testing in a unit-testing sense here).

Took a look around the MinecraftForge GitHub repo and found the expected src/test/java folder, but most of the tests in here seem to be standalone mods (activated individually with booleans) designed to test a specific behavior.

Some, however, such as the LazyOptionalTest use JUnit. I suppose this is because game behaviours are really hard to mock-up for testing?

What's Forge's testing ethos?

Edited March 7, 2021 by GenElectrovise

[GenElectrovise]

Right then. I've tried some thing out over the last few days and come to the following verdict:

 

Unit-testing Minecraft and Forge classes is really hard. Let me give you an example:

 

For context, I'm using JUnit 5 with Mockito 3.8. 

I have a custom implementation of a fluid tank. I first tried writing a simple unit-test to check that "when there is no fluid in the tank and I try to drain some, drain zero units". I used to extend FluidTank, but in trying to write a simple test, I niow implement IFluidHandler and IFluidTank directly. The reason for this is that creating my object to test, it constructs a FluidTank, which contains this field:

@Nonnull
protected FluidStack fluid = FluidStack.EMPTY;

When FluidStack.EMPTY (a static field) is called on, the JVM tries to also load FluidStack's other static field, FluidStack.CODEC. This in turn causes a cascade of other static initialisations which  culminates in crashing the JVM with a NullPointerException sometime around it trying to either load a World or access a Registry. So not good.

I tried to @InjectMocks on that field, but the injection occurs after the object is instantiated, so the JVM has already crashed.

 

In order to get around this, you have to implement the aforementioned interfaces directly because: 1) They are the required interfaces so they have to be there anyway; 2) They contain no static fields, so they won't start a cascade.

 

The upshot? Forge and Minecraft are not designed to be tested in this way (not surprising), and have interlinking fields with will cause crashes.

What can we do? Well, you can test anything that you write. You can't test a FluidTank, but you can test an IFluidTank. Try to not have unnecessary static fields, and use constructor injection when possible. (This would fix the problem above: see my code below)

 

// How to fix that crash
// In FluidTank.java:

// Don't initialise the field now!
protected FluidStack fluid;

// Add a new constructor to allow modders to specify a starting stack.
public FluidTank(int capacity, Predicate<FluidStack> validator, FluidStack initialFluid)
  // Using constructor injection here means that a tester *can* inject a mock object for the initialFluid. This means that no initialisation 		cascade occurs!
    {
        this.capacity = capacity;
        this.validator = validator;
  
  		// Probably a good idea to test the stack for safety.
  		if (!validator.test(initialFluid)) throw new IllegalStateException("Whatever kind of error you want");
  		this.fluid = initialFluid;
    }

// Changing the current constructor to just use the overloaded one.
public FluidTank(int capacity, Predicate<FluidStack> validator)
    {
  		// This time it's safer to default to EMPTY
  		this(capacity, validator, FluidStack.EMPTY);
    }

 

I realise that this all sounds very... ahhh... something? (can't think of the word) I'm certainly not the expert on this and I'm not a professional tester so I've probably made some errors or assumptions here.

I would put this in as a pull request on the Forge Github if I could actually clone it (never works for some reason...)

 

Best,

GenElectrovise

[diesieben07]

You could probably write a custom JUnit runner to start up the game and run unit tests in FMLCommonSetupEvent or something (similar to how data generation works). but not sure how feasible this is.

[GenElectrovise]

There'd also be a small startup time tradeoff but I don't know how much of a difference that would make. Interesting idea with running tests later on... I hadn't thought of that... Might try that out just to satisfy curiosity!