[1.11.2] [SOLVED] Energy network performance

[Bektor]

Hi,

 

I've got a problem with the code for my energy network. As soon as I build the system to be larger than a some really small value my FPS

drops significantly from over 100 to 15-24fps.

The size of the network I tested it currently was 11x6.

 

Layer 1: 11x6 solar panels

Layer 2: 11x6 transfer pipes

Layer 3: 11x6 cables

Layer 4: 1x1 transfer pipe

Layer 5: 1x1 machine

 

I don't know what exactly causes the problem, but it's not the machine block, as the energy doesn't even reach this block. 

And from my 66 solar panels, the energy storage of 56 of them is full while the energy storage of the last 10 is empty and stays empty.

 

public class TileEntityPipeTransferEnergy extends TileEntityEnergy {
    
    private boolean flag = false;
    private HashMap<BlockPos, EnumFacing> connected = new HashMap<>(); // without pre-calculated weight
    
    public boolean shouldRecalculate = false;
    
    public TileEntityPipeTransferEnergy() {
        super(10, 10);
        this.container.setTransferMode(EnergyTransfer.PRODUCER);
    }
    
    @Override
    public void update() {
        this.flag = false;
        if(this.shouldRecalculate || this.connected.isEmpty()) {
            this.shouldRecalculate = false;
            this.findTransferPipes();
            this.flag = true;
        }
        
        this.connected.forEach((pos, side) -> {
            final TileEntity tile = this.getWorld().getTileEntity(pos);
            if(tile == null) return;
            
            IEnergyStorage storage = tile.getCapability(CapabilityEnergy.ENERGY, side);
            if(storage == null)
                storage = tile.getCapability(CapabilityEnergy.ENERGY, null);
            
            if(storage != null) {
                storage.receiveEnergy(this.container.extractEnergy((int)(10 / this.connected.size()), false), false);
                this.flag = true; // flag has to be a class member because of inner classes
            }
        });
        
        if(this.flag)
            this.markDirty();
    }
    
    private void findTransferPipes() {
        if(this.connected.size() > 0)
            this.connected.clear();
        
        // we want to store the facing and the position
        Queue<Pair<BlockPos, EnumFacing>> toSearch = new ArrayDeque<>();
        HashSet<BlockPos> scanned = new HashSet<BlockPos>();
        scanned.add(this.getPos());
        
        if(toSearch.isEmpty() || toSearch.peek() == null)
            this.getBlocksToScan(toSearch, scanned, this.getPos());
        
        // temp object because we poll stuff out of the list
        Pair<BlockPos, EnumFacing> pair = null;
        BlockPos current = null;
        EnumFacing face = null;
        
        while(toSearch.peek() != null) {
            pair = toSearch.poll();
            current = pair.getLeft();
            face = pair.getRight();
            scanned.add(current);
            
            if(this.getWorld().getTileEntity(current) instanceof TileEntityPipeEnergy) // check cable connection
                this.getBlocksToScan(toSearch, scanned, current);
            else if(this.getWorld().getTileEntity(current) instanceof TileEntityPipeTransferEnergy &&
                    !this.getPos().equals(current) && !this.connected.containsKey(current))
                this.connected.put(current, face); // found end of line
        }
    }
    
    private void getBlocksToScan(Queue<Pair<BlockPos, EnumFacing>> toSearch, HashSet<BlockPos> scanned, BlockPos pos) {
        // EnumSet.allOf(EnumFacing.class) does not include the null, so machines which don't require a specific
        // site will be ignored
        for(EnumFacing face : EnumFacing.VALUES) {
            // same as pos.north(), just for all directions
            BlockPos offset = pos.offset(face);
            if(!scanned.contains(offset))
                // create a new pair and add it to the list
                // a pair is just some class which holds two fields
                toSearch.add(Pair.of(offset, face.getOpposite()));
        }
    }
    
    @Override
    public void readFromNBT(NBTTagCompound compound) {
        super.readFromNBT(compound);
    	if(compound.hasKey("connection")) {
            NBTTagList list = compound.getTagList("connection", Constants.NBT.TAG_COMPOUND);
            for(int i = 0; i <= list.tagCount(); i++) {
                NBTTagCompound com = list.getCompoundTagAt(i);
                if(com.hasKey("posX") && com.hasKey("posY") && com.hasKey("posZ")
                        && com.hasKey("facing")) {
                    
                    this.connected.put(
                            new BlockPos(
                                    com.getInteger("posX"),
                                    com.getInteger("posY"),
                                    com.getInteger("posZ")), 
                            EnumFacing.byName(com.getString("facing")));
                }
            }
        }
    }
    
    @Override
    public NBTTagCompound writeToNBT(NBTTagCompound compound) {
    	NBTTagList list = new NBTTagList();
        this.connected.forEach((pos, side) -> {
            NBTTagCompound com = new NBTTagCompound();
            com.setInteger("posX", pos.getX());
            com.setInteger("posY", pos.getY());
            com.setInteger("posZ", pos.getZ());
            com.setString("facing", side.getName());
            list.appendTag(com);
        });
        compound.setTag("connection", list);
        
        return super.writeToNBT(compound);
    }
    
    @Override
    public void onDataPacket(NetworkManager net, SPacketUpdateTileEntity pkt) {
        super.onDataPacket(net, pkt);
        this.readFromNBT(pkt.getNbtCompound());
    }
}

 

The code of the cable:

public class TileEntityPipeEnergy extends TileEntity {
	
    private HashSet<BlockPos> connected_machines = new HashSet<>();
    
    public void searchNetwork() {
        if(this.connected_machines.size() > 0)
            this.connected_machines.clear();
        
        ArrayDeque<BlockPos> toSearch = new ArrayDeque<>();
        HashSet<BlockPos> scanned = new HashSet<>();
        
        if(toSearch.isEmpty() || toSearch.peek() == null)
            this.getBlocksToScan(toSearch, scanned, this.getPos());
        
        BlockPos current = null;
        
        while(toSearch.peek() != null) {
            current = toSearch.pop();
            scanned.add(current);
            
            TileEntity tile = this.getWorld().getTileEntity(current);
            if(tile != null) {
                if(tile instanceof TileEntityPipeEnergy)
                    this.getBlocksToScan(toSearch, scanned, current);
                else if(tile instanceof TileEntityPipeTransferEnergy)
                    this.connected_machines.add(current);
            }
        }
    }
    
    private void getBlocksToScan(ArrayDeque<BlockPos> toSearch, HashSet<BlockPos> scanned, BlockPos pos) {
        for(EnumFacing face : EnumSet.allOf(EnumFacing.class)) {
            // same as pos.north(), just for all directions
            BlockPos offset = pos.offset(face);
            if(!scanned.contains(offset))
                toSearch.add(offset);
        }
    }
    
    public void informNetwork() {
        this.connected_machines.forEach(pos -> {
            TileEntity tile = this.getWorld().getTileEntity(pos);
            // make sure the tile entity at 'pos' is really the one we think it is
            if(tile != null && tile instanceof TileEntityPipeTransferEnergy) {
                TileEntityPipeTransferEnergy transfer = (TileEntityPipeTransferEnergy) tile;
                transfer.shouldRecalculate = true;
            }
        });
    }
    
    @Override
    public NBTTagCompound writeToNBT(NBTTagCompound compound) {
        NBTTagList list = new NBTTagList();
        for(BlockPos pos : this.connected_machines) {
            NBTTagCompound com = new NBTTagCompound();
            com.setInteger("posX", pos.getX());
            com.setInteger("posY", pos.getY());
            com.setInteger("posZ", pos.getZ());
            list.appendTag(com);
        }
        compound.setTag("machines", list);
        
        return super.writeToNBT(compound);
    }
    
    @Override
    public void readFromNBT(NBTTagCompound compound) {
        super.readFromNBT(compound);
        
        if(compound.hasKey("machines")) {
            NBTTagList list = compound.getTagList("machines", Constants.NBT.TAG_COMPOUND);
            for(int i = 0; i <= list.tagCount(); i++) {
                NBTTagCompound com = list.getCompoundTagAt(i);
                if(com.hasKey("posX") && com.hasKey("posY") && com.hasKey("posZ")) {
                    
                    this.connected_machines.add(
                            new BlockPos(
                                    com.getInteger("posX"),
                                    com.getInteger("posY"),
                                    com.getInteger("posZ"))
                            );
                }
            }
        }
    }
}

 

The code of my solar panel:

public class TileEntitySolarPanel extends TileEntityEnergy {
    
[...]
    
    @Override
    public void update() {
        super.update();
      
      [...]
        
        this.container.receiveEnergy(this.inputRate, false);
        
        if(this.world.getTileEntity(this.getPos().down()) != null) {
            int received = sendEnergy(this, this.world.getTileEntity(this.getPos().down()), EnumFacing.DOWN, 10);
            if(received > 0) flag = true;
        }
        
        if(flag) {
            this.getWorld().notifyBlockUpdate(this.getPos(), this.getWorld().getBlockState(this.getPos()), this.getWorld().getBlockState(this.getPos()), 3);
            this.markDirty();
        }
    }
  
    public static int sendEnergy(@Nullable TileEntityEnergy from, @Nullable TileEntity sendTo, EnumFacing orientation, int transferAmount) {
        if(from == null || sendTo == null)
            return 0;
        
        EnumFacing side = orientation.getOpposite();

        if(sendTo.hasCapability(CapabilityEnergy.ENERGY, side)) { // Forge Energy Support
            IEnergyStorage storage = sendTo.getCapability(CapabilityEnergy.ENERGY, side);
            if(storage != null && storage.canReceive() && from.container.canExtract())
                return storage.receiveEnergy(from.container.extractEnergy(transferAmount, false), false);
        }
        return 0;
    }
    
    @Override
    public SPacketUpdateTileEntity getUpdatePacket() {
        return new SPacketUpdateTileEntity(this.getPos(), 3, this.getUpdateTag());
    }
    
    @Override
    public NBTTagCompound getUpdateTag() {
        return this.writeToNBT(new NBTTagCompound());
    }
    
    @Override
    public void onDataPacket(NetworkManager net, SPacketUpdateTileEntity pkt) {
        super.onDataPacket(net, pkt);
        this.handleUpdateTag(pkt.getNbtCompound());
    }
}

 

The debug mode of eclipse also showed me the information that in TileEntityPipeTransferEnergy the shouldRecalculate value is set to false and the connected list also contains data.

 

So it must have something to do with the energy transfer from one block to anther, thought I'm not sure what could cause the problem there. 

I just suspect that my solar panels are transfering the energy between themselves instead of putting it into the machine, but I don't see why this would cause

a significantly fps drop.

 

Also to note: for rendering I'm just using a default json file with a texture. So nothing really special about it. The RAM stays with about 1.5GB of 3GB available and the chunk updates are above 200.

 

Thx in advance.

Bektor

 

EDIT: further note: I had no problems before having so many solar panels, so I think its a problem with the cable network

Edited August 15, 2017 by Bektor

[jabelar]

Generally the only thing that can take a lot of time are loops, especially ones like while loops which are waiting for a condition that might not be there. So I would suspect it is your while loop in your cable class.

 

When looking at performance problems like this, it can be useful to printout actual timestamps or use a real profiler to time each part of your code. It is usually pretty easy at that point to determine what is taking the most time and then inspection of that code usually quickly leads to realizing what the problem is.

 

Once you isolate the problem, you need to think about alternative techniques that might be more efficient. For example, imagine if you have a bunch of nested if statements you should order them so the least likely thing is tested first. For example, imagine if you wanted something to happen if the player was holding a certain item at night time -- you could first check for holding the item but that would be wasteful because daytime happens half the time in a game whereas holding a specific item might never happen for some players.

 

Sometimes algorithms just don't scale well. In math there are a lot of cases, particularly in networking (which is basically what your energy network is implementing) that don't scale well. In other words there is usually a max size for networks. In fact I think that is why redstone has limited ability to propagate and why entity pathfinding is limited in total area that it checks. In cases where the overall size of the network is limited by the algorithm, you can still improve performance by reducing the frequency of the update operations. That is why "random ticking" is useful -- you don't need leaves on trees to get updated every tick, but you still want them updated sometimes. So you could do the same with your network. For example, you could randomly update each tile entity. Or you could update cables on a different tick than the solar panels which are on a different tick from the machines. 

 

Overall though, pretty much every network will be hit a limit where performance doesn't scale well after a certain size.

[Bektor]

15 hours ago, jabelar said:

use a real profiler to time each part of your code

As I've never used an profiler before, which one would you suggest to use?

15 hours ago, jabelar said:

printout actual timestamps

I don't think this would work as I've got 66 blocks and even with less than those blocks, it would just be a mess to read the data out.

 

15 hours ago, jabelar said:

For example, you could randomly update each tile entity.

This would not work for my part as my system is build upon the idea that a machine gets every tick its energy, like in other energy mods itself.

But I have a system in place which might work a bit like this and this is the process to build up the energy network as it doesn't run every tick, it just checks if it needs to run every tick and if it

doesn't then it stops there, otherwhise the whole data will be updated, like when a new block is placed and thus the network changed.

15 hours ago, jabelar said:

Overall though, pretty much every network will be hit a limit where performance doesn't scale well after a certain size.

Yeah, that clear, but it shouldn't hit the limit with only 66 blocks.

[Bektor]

Ok, I've now used VisualVM and got the following results:

Spoiler

It might also be noted that after removing everything and building it again(this time larger) I've got 120fps. Well, as long as I didn't placed a block, this got me an fps drop.

 

[Animefan8888]

On 8/11/2017 at 2:51 PM, Bektor said:

FPS drops significantly from over 100 to 15-24fps.

Your FPS shouldn't be consistently low unless you are handling all of this on the client side as well as the server side which is entirely unnecessary.

 

On 8/11/2017 at 2:51 PM, Bektor said:

@Override
    public void update() {
        this.flag = false;
        if(this.shouldRecalculate || this.connected.isEmpty()) {
            this.shouldRecalculate = false;
            this.findTransferPipes();
            this.flag = true;
        }

 

Also calling findTransferPipes() in an update method is rather ridiculous. Adding things to the network could simply be done in the Block.onBlockPlaced and Block.breakBlock, or Block.onNeighborChanged. And instead of storing the network in the pipes/energy tileentities. You could store it in a WorldSavedData and simply reference it when ever you have a World instance. And then you update the network in a world tick event.

 

Not sure if this is the problem, but it could be.

[Bektor]

9 hours ago, Animefan8888 said:

And instead of storing the network in the pipes/energy tileentities. You could store it in a WorldSavedData and simply reference it when ever you have a World instance.

The problem I see with this is when multiply players build up their networks and these networks aren't connected to each other.

 

9 hours ago, Animefan8888 said:

Adding things to the network could simply be done in the Block.onBlockPlaced and Block.breakBlock, or Block.onNeighborChanged

Here we have the problem that when adding blocks simply to the network in those methods: How do you know they are really connected to the network? Who tells you that they are connected to other cables? How do you know they are connected to other cables and those cables are part of the network and not some random cables in the world connected to no network??

 

9 hours ago, Animefan8888 said:

Your FPS shouldn't be consistently low unless you are handling all of this on the client side as well as the server side which is entirely unnecessary.

 

Well, looking at the code, I guess this one if-check got lost in the endless code updates. ^^

 

Edited August 15, 2017 by Bektor

[Animefan8888]

2 hours ago, Bektor said:

The problem I see with this is when multiply players build up their networks and these networks aren't connected to each other.

Simple solution; multiple 'network' instances/multiple lists/maps.

2 hours ago, Bektor said:

Here we have the problem that when adding blocks simply to the network in those methods: How do you know they are really connected to the network? Who tells you that they are connected to other cables? How do you know they are connected to other cables and those cables are part of the network and not some random cables in the world connected to no network??

You would check around for blocks that should be connected to the network and then get their network from them or search for a network with that BlockPos. Of course with this you would also need to check for connecting two networks together.