How many blocks per tick max is recommended for custom structure generation?

[jabelar]

If one has code that generates a lot of blocks (e.g. custom structure) Minecraft will understandably start to lag. And even a "small" structure can have a lot of blocks -- 20 x 20 x 20 = 8000! I can break up the structure into small parts, but how small is recommended? To continue the example, if you need to place 8000 blocks and do it 20 at a time it would take 400 ticks which is 20 seconds -- kinda slow.

 

I guess it also depends on the computer's processing power, and I haven't timed it but I suspect that some Forge versions are faster than others in block set speed.

 

I'm going to play around with it, but just wondering if there is a consensus on what is good limit.

 

One other question, when the console complains about ticks taking too long and then says it skips ticks -- what does that mean exactly? Does it simply add to the tick counter, or is it actually short-cutting other code execution in some way?

[Draco18s]

Uh, worldgen, or some other time?

 

Because generating a whole chunk is 16,384 blocks just to get to sealevel.

[coolAlias]

I think worrying about number of blocks generating is premature optimization, and it all depends on the algorithm(s) you use to do the generation. If you just call setBlock(someBlock) 50,000 times or so, it won't take very long at all, but if you have a complex algorithm to figure out how to place each block, then placing those same blocks will take exponentially longer.

 

I'm not sure what the exact ramifications of the 'ticks taking too long' message, but it happens to me all the time ever since 1.7.x.

[jabelar]

This is not during world-gen. I want structure to appear during gameplay based on an item's use.

 

I think worrying about number of blocks generating is premature optimization, and it all depends on the algorithm(s) you use to do the generation. If you just call setBlock(someBlock) 50,000 times or so, it won't take very long at all, but if you have a complex algorithm to figure out how to place each block, then placing those same blocks will take exponentially longer.

 

I'm not sure what the exact ramifications of the 'ticks taking too long' message, but it happens to me all the time ever since 1.7.x.

 

It's not premature optimization -- I have a simple structure that is freezing the game for many seconds. It is 46 x 30 x 24. It is stored in a string array (name of block) plus an int array (metadata). It simply loops through the dimensions, takes the array and sets the block using Block.getBlockFromName() using the string from the array.

 

I do two passes through because some metadata blocks need to be placed after basic blocks.

 

The just do a few checks (they don't look onerous) in the inner loop is I check for tripwire (just by looking at string from my array) since that needs to be generated after other blocks or it ends up as entity item, I check for dirt (just by looking at string) to allow replacement with a custom block, and I check for lava because I found that if I don't convert all lava to lava source blocks the delay in generation can allow it to flow before structure completes. Lastly I also check if block has a tile entity and create that if necessary (there are only a few in the structure I'm currently trying).

 

Here's the main code for the generation loops:

for (int indY = 0; indY < dimY; indY++)
{
       for (int indX = 0; indX < dimX; indX++)
        {
            for (int indZ = 0; indZ < dimZ; indZ++)
            {
                if (blockMetaArray[indX][indY][indZ]==0) // check for basic block
                {
                    String blockName = blockNameArray[indX][indY][indZ];
                    if (!(blockName.equals("minecraft:tripwire"))) // tripwire/string needs to be placed after other blocks
                    {
                        if (blockName.equals("minecraft:dirt") || blockName.equals("minecraft:grass"))
                        {
                            theWorld.setBlock(startX+indX, startY+indY, startZ+indZ, 
                                    MagicBeans.blockCloud, 0, 2);
                        }
                        else if (!(theWorld.getBlock(startX+indX, startY+indY, startZ+indZ) instanceof BlockMagicBeanStalk))
                        {
                            theWorld.setBlock(startX+indX, startY+indY, startZ+indZ, 
                                    Block.getBlockFromName(blockName), 0, 2);
                        }
                    }
                }
            }
        }
}

for (int indY = 0; indY < dimY; indY++)
{
        for (int indX = 0; indX < dimX; indX++)
        {
            for (int indZ = 0; indZ < dimZ; indZ++)
            {
   
                if (!(blockMetaArray[indX][indY][indZ]==0))
                {
                    Block theBlock = Block.getBlockFromName(blockNameArray[indX][indY][indZ]);
                    int theMetadata = blockMetaArray[indX][indY][indZ];
                    if (theBlock == Blocks.lava) // in Jaden's castle there was issue with lava so making them all sources
                    {
                        theMetadata = 0;
                    }
                    theWorld.setBlock(startX+indX, startY+indY, startZ+indZ, 
                            theBlock, theMetadata, 2);
                    if (theBlock.hasTileEntity(theMetadata))
                    {
                        customizeTileEntity(theBlock, theMetadata, startX+indX, startY+indY, startZ+indZ);
                    }
                }    
            }
        }
    }
}

 

This takes about 50 seconds to generate on a very good gaming computer (Intel i5, 16GB RAM, solild state drive and GTX780 graphics).

 

Is one of the methods I'm using slow? Is the problem that I'm using flag 2 in the set block method -- is this creating unnecessary overhead of trying to sync the client block by block? Should I generate them all without update and then update the chunks after block placement is complete?

 

Or is it because I'm looking up the blocks using strings, is that too slow?

 

 

[perromercenary00]

i have similar trouble but breaking blocks

even breaking a block per tick causes lag

and teh funny thing is that i was only doing worldIn.destroyBlock(pos , false); over an arrayList of Blocks

[Ernio]

If you are looking for opinions - I am pretty sure it's the strings that are the problem.

 

WorldEdit (Bukkit) works the same - if you set (/set or even do noise-random generation) cuboid programmatically it will be very fast (comparable to world-gen, because it uses references), but if you load it from schematic it will take SOME time.

 

You can try using dictionary-like compression. How to do that - depends how you load your data. Idea is that you convert names to internal name-id's. E.g '1' = 'minecraft:stone'.

 

EDIT:

As to block per tick - You can write server-side manager that will be working on ServerTickEvent and will simply go "some" blocks per tick.

[jabelar]

If you are looking for opinions - I am pretty sure it's the strings that are the problem.

 

WorldEdit (Bukkit) works the same - if you set (/set or even do noise-random generation) cuboid programmatically it will be very fast (comparable to world-gen, because it uses references), but if you load it from schematic it will take SOME time.

 

You can try using dictionary-like compression. How to do that - depends how you load your data. Idea is that you convert names to internal name-id's. E.g '1' = 'minecraft:stone'.

 

EDIT:

As to block per tick - You can write server-side manager that will be working on ServerTickEvent and will simply go "some" blocks per tick.

 

Thanks. Yeah, basically I'm working on what you said above. Each tick I'm only doing one y-layer. It doesn't seem to really change overall generation time but it makes the generation seem a bit more managed -- client gets to see it grow up.

 

And like you said, I'm looking at pre-converting the strings to a temporary map. Although I ran a test where instead of looking up any blocks I just placed sandstone. Still took a long time.

[Draco18s]

Question:

 

Why do you loop through the entire structure, placing only the non-meta blocks, then loop through the whole structure a second time for meta blocks?  Why not do both in the same pass?

[jabelar]

Question:

 

Why do you loop through the entire structure, placing only the non-meta blocks, then loop through the whole structure a second time for meta blocks?  Why not do both in the same pass?

 

I find that some meta blocks won't place right if there isn't already the basic blocks. Like placing a torch wants to have a wall already.

 

I actually do a third pass as well for things like tripwire. If you place tripwire before the block under it then it ends up as a entity item not placed block.

 

And if I break up the generation over multiple ticks you especially have to worry about order of placement -- for example placing a lava flow block without a source might drain before you place the source several ticks later.

[Draco18s]

Ah, for specific types.  Ok, then yeah.  I was thinking more like "wool and planks" not "torches."

[coolAlias]

It's much faster to make a list of blocks that need to be placed after the structure is complete during the first loop (storing x/y/z as well), then iterate through that for the second pass. Beats going through the full loop again by a long-shot. I'd do the same for the 3rd pass, though I've never found a need for a third.

[jabelar]

Okay, I made it as fast as I could -- I used actual blocks instead of strings (I converted the array from strings to blocks before I tried the generation), I removed all testing for special cases, and did 11000 blocks in one pass. It still takes about 40 seconds. In fact it didn't really improve on looking up from strings or my original implementation.

 

Here is the console output showing the duration (console statements generally can slow things down so I just output every 500 blocks placed:

[16:30:07] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:297]: Starting to generate with sparse array

[16:30:07] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 0

[16:30:08] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 500

[16:30:09] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 1000

[16:30:11] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 1500

[16:30:15] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 2000

[16:30:17] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 2500

[16:30:20] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 3000

[16:30:24] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 3500

[16:30:26] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 4000

[16:30:27] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 4500

[16:30:27] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 5000

[16:30:27] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 5500

[16:30:27] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 6000

[16:30:27] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 6500

[16:30:28] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 7000

[16:30:28] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 7500

[16:30:28] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 8000

[16:30:28] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 8500

[16:30:29] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 9000

[16:30:29] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 9500

[16:30:29] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 10000

[16:30:30] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 10500

[16:30:31] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:311]: Index = 11000

[16:30:32] [server thread/WARN]: Can't keep up! Did the system time change, or is the server overloaded? Running 25322ms behind, skipping 506 tick(s)

[16:30:32] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:297]: Starting to generate with sparse array

[16:30:32] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.StructureCastleJaden:populateItems:50]: Finished populating items in structure.

[16:30:32] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:297]: Starting to generate with sparse array

[16:30:32] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.StructureCastleJaden:populateEntities:112]: Populating golden goose at 161.0, 140.0, 287.0

[16:30:32] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.StructureCastleJaden:populateEntities:132]: Populating giant at 161.0, 139.0, 297.0

[16:30:32] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.StructureCastleJaden:populateEntities:141]: Finished populating entities in structure.

[16:30:32] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:297]: Starting to generate with sparse array

[16:30:32] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.structures.Structure:generateSparse:332]: Structure setting MagicBeansWorldData hasCastleBeenSpawned to true

[16:30:32] [server thread/INFO] [sTDOUT]: [com.blogspot.jabelarminecraft.magicbeans.ModWorldData:setHasCastleSpawned:54]: World Data setHasCastleSpawned = true

 

My code is simply this:

            for (int index = 0; index < numSparseElements; index++)
            {
                // DEBUG
                if (index % 500 == 0) System.out.println("Index = "+index);
                theWorld.setBlock(startX+theSparseArray[index].posX, startY+theSparseArray[index].posY, startZ+theSparseArray[index].posZ, 
                        theSparseArray[index].theBlock, theSparseArray[index].theMetaData, 2);
            }

 

And I'm holding the block and position in an array of a custom class:

public class StructureSparseArrayElement
{
    public Block theBlock;
    public int theMetaData;
    public int posX;
    public int posY;
    public int posZ;

    public StructureSparseArrayElement(Block parBlock, int parMetaData, int parX, int parY, int parZ)
    {
        theBlock = parBlock;
        theMetaData = parMetaData;
        posX = parX;
        posY = parY;
        posZ = parZ;
    }
}

 

So that should be about as efficient as I can imagine. One dimensional array containing actual block, metadata and position, and just setting that.

 

Anyway, it works well enough I guess. Just one of the charms of Minecraft -- that new block rendering into view is fairly laggy.

[Abastro]

I think the flag 2 might cause the issue.

[coolAlias]

That's strange - when I was working on my structure gen code in 1.6.4, I could easily generate 10-20,000 blocks with tons of metadata blocks (many of them rotated based on the structure), complicated tile entities (chests filled with loot), and loads of other stuff, and it never took more than 3-5 seconds.

 

Note that this was not during world-gen either; structures were generated on right clicking with an item. I haven't played much with that kind of structure gen in 1.7.x and up, but I can't imagine it would have slowed down that much.

 

Have you tried simply generating the structure all in one go, rather than using a tick event to do it? Perhaps forcing it to break up like that is making it worse?

[Tschallacka]

In my mod i'm generating a nether fortress sized structured with multiple hallways etc.. and quite some blocks and it's really fast, and I even load the structure components from json.

 

What I found how crazy as it sounds is that loading everything from a json file and then rendering it would go really fast.

 

Also what helps is that I split up my structure in components like the nether fortress does. Stuff only gets rendered if the chunk is generated. Also it has the added benefit of having specific bounding boxes for mob spawns.

 

Are you generating the entire structure in one pass or have you split it up in multiple pieces? If your structure crosses chunk boundaries and it gets generated when one chunk is loaded and the other is not yet populated by world gen it might end up waiting for the other chunk to be populated before it can continue.

 

What is your algorithm to start it? Do you first register the bounding boxes and then when the world calls for the generation generate the structures? or do you immedeately generate it?

[jabelar]

Are you generating the entire structure in one pass or have you split it up in multiple pieces? If your structure crosses chunk boundaries and it gets generated when one chunk is loaded and the other is not yet populated by world gen it might end up waiting for the other chunk to be populated before it can continue.

 

I started with one pass, then broke it up, and now tried one pass again. My last post above was a one-pass attempt.

 

I think your comment about chunk boundaries is interesting, although I'm generating this in a location that seems to be fully loaded -- basically right beside the player.

 

I'm wondering about the flag 2 for the set block method. If I use flag 0, then can I set the chunks to dirty (or whatever would force an update) right at the end of the generation?

 

Can anyone try simple experiment -- just set 10000 blocks and see if it is as slow as I'm seeing? By the way this is 1.7.10 that I'm testing in.

[jabelar]

I wrapped my generation code in System.currentTimeMillis() and console says that it takes Time to generate = 20261 milliseconds for 11,000 blocks. So about 20 seconds which means an average of about 500 blocks per second.

 

The last question I have is does this mean the server is effectively not doing anything except generate these blocks for 20 seconds? So gameplay for everyone would be handled by the client until server comes back online? And if the server is busy doing generation, could that cause a timeout on dedicated server?

[Abastro]

No, there would be no timeout. As far as I know, Client-Server Connection is handled by independent thread.

[jabelar]

Okay, I found something really interesting. I've been using system milliseconds to measure the execution time.

 

As expected a loop of 11k that does nothing is almost instantaneous -- less than 1 millisecond

A loop of 11k that places a dirt block over and over in same location takes 4 seconds.

 

 

Here is the interesting part, if I generate my structure at Y of 74 it generates in 6 seconds but if I generate at Y of 133 it takes 23 seconds.  The structure I'm generating is a castle in the clouds so I want it to generate at 133.

 

Is there any reason why setting blocks at such a height would be so much slower? I'm thinking that maybe there is some optimization that assumes that blocks won't normally be placed that high so maybe it is slower.

 

[jabelar]

Okay, I've tested generating at a bunch of different Y heights, and it seems to be fairly linear -- the higher you build the structure longer it takes.

 

At height 74, 11k blocks takes 6 seconds

At height 85, 11k blocks takes 8.7 seconds

At height 100, it takes 13 seconds

At height 120, it takes 20 seconds

At height 133, it takes 24 seconds

 

I looked through the setBlock() call hierarchy code and there wasn't anything obvious, except there is a fair bit of light value processing. I should also mention that many of my blocks are translucent. So maybe the higher it is, the more time it takes to process the light values below it?

 

EDIT: After looking at it further, I'm getting convinced that the light value updates are what are causing the delay. There is a method called Chunk#recheckGaps() that I suspect is causing the additional delay. There is also another method Chunk#enqueueRelightChecks() that has a comment that specifcally says it can cause up to 1.6seconds lag per chunk.

 

Does that sound plausible to explain why my structure in the clouds is taking so long to generate?

[Abastro]

Yes, It's due to the light update. I think changing flag parameter of the setBlock would make the light update not done during placing blocks..

[Tschallacka]

Yes, having to recalculate light levels is fairly intensive because per block it polls the blocks around it to calculate.

This is why I do my structure generation during world gen. There is no light updates to worry about sorta speak, at least not that I have noticed. And when I come to my structure in time I still have "black spots" where the server hasnt updated light levels yet. That might be why my structure is fast as opposed to yours.

 

Could you try to put the generation in the worldgen, and see how fast it is during the world generation? Register it as a structure whilst you are at it with proper bounding boxes. then other mods can test for your structure at that location instead of barraging straight through your castle ;-)

[jabelar]

Okay, I'll do some more experimenting with the update flags and world gen. Note that this structure needs to be generated during play so it isn't an actual solution for me to do it during world gen, but it still might be an instructive test.

[Draco18s]

By "during play" do you mean "the player activates an item/places a block/does a thing" or something else?

[jabelar]

By "during play" do you mean "the player activates an item/places a block/does a thing" or something else?

 

Yes. In this case a tile entity checks for a condition and then generates it. And ideally the code is also useful for generation with an item or command.