[1.8] NBT written to region file larger?

[OreCruncher]

I pulled the master branch of Forge yesterday and have been working with it.  I noticed that the amount of NBT data that is being written to disk is larger than that of 1.7.10.  I don't have any mods running - it's just Minecraft and Forge.  Is this the new normal, or is there extra information being placed into the NBT stream because I am running debug in Eclipse?

 

(Also, the NBT stream doesn't appear to compress as well, either.)

 

EDIT: How about them NibbleArrays?  Flat out allocation of 2K, and each gets written.  Prior they looked to be a bit more dynamic.

[LexManos]

Things changed in 1.8, also stop being vague.

[OreCruncher]

Vague?  I noticed that NibbleArrays got larger - was wondering if that was the source of the growth.

 

EDIT: In retrospect I guess it was kinda vague.  The reason I am wondering about the size of the NBT is that I spent the past 4 days on 1.7.10 learning the dynamics of the RegionFile system.  I moved over to 1.8 to apply some of my code changes and noticed much larger data sizes.  My first thought was that I fat fingered something in my code, but then I did some digging in the 1.8 NBT and was wondering about the size.  I have no issue if it got larger - things change.  However, I need to get a sense of what reality is so I can view things more objectively rather than with suspicion.

 

[LexManos]

The anvil format has not changed in ages, if you are seeing oddities speak specifics 'NibbleArrasys' means nothing as its used all over the place.

[OreCruncher]

The NBT I am referring to is the NBT generated in AnvilChunkLoader when saving a chunk to disk.  Part of that NBT is comprised of NBTTagByteArray objects created from the NibbleArrays that are part of the chunk data ("Data", "Add", "BlockLight", etc.).  Some of the functionality of the NibbleArray changed between 1.7.10 and 1.8, specifically, in 1.8 it creates a fixed 2K byte array whereas the 1.7.10 was dynamic - allocated what was needed.

 

In 1.8 each of those NibbleArrays attaches 2K worth of data each to the chunk NBT that has to be compressed and stored in the chunk stream within the RegionFile.  My guess is that these 2K buffers are a substantial portion of the size increase I am seeing.

 

To be clear I am not complaining about this.  I am just trying to better understand what is going on and establish the facts of 1.8 vs. 1.7.10 in order to make better choices.

[LexManos]

After digging around trying to figure out what the hell you're talking about.

Yes 'NibbleArray changed from a dynamic buffer size to a fixed 2048 array in 1.8'. Not sure why they did this but this something completely different then the random NBT saving you're talking about. Because, the 'Data' and other tags are always the same size because chunks are always the same size. They probably moved to 2048 to optimize it and not have to have it resize the arrays randomly.

The data itself in 1.7.10 and 1.8 end up being the same format/layout and the same size. In theory, unless something have fucked up.

Things should compress the same as at the end of the day the byte data should be the same.

SO how about this, start throwing down debug data, exactly what sections of the NBT are 'incorrectly' sized. or 'incorrectly' laid out.

 

Anyways if you notice the key part form 1.7.10 is here:

        this.blockLSBArray = new byte[4096];
        this.blockMetadataArray = new NibbleArray(this.blockLSBArray.length, 4);
        this.blocklightArray = new NibbleArray(this.blockLSBArray.length, 4);
        this.skylightArray = new NibbleArray(this.blockLSBArray.length, 4);

The nibble arrays are ALWAYS initialized with sizes of 2048 {the constructor 1/2s them} just like in 1.8.

So ya, nothing 'got larger' post actual debug data/cases and exact specs please.

 

[OreCruncher]

Thanks for looking.  Next time if I will see if I can come up with more concrete information.  Part of the challenge is coming up to speed on something before being able to ask intelligent questions.