OreCruncher

This just started happening recently. I have an AT cfg in my project so I occasionally do a clean setupDecompWorkspace. The past couple days the build time is excessive. Normally it would take 10 minutes or so, now it is taking much much longer (1 hour). Also, my Travis build I use for sanity checking is failing because of timeouts and the like. It looks like it is taking an inordinate amount of time resolving dependencies. Are there any additional command line parameters I can pass in to narrow in on what the problem is?

Thanks for the tip. I will take a look and get some ideas.

Before I go too far down a rat hole I figured I would ask some questions. What I am looking to do is render client side effects as blocks. These "phantom" blocks could occur anywhere in the world, x, y, or z. These blocks do not exist in a chunk and are not accessible using the standard world.getBlock()/world.getBlockState(). In some cases they will be generated on the fly based on criteria. State does not have to be persisted between world restarts. These blocks would have no collision box as they are just visual effects. Yada yada yada... So, my questions... How should I go about integrating my phantom block rendering into the Minecraft rendering system? I have Googled examples of what others have done and some of them listen for rendering events in order to do rendering. Seems straight forward if it is the right approach. What I am not sure about is clipping - will my blocks be properly clipped if they happen to be behind other blocks or blocks that have transparency (leaf blocks)? Thanks!

Instead of spawning the entity try adding it to the effect renderer: Minecraft.getMinecraft().effectRenderer.addEffect(effect);

Nice! Much easier than IExtendedEntityProperties. Thanks!

Before I go through and implement my own data tracking is there a piece of data squirreled away somewhere that I can look at during a PlayerLoggedInEvent to determine if this is the players first login?

If you are talking about a .schematic file such as one produced by World Edit, you need some code to handle loot population in the chest and manage the mobs. I built a mod that does just that - it uses schematics to generate village/world structures, ability to generate loot inside chests of that structure, and to properly handle entities that may be saved with it. The code is on github if you want to take a look at how I did it.

I have some rendering code that renders an aurora in the client display. It works fine with regular Vanilla/internal shaders. However, when I use a shader mod like SEUS the aurora does not render. I can toggle it back to internal and things are fine. So my question is whether there is something I can do in my code to make it work with shaders.

Thanks for looking. Next time if I will see if I can come up with more concrete information. Part of the challenge is coming up to speed on something before being able to ask intelligent questions.

The NBT I am referring to is the NBT generated in AnvilChunkLoader when saving a chunk to disk. Part of that NBT is comprised of NBTTagByteArray objects created from the NibbleArrays that are part of the chunk data ("Data", "Add", "BlockLight", etc.). Some of the functionality of the NibbleArray changed between 1.7.10 and 1.8, specifically, in 1.8 it creates a fixed 2K byte array whereas the 1.7.10 was dynamic - allocated what was needed. In 1.8 each of those NibbleArrays attaches 2K worth of data each to the chunk NBT that has to be compressed and stored in the chunk stream within the RegionFile. My guess is that these 2K buffers are a substantial portion of the size increase I am seeing. To be clear I am not complaining about this. I am just trying to better understand what is going on and establish the facts of 1.8 vs. 1.7.10 in order to make better choices.

Vague? I noticed that NibbleArrays got larger - was wondering if that was the source of the growth. EDIT: In retrospect I guess it was kinda vague. The reason I am wondering about the size of the NBT is that I spent the past 4 days on 1.7.10 learning the dynamics of the RegionFile system. I moved over to 1.8 to apply some of my code changes and noticed much larger data sizes. My first thought was that I fat fingered something in my code, but then I did some digging in the 1.8 NBT and was wondering about the size. I have no issue if it got larger - things change. However, I need to get a sense of what reality is so I can view things more objectively rather than with suspicion.

I pulled the master branch of Forge yesterday and have been working with it. I noticed that the amount of NBT data that is being written to disk is larger than that of 1.7.10. I don't have any mods running - it's just Minecraft and Forge. Is this the new normal, or is there extra information being placed into the NBT stream because I am running debug in Eclipse? (Also, the NBT stream doesn't appear to compress as well, either.) EDIT: How about them NibbleArrays? Flat out allocation of 2K, and each gets written. Prior they looked to be a bit more dynamic.

Threading is indeed fun. It's one of the aspects of development I enjoy. Also like databases, but I don't say that too loudly. Databases are like threads in some ways.

I have a further question related to your statement about the time between a given chunks saves and desyncs. What I am wondering is if multiple IO threads were indeed operating it would be possible that two different IO threads could wind up with two different states for a given chunk at the same time assuming the writes to disks were horribly tardy. Would be safe to put this situation in the same category as the NBT buffer sharing (i.e. stop worrying about it and buy lottery tickets because the chance of hitting the jackpot are greater than running into this race condition)?

No, I don't. Found the condition via code inspection. If this problem were to occur it could manifest in some subtle ways so it would be hard to spot. Other thing is that the saved data could work itself out - a subsequent save would be consistent. I would have to wonder, though, about a read of the NBT while the chunk data is waiting to be written (cache hit). Not sure of all the conditions that could occur - I just know that it does happen. Yes, the performance impact would be significant. Not only do the buffers up in AnvilChunkLoader have to be duplicated, the NBT tree would have to be crawled looking for other buffer type tags and do the duplicate as well. Reason is that the tags for modded items aren't aware of what is happening down in IO land. I will have to rely on your experience about the seriousness of the issue. As you point out the conditions would be rare, and if it does occur the impact wouldn't be destabilizing. EDIT: Did a quick wiff test as to the performance impact. What I did was introduce a "freeze" method in the NBT class hierarchy that will cause a tag to duplicate it's buffer when invoked, and is smart enough to figure out when it doesn't have to do it (i.e. NBTTagList of NBTTagInts don't need to be frozen since the ints are immutable at this stage). The allocated NBT structure will traverse the tree as needed to perform this deed. With the freeze, writeChunkToNBT() took on average 35ms, and without the free it took about 25ms to complete. The freeze process takes 40% more time.

If that is a bother you can use the use program arguments in the debug configuration to specify a user account: --username=<minecraft username> --password=<minecraft password> Also, just curious as to why it is the player name specifically that you are concerned with. In general, if you want to track a player use getPersistentID() - the value will be the same for a player account even if they rename the character.

Do I need to open a ticket or will it be mystically handled by the powers of the Forge devs?

I would assume the same would apply to the NBTTagIntArray as well. Thanks for the response. BTW, does Forge modify AnvilChunkLoader, or does that come straight from Vanilla?

I was looking at the writeChunkToNBT() code in order to understand how the save data is generated. I noticed that a bunch of byte arrays are stuffed into the NBT save object. Looking underneath the NBT code, all it will do is wrap the byte buffer without copying. The net result is that the byte buffer is now shared between the NBT "persisted" state and the Chunk object. If that Chunk object updates the byte array before that NBT gets written to disk it will magically update in the NBT state which will no longer make that state consistent, and the results recorded into the RegionFile cannot be reliable. Is this something to worry about? Or is the shared state an OK thing? An underlying question I have is whether an NBTByteArray object can be considered immutable, like a String.

I'm doing it for several reasons: Currently unemployed and going stir crazy. I have played on servers that have had bouts of lag for inexplicable reasons, and I decided to take a look at how Minecraft works from the ground up to understand what may be happening. As I dig up from the bottom I am making tweaks that I believe reduce contention with threading and memory management and improve on file access. If I can come up with justifiable set of improvements I am looking at folding the logic into some of my mods. The method I am using of replacing the Minecraft classes has me a bit nervous because I am intercepting the class load and replacing the entire class with my own implementation. I have never done this before and I am just waiting to run afoul of the law of unintended consequences.

OK - I been poking a stick at the underlying mechanics of RegionFile and RegionFileCache. In my local sandbox I have a set of replacement classes which were a bear trying to get into the Minecraft runtime using class transformers. :\ Anyways, based on what I have seen I think that chunk write performance can be enhanced by changing the init parameters to the Deflator. The measurements I took weren't 100% scientific (i.e. get the nano time between when the ChunkBuffer is given to the upper level write routine until after the write completes within ChunkBuffer), and the ChunkBuffer I used was a replacement I came up with in order to get some additional streamlining. With the Vanilla Minecraft defaults for the Deflater (Deflater.DEFAULT_COMPRESSION, Deflater.DEFAULT_STRATEGY, buffer of 512 bytes) a write of a chunk stream to the region file took about 2.5 seconds, and wrote on average 2.3K of information. Keep in mind that the amount of time taken is inclusive of the compression of chunk NBT as well as the write to disk. With new parameters I have been working on (compression level of 3, Deflater.FILTERED, 4K buffer size) a write of a chunk stream to the region file took about 1.5 seconds and wrote on average 3K of information. My observations: My tweaked compression settings resulted in a 66% increase write performance. My tweaked compression settings resulted in a 30% increase in data size. Though the amount of information written is larger, the minimum allocation size within the region file is a single 4K sector. If the region had a bunch of modded items in a chunk I would expect the chunk stream to be larger with the tweaked compression settings than what Vanilla would do. Disclaimers: I have not looked in detail at the upper level Minecraft routines to see what they are actually doing. My tweaks and changes are based on my observations of behavior at the lower levels and how they manifest. As such the upper level routines could be taking 5 minutes to do something which makes the gain of 66% at the low level where I am looking kinda pointless. The development system I am using is high end Windows 10 i7 system. Not sure how these changes will play on other OS platforms or other less capable systems. The code I used has been heavily modified from the originals. My intent is to streamline operations and reduce impacts of buffer allocation, information caching, etc. If you want to take a look at the code I am working with you can find it here. This code is highly experimental and should be more of a proof of concept than anything production ready.

This isn't a specific modding question, rather, I am trying to understand the foundation of what Minecraft is doing. During my travels I came across RegionFileCache. I see that createOrLoadRegionFile() will "purge" the internal cache once it hits a total of 256 entries. This is kinda strange to me for several reasons: * I would have expected some sort of LRU mechanism, not a dump of the entire cache. * On a modded server with players travelling around and loading up regions in separate worlds I would imagine that this cache can reach full pretty quick. Given these two it seems the cache would thrash which in turn could cause ripples in performance. I was wondering from those in the know if my concerns are valid, and if not how should I be looking at this. Thanks! EDIT: For clarity, I am looking at the decompiled forgebin JAR file.

Excellent! On the surface this is exactly what I am looking for. Now I have my project for the day.

Yeah. My understanding, though, that those are for custom packs that my mod generates and send. Of course, there may be some trick to these guys where I can get vanilla packets.

Before I resort to rocket science I figured I would ask the question in hopes that I have overlooked something. What I am wanting to do is intercept incoming packets from clients before they get handed off to FML/Vanilla packet processing. Is there a framework in place that allows me to register an interceptor so that my mod can get a crack at the packet before/after processing? The specific thing I am wanting to do is intercept incoming sign text update packets so that I can do some pre/post processing of the event. What I am hoping for is clean server side entry point to do this without resorting to custom signs/tile entities, ASM, or some other fancy hackery. The mod I am working on should not be required on a client. (I did look for a Forge event for sign update, but I did not see any.)