[1.12.2] Rending entity in GUI + exporting it to PNG?

[Kinniken]

Hi,

 

Could anyone give me pointers on how to render entities for use as a 2D picture in a GUI? I.e. say a way to get a render of a sheep or any other mob to display in a custom GUI? I believe some mods can do this.

 

Bonus question, is there a way to export the render in question to a PNG file?

 

Thanks

[V0idWa1k3r]

3 hours ago, Kinniken said:

a way to get a render of a sheep or any other mob to display in a custom GUI?

Do you mean the same way the player is rendered in their inventory screen? Use GuiInventory#drawEntityOnScreen.

 

3 hours ago, Kinniken said:

is there a way to export the render in question to a PNG file?

A render of what? The entity? The GUI? The screen? If it's the screen then you can do the same thing the game does when taking screenshots. Otherwise you would need to create a framebuffer, bind a texture as a render output to that, bind that framebuffer, render your stuff, get the texture from that framebuffer, read that texture into a BufferedImage and save it to a file. You can see how vanilla does the last 3 actions in ScreenShotHelper.createScreenshot, but everything else you would have to do yourself since the game never does anything similar. Feel free to ask for help though.

Edited October 24, 2018 by V0idWa1k3r

[Kinniken]

30 minutes ago, V0idWa1k3r said:

Do you mean the same way the player is rendered in their inventory screen? Use GuiInventory#drawEntityOnScreen.

 

Oh, lol. I've seen it so often I don't notice it anymore. Indeed seems like what I'm looking forward, thanks.

 

For the export, I meant the render of the entity, on its own. I'll start by working my way through createScreenshot and work from there, good idea.

[V0idWa1k3r]

Okay, I normally do not provide people with ready-to-use code because it is my firm belief that people need to learn how to code themselves, not use ready solutions. However this might be a special case because it requires a descent OpenGL knowledge that you might not be willing to learn because you don't really need it in minecraft modding or even usual java programming. It is also quite difficult to understand out of the box if you've never worked with it before. So I will deviate from my usual ideas and provide you with some code that will be commented so you have an understanding of what's going on and can actually learn something from it.

// Define the width and the height of the framebuffer, the texture and as a result the final png file.
int width = 256;
int height = 256;

// Get the framebuffer object that was already in use since we have to restore the state when we are done
Framebuffer fbo = Minecraft.getMinecraft().getFramebuffer();

// Create a new framebuffer object with the width and the height defined. The last parameter defines whether to use depth or not.
Framebuffer framebuffer = new Framebuffer(width, height, true);

// Bind the created framebuffer as the active framebuffer. The last parameter also adjusts the viewport to the dimensions of our framebuffer.
framebuffer.bindFramebuffer(true);

// These are not really needed, however I prefer do draw over black. By default the texture would be white.
GlStateManager.clearColor(0, 0, 0, 1);

// No need to clear depth/stencil since those are clean as is since nothing has been drawn yet.
GlStateManager.clear(GL11.GL_COLOR_BUFFER_BIT);

// Draw the actual entity. You might want to play with positions and scaling. 
GuiInventory.drawEntityOnScreen(200, 200, 100, 0, 0, new EntitySheep(Minecraft.getMinecraft().world));

// Alternatively if the GL matrix isn't as desired:
/*
GlStateManager.pushMatrix();

// Do matrix manipulations - scaling, translating, rotating.

GuiInventory.drawEntityOnScreen(200, 200, 100, 0, 0, new EntitySheep(Minecraft.getMinecraft().world));
GlStateManager.popMatrix();
*/

// Allocate a buffer for GL to dump pixels into.
IntBuffer pixels = BufferUtils.createIntBuffer(width * height);

// Bind the framebuffer's texture.
GlStateManager.bindTexture(framebuffer.framebufferTexture);

// Dump the pixels onto the IntBuffer. Note that the pixel format is BGRA and the pixel type is 8 bits per color.
GlStateManager.glGetTexImage(GL11.GL_TEXTURE_2D, 0, GL12.GL_BGRA, GL12.GL_UNSIGNED_INT_8_8_8_8_REV, pixels);

// Allocate the array to hold pixel values.
int[] vals = new int[width * height];

// Copy the buffer to the array.
pixels.get(vals);

// Rearrange pixel values to correct positions so they can be read by the BufferedImage correctly.
TextureUtil.processPixelValues(vals, width, height);

// Create the BufferedImage object.
BufferedImage bufferedimage = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);

// Copy the pixels from the array onto the BufferedImage.
bufferedimage.setRGB(0, 0, width, height, vals, 0, width);

// Create a file to store the image within. Here the file will be outputted to the game's base directory with a name of img.png.
File f = new File(Minecraft.getMinecraft().mcDataDir, "img.png");
f.createNewFile();

// Finally write the buffered image into the file.
ImageIO.write(bufferedimage, "png", f);

// Delete the framebuffer from memory. It is no longer needed.
framebuffer.deleteFramebuffer();

// If the game had a buffer bound. In most cases it did but who knows what could be the case with mods and such.
if (fbo != null)
{
    // Restore the original framebuffer. The parameter set to true also restores the viewport.
    fbo.bindFramebuffer(true);
}
else
{
    // If the game didn't have a framebuffer bound we need to restore the default one. It's ID is always 0.
    GL30.glBindFramebuffer(GL30.GL_FRAMEBUFFER, 0);
    
    // We also need to restore the viewport back in this case. 
    GL11.glViewport(0, 0, Minecraft.getMinecraft().displayWidth, Minecraft.getMinecraft().displayHeight);
}

And here is the resulting image:

Note that you must do all this in a thread that is the main render thread!

Edited October 24, 2018 by V0idWa1k3r

[Kinniken]

Thanks, much appreciated! As the author of a fairly large and complicated mod I normally want to understand everything that's going on, but indeed OpenGL is so specific and so "niche" for a mod like mine that I never bothered.

 

I'll try your code, thanks for documenting it so cleanly.

[Kinniken]

That drawEntityOnScreen() is great, exactly what I needed and simple to use too!

 

Bonus question: is there a similar way to render a set of blocks, outside of any world context? Say I have a method that creates a structure made of X blocks, I would like to be able to render it in a UI on a transparent background. Is that doable?

[V0idWa1k3r]

7 minutes ago, Kinniken said:

is there a similar way to render a set of blocks, outside of any world context? Say I have a method that creates a structure made of X blocks, I would like to be able to render it in a UI on a transparent background. Is that doable?

Unfortunately not really. You can use BlockRendererDispatcher#renderBlock, but it is only good for rendering a couple of blocks at a time and it doesn't account for blocks being connected to eachother(so it would render all faces regardless, won't do connected textures, etc.) and it isn't good for performance.

If you want to render a structure of some kind you would need to use a "Fake World". Basically you would create a fake World object, put your blocks into places with their TEs and then render that world as if it was a normal world. I've never worked with that though, so I only know about the general concept.

[Kinniken]

I feared as much. I had thought of the "mock world" idea but it seems like a lot of complexity and probably resource usage to add some pictures in a UI. Ah well, thanks for the info.

[Kinniken]

I just tried your code @V0idWa1k3r, it works fine except that the proportions of the exported entity are wrong (it's "squashed", too thin on the X axis). When displaying it in a GUI it works fine. Any tip? Am I supposed to tweak the horizontal scale myself?

 

Thanks

[V0idWa1k3r]

8 minutes ago, Kinniken said:

the proportions of the exported entity are wrong (it's "squashed", too thin on the X axis). When displaying it in a GUI it works fine. Any tip? Am I supposed to tweak the horizontal scale myself?

Yeah I would assume something is up with the GL matrix at the time of the rendering. Tweak it yourself untill you acheive the desired result.

[Kinniken]

Ok, I'll try that.

 

Bonus question, how do I export them on a transparent background, if that's possible?

[V0idWa1k3r]

19 minutes ago, Kinniken said:

how do I export them on a transparent background, if that's possible?

There are 2 things that need to be changed in the code to save with transparent background. Firstly, you need to set the clear color to be transparent:

GlStateManager.clearColor(0, 0, 0, 0);

The clear color really simply tells GL how to fill all pixels of a texture when the color buffer is cleared. Setting it to having 0 as alpha will make sure the texture is filled with transparent black pixels as it's background.

 

The second and the final thing to change is the format of the BufferedImage created. From

BufferedImage bufferedimage = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);

Which would save every pixel passed to it as 0xFFRRGGBB to

BufferedImage bufferedimage = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);

Which will change the format to 0xAARRGGBB.

[Kinniken]

Thanks, that worked fine.

 

For the stretching issue, it seems like drawEntityOnScreen() "assumes" that the target texture has the same size as the screen, and since in your example it was 256x256 instead, it scaled the output to that. I changed the size settings the following way:

 

final int width = Minecraft.getMinecraft().getFramebuffer().framebufferTextureWidth;

final int height = Minecraft.getMinecraft().getFramebuffer().framebufferTextureHeight;

 

And I get an output whose proportions are fine, but of course whose size matches the screen which is not what I want. I'm guessing I need to adjust the matrix then? Any hint or doc on this?

[V0idWa1k3r]

5 hours ago, Kinniken said:

since in your example it was 256x256 instead, it scaled the output to that.

That's due to the projection matrix being setup for the display. You can change that too - see this topic.