EDIT: I switched to using scissor test.

I am trying to set up stencil buffer to cut everything that gets drawn outside of a specific rectangle in the gui. Instead, everything that is drawn after the stencil buffer is set up gets cut;

Here is the stencil set up code:

public static void setUpStencil(int left, int top, int right, int bottom) {
	if (left < right) {
		int i = left;
		left = right;
		right = i;
	}

	if (top < bottom) {
		int j = top;
		top = bottom;
		bottom = j;
	}
	Tessellator tessellator = Tessellator.getInstance();
	VertexBuffer vertexbuffer = tessellator.getBuffer();
	GL11.glEnable(GL11.GL_STENCIL_TEST);
	GL11.glStencilFunc(GL11.GL_ALWAYS, 1, 0xFF);
	GL11.glStencilOp(GL11.GL_KEEP, GL11.GL_KEEP, GL11.GL_REPLACE);
	GL11.glStencilMask(0xFF);
	GL11.glDepthMask(false);
	GL11.glClear(GL11.GL_STENCIL_BUFFER_BIT);
	vertexbuffer.begin(GL11.GL_QUADS, DefaultVertexFormats.POSITION);
	vertexbuffer.pos(left, bottom, 0.0D).endVertex();
	vertexbuffer.pos(right, bottom, 0.0D).endVertex();
	vertexbuffer.pos(right, top, 0.0D).endVertex();
	vertexbuffer.pos(left, top, 0.0D).endVertex();
	tessellator.draw();
	GL11.glStencilFunc(GL11.GL_EQUAL, 1, 0xFF); 
	GL11.glStencilMask(0x00);
	GL11.glDepthMask(true);
}

Here is the actual drawing code:

drawDefaultBackground();
setUpStencil(20, 20, width - 20, height - 20);
drawRect(0, 0, 50, 65); //Some drawRect calls here

 

Edited July 1, 20178 yr by Lucius Q. User

You would have to reset the stencil buffer.

Sorry, what? I have just set it up and it cuts everything drawn while it is active.

Make sure that you enabled (created) stencil buffer. By default, MC does not create one.

Alternatively, because you're in 2D, use scissors.

Edited July 1, 20178 yr by Elix_x
Added link to stencil buffer creation with Forge

I did enable it. The problem is not that it is not cutting anything, but rather that it cuts too much.

 

EDIT:

The problem is as follows:

drawRect(...)// <- gets drawn

setUpStencil(...)

drawRect(...) // <- gets cut entirely (should get cut only partially)

disableStencil()

drawRect(...) // <- gets drawn

Edited July 1, 20178 yr by Lucius Q. User

When stencils don't work for me right away, i usually enable the color write and draw the stencil'ed region with colors, then i go into debug mode and play with it until i get it to work.