Jump to content

[1.19] Screen point (x,y) to world coordinates (x,y,z)


Recommended Posts

 Hi there

I am attempting to make a gui where the player will be able to click whatever block is in front of them.

I've tried a few approaches and none of them have worked. (Bare with me as I have absolutely zero knowledge in 3D rendering.)


I've looked online for a bit and found this on stackoverflow using OpenGL.

Here is my 1:1 attempt in Java on my container screen:

    public void render(@NotNull PoseStack pose, int mouseX, int mouseY, float partialTick) {
        // https://stackoverflow.com/a/7702895/13844398
        // STEP 1: get mouse coords. (mouseX, mouseY)
        // STEP 2
        Matrix4f modelviewMatrix = RenderSystem.getModelViewMatrix();
        Matrix4f projectionMatrix = RenderSystem.getProjectionMatrix();

        // STEP 3

        // STEP 4

        // STEP 5
        float x = Mth.clampedMap(mouseX, 0.0F, width, -1.0F, 1.0F);
        float y = Mth.clampedMap(mouseY, 0.0F, height, -1.0F, 1.0F);
        float z = 1.0F; // the depth value ?
        float w = 1.0F;

        Vector4f vector4f = new Vector4f(x, y, z, w);

        // STEP 6

        // STEP 7
        vector4f.mul(1 / vector4f.w());

        // presumably results in a direction vector ??
        Vec3 vec3 = new Vec3(vector4f.x(), vector4f.y(), vector4f.z());

        getMinecraft().font.draw(pose, vec3.toString(), 10.0F, 10.0F, 14737632);

My screen prints something that resembles this: (239.75..., 126.00..., -2003000.125). Which simply looks like my cursor's coordinates (with an extra`z` value) on screen an nothing more.


Am I heading towards the right direction? Is there an easier way?

Edited by than00ber1
added missing link to post
Link to comment
Share on other sites

I have zero knowledge about this stuff either. 🙂 I do know you don't want to modify those internal matrices like you are.

You should at least clone them. But I don't think they are what you want anyway.


Take what is below with a pinch of salt!

I would guess the matrix values you want are those used in RenderTarget._blitToScreen()? 

But even then this isn't going to help you. You don't know the depth of the block. The examples in the link assume you know the value or use z=1 (a point on the far plane).

You probably really need to do some kind of depth test on the buffer at the clicked "pixel". I have no idea how to do that.


Your better bet would to ask on the discord channel of one of the graphics optimisation mods. They will have a lot more knowledge about this stuff.


If you don't post your logs/debug.log we can't help you. For curseforge you need to enable the forge debug.log in its minecraft settings. You should also post your crash report if you have one.

If there is no error in the log file and you don't have a crash report then post the launcher_log.txt from the minecraft folder. Again for curseforge this will be in your curseforge/minecraft/Install

Large files should be posted to a file sharing site like https://gist.github.com  You should also read the support forum sticky post.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Create New...

Important Information

By using this site, you agree to our Terms of Use.