[1.14.4]Simulate mouse and keyboard input

[ATomo9_CY 0]

Im trying to write a neural network mod that produces its own game input to play the game.

I cant figure out how to make the game use the input i produce in my mod and ignore mouse and keyboard input so i can run multiple clients and be able to do other things while it plays.

[cookiedragon234 6]

KeyBinding.setKeyBindState(mc.gameSettings.keyBind[whatever].getKeyCode(), true);

 

[ATomo9_CY 0]

24 minutes ago, cookiedragon234 said:

KeyBinding.setKeyBindState(mc.gameSettings.keyBind[whatever].getKeyCode(), true);

 

Is this only for keyboard input?Also if i execute this every tick will user input be ignored?

[cookiedragon234 6]

@ATomo9_CY yes, it will override the user's input. Im not 100% sure about mouse input, but you can use stuff like mc.player.swingArm() for that

[ATomo9_CY 0]

That line of code works very well for simulating keyboard input.

I tried using it with keyBindUseItem and keyBindAttack to try to simulate mouse clicks but it didnt work.I used mc.player.swingArm(mc.player.getActiveHand()); but the game crashes because getActiveHand() returns null.There is also the problem of moving the players camera around that i haven't figured out how to do yet.

Edit:keyBindUseItem and keyBindAttack actually do work but for some reason i have to click first once manually before they start to effect the game.

Edited December 28, 2019 by ATomo9_CY

[ATomo9_CY 0]

@SubscribeEvent
public static void EntityViewRenderEvent(CameraSetup event){
    event.setYaw(yaw);
    event.setPitch(pitch);
    event.setRoll(roll);
}

 

The above code takes yaw pitch and roll as inputs to determine the position of the camera.

i think ill still need some code to determine how the user will interact with guis since it has to be done without the mouse pointer.Ill be trying to do that but if anybody has some info that could help me please do share it.

 

Edit:The code above it seems is not very usefull for what im trying to do since it just moves the camera independendly of mouse input while everything else functions based on user mouse input.

Edited December 28, 2019 by ATomo9_CY

[ATomo9_CY 0]

 the code below is in MouseHelper

 public void registerCallbacks(long handle) {
      InputMappings.func_216503_a(handle, this::cursorPosCallback, this::mouseButtonCallback, this::scrollCallback);
   }

 

the methods it mentions are all private but i think they are the ones responsible for determining the mouse position and actions ingame.

Is there a way to call those methods or fake a mouse movement to get them called?

 

 

[ATomo9_CY 0]

I made my own mousehelper class with public methods but when i call them nothing happens.I can have the game playing without focus and without it holding my cursor but i cant make it behave like the mouse is moving.This is the only thing left to get it working.If anybody can help me this is the only thing i need

[ATomo9_CY 0]

Nevermind i fixed it.It works now.I had commented some code that changed the mouse position while trying something else previously and forgot to uncomment it.

I think simulating game input could be important to other modders so if there is any place to show how i did it let me know.ill share the code and changes needed

[eabjab 0]

I am trying to do something similar and ran into the same issue. Would you mind sharing your solution to change mouse position?