The fact that arrows work but player hitting doesn't supports my idea that the problem is somewhere in the mouse over method. Once the arrows are flying they use their own collision box to check for collisions. When the player tries to hit something it checks the space around the "look vector" which involves methods like the one I've mentioned.
So I still feel the problem can be understood by looking at the EntityRenderer.getMouseOver() method. I'm not certain that radius field is the problem, but it sure sounded like it should be related. Maybe you should use debug mode in Eclipse and set a breakpoint for when user clicks button and step through to see what is happening.