I have an event handler that I'm trying to use to automatically give the glowing effect to any mobs the player looks at while priming his bow. So I'm using this code (including all the debug output for now):

 

@SubscribeEvent
	public void onPlayerUsingBow(PlayerTickEvent ev) {
		if (ev.phase == TickEvent.Phase.END) {
			EntityPlayer player = ev.player;
			ItemStack usingItem = player.getActiveItemStack();
			int isInUse = player.getItemInUseCount();
			if (usingItem.getItem() instanceof ItemBow && isInUse > 0) {
				RayTraceResult trace = player.rayTrace(80.0d, 1.0f);
				System.out.println("Type: " + trace.typeOfHit);
				System.out.println("Entity: " + trace.entityHit);
				System.out.println("Is Living?: " + (trace.entityHit instanceof EntityLivingBase));
				if (trace.typeOfHit == RayTraceResult.Type.ENTITY && trace.entityHit != null
						&& trace.entityHit instanceof EntityLivingBase) {
					EntityLivingBase livingEnt = (EntityLivingBase) trace.entityHit;
					livingEnt.addPotionEffect(new PotionEffect(MobEffects.GLOWING, 20));
					System.out.println("Added glowing to " + livingEnt);
				}
			}
		}
	}

 

According to the debug output, though, the RayTraceResult's type is ALWAYS Block, even if I'm staring directly at a zombie or some such mob. Like, I can be just one block away, aiming directly at the zombie's head or body, and it still gives me a Block type result with a null entityHit.

Am I misunderstanding how to use the EntityPlayer#rayTrace method, or is there a better way to go about finding which mob the player is looking at (within 80 blocks or so)?

Entity::rayTrace only looks for blocks as it reroutes to World::rayTraceBlocks. You will need a custom ray-tracing method. You can see an example at EntityArrow::findEntityOnPath.

You might actually want the function getMouseOver() in EntityRenderer, it's pretty much what you're looking for (note: this existing method is client-side only and if you need to run the code on the server, you will have to duplicate it).