This is actually not caused by not enough memory for the OS.
Java's garbage collector tend to clear the memory when the allocated RAM is full. If the allocated RAM is little (i.e. 2GB), the clearing would be fast as it only need to clear 2GB, and the delay would be negligible to the user. However, if the allocated RAM is large (i.e. >10GB), then each clearing would take up significant longer time, thus lagging the game.