Forge is lowering fps or limiting gpu usage

So i'm running modded minecraft with shaders and noticed my fps was quite low. I did some tests and found that on minecraft with optifine alone, my pc could handle shaders at 130 fps at 48 render distance and also that the gpu usage was on 100%, while when i had forge and optifine at the same time, it wouldn't even handle 90 constant at 12 render distance, and the gpu usage varied from 30-50%. Is there any fix to this?