Twitch is not very reliable when it comes to setting Forge up correctly I'm afraid.
I'd recommend using the method detailed in the video in the logs section of my signature below, it's how to run a Twitch pack without using Twitch
Twitch seems ok at distributing their mods and not much else.
Personally I use Twitch only to install packs, then I set the GameDir to the modpack dir and use the regular launcher
Please stop bumping your thread.
Just make sure the server and your client are stopped, start both, reproduce the issue, stop both, then provide the debug.log from both.
Changing the GameDir doesn't tell the launcher to use Forge, it tells it where to look for saves, logs, configs, screenshots, mods, etc.
You tell it to use Forge by changing the Minecraft Version to a Forge version