I have two new server instances setup with a custom modpack, Minecraft 1.7.10, and Forge 1291. The servers are test and production for the same code, all mods and config files are identical and copied out of my Git repo. They run on different ports with more memory to prod. Both servers save to SSD. Host is Linux, no firewall locally. LAN connections have the same issues as remote NAT connections, so Internet facing firewall isn't the issue.
No problems in test. No errors on launch, server is visible in server list and clients connect normally. 10 players max online during test, no network issues.
Prod instance on the same server (separate port) *always* shows as down in the server list (red X) on first open, even when it's up. You can hit refresh, normally it doesn't change the state. If you try to connect anyway you can get in after several attempts. Half the time it must timeout once first (if you cancel, it still won't work until you timeout). I have 15+ players online and no lag, but frequent odd network issues like this.
The logs below are too big for pastebin, the client has TRACE enabled. This was recorded during a client trying to connect and timing out.
http://demosthenes.org/tmp/client.log
http://demosthenes.org/tmp/server.log
Other than these logs I'm not sure where to start troubleshooting. I've not yet seen any one mod fingered in the logs, there's no crashes. Just failure to connect to one server when it's the duplicate of another.