[SOLVED] Performance Bottleneck with Multithreading

[FlameAtronach93]

Hello there!

 

I've been working a bit on my mod which uses a number of threads "hosted" in a cached thread pool to do things in the background. However it seems I've introduced quite a lag to the server and can't quite find out why... I've tried adding profiler sections to crucial parts of my code but still haven't found any critical and unacceptable executions. So I am suspecting the cached thread pool to somehow be the cause of this misfortune especially since I've noticed the lag persists as long as the worker threads (2 so far) are alive, but also as long as the player is standing still (animals in the background suddenly move fluently, e.g.).

 

I'm at doubt where the exact issue lies and thought maybe somebody here has an idea...

 

Also I'd like to ask what the desired average tick should be? When the worker threads die (i.e. no player is on the server for a while) the average tick is around 1ms, otherwise without worker threads and the payload loaded directly onto the main thread the average tick is 15ms and with worker threads it's an average of 55ms - definitely unacceptable.

 

Cheers and thanks in advance for any assistance!

 

 

Edit:

 

After some digging around I found the potential culprit. It had little to nothing to do with the multithreading but with particular data being permanently (until program termination) stored. I don't know what is causing this, possibly the implementation of the third party class I'm extending. Nonetheless I am working on a usable workaround. Thus I consider this thread solved.

[TheGreyGhost]

Hi

 

I reckon adding extra threads is only an advantage if your performance is bound by the CPU.  If you're constrained by memory or disk access then the extra threads probably won't help and in fact if they're blocking the server thread they will make things worse.

 

You might benefit from a good profiling/system monitoring tool.  Even something like SysInternal's Process Explorer will be a good start.

 

If the server starts falling behind on its tick execution for more than a half a second, the player will probably notice it, i.e. it's a problem, otherwise not.  If the server run() method is getting at least some idle time each tick then it's ok.

An average above 50ms is of course unacceptable. You want to have some spare time per tick for when something happens, but guaranteed the player can always make your server lag (eg blowing up a stack of TNT).

 

If you want to maximise the time per tick, instead of creating a new thread for your 'background' tasks, you could instead put them into a server tick event and keep track of the server load using System.nanoTime().  i.e. as long as the tick spacing remains stable, slowly increase the amount of time your background tasks use per tick.  But if your tick event starts arriving later than 50 ms intevals, throttle back your background tasks or skip them entirely until the server catches up again.    The Server tick event has pre and post events which should give you enough information to tell how long each tick is taking.

 

-TGG

 

 

 

[FlameAtronach93]

Hello again,

 

thank you for your input. The weird part about the background tasks is that actually there is not a single constraint. The tasks are performed in due time, in fact the worker threads of the thread pool seem to be blocking according to the tool I've tried. (I forgot its name, also since I'm currently sitting in a lecture I currently cannot provide screenshots.)

 

The tasks are prepared before the game starts. The necessary information is kept in RAM. The profiler tool I've used shows the heap is at the utter most 50% in use, CPU usage is around 25%. Screenshots to follow, maybe this night, otherwise tomorrow.

 

Once again, if I run the tasks in the main thread everything is fine. Furthermore I simply use the threads/futures to allow interrupting the calculation in case it takes too long. In fact, I expect the result of the computation while blocking the main thread. Quite the irony.

 

I reckon I am missing something... I suppose there is something to learn here for me.

 

Cheers!

[Anon10W1z]

No! Do not do that!

 

 

There is ServerTickEvent, ClientTickEvent, etc...do some research on tick events. They get run 20 times/second.

[FlameAtronach93]

I dug around a little bit in the documentation of said 3rd party library and may have found a solution which allows me to dump the executor service completely...

 

  On 1/24/2015 at 2:30 AM, Anon10W1z said:

No! Do not do that!

 

 

There is ServerTickEvent, ClientTickEvent, etc...do some research on tick events. They get run 20 times/second.

No, don't do what? You haven't addressed a single of my previous statements - did you even read any of them?

 

I have done my research on the topic long before you created that account. Until I found above said potential solution there was no way to temporarily suspend the work of a third party library of which I have little control. Nonetheless I'd still like to know what I'm doing wrong with the thread pool.