[1.10.2] Frame-based recording and synchronization issues

[horsewithnoname]

Hello there!

 

I'm working on a mod that allows recording player movement and some actions (like block interaction, dropping items, etc.). In previous versions, the record and playback of player movement and actions, was pretty primitive, it simply recorded position, rotation, etc. in separate thread and played back those actions also in separate thread. For the next release, I wanted to update recording code to the next level (so it would be very smooth and Minema compatible). I did implemented smooth recording, but I experience tick desynchroinzation issues sometime.

 

Right now, the recording code works following way (source code if somebody needs more than an explanation):

 

1. Player notifies server that he wants to record himself (via command or interacting with actor mob)

2. Server starts recording and sends a message to the client that recording has started

3. Server records actions (pushed via event handler on the server-side) and client records position, rotation, motion, fall distance and some booleans (onGround, isSneaking, etc.)

4. When player asks the server to stop recording (via command or interacting with actor mob), server asks player for the recorded frames, and puts recorded actions on hold until client frames will arrive

5. When client frames are arrived, server adjusts the length of the action list and saves frames and actions to the file

 

There are no issues with recording, but with playback, I experience tick desynchronization issue. I don't know why it happens, but sometimes, client update loop can't keep up with server loop, I guess, and in this case it makes playback look really awful (especially while flying elytra).

 

Playback code works similar to recording code, but it's more complicated:

 

1. When actor is starting playing back (requested by player), actor being played from recorded record on the server

2. On the client, however, recording might not exist, so the server will send actor tracking clients record, if they don't have it, while giving an actor empty playback which will just run ticks (in order to keep up with the server)

3. When the record is finally arrived on the client, it being injected into the empty playback which will then start played back on the client every tick

4. When playback is finished, server notifies the players that an actor stopped playing

5. Clients will simply remove the playback from the actor and next time, they would be able to use the same record for playback

 

tl;dr. Yes, it's a pretty complicated system, but I'm not asking to rewrite it for me. I described how it works, if you needed that info, to answer my question/help me with my problem.

 

So now is the question time. How can I resolve this issue with tick desynchronization? I see only two options:

 

1. Rewrite playback code so it will be played only on server (i.e. every tick server going to send packets with position, rotation, motion, etc.)

2. Every N ticks, send to the client the number of ticks server has

 

I implemented currently 2. as a workaround, but I feel that with slow connection the arrived tick will probably be outdated by the time it arrived. But with 1., I fear it will create really dense server load when there are lots of players and lots of actors being played back.

 

What will be the optimal solution?

 

Thanks for attention! (sorry for the wall of text)

[TheGreyGhost]

Hi

 

I think your option 1 is the best bet for live action stuff and probably the most resistant to network lag.

 

However if you're actually aiming for a machinima rendering that doesn't need to be in real time, you might be better off locking the ticks between client and server eg with a handshaking, to ensure there is never any desynchronisation. 

 

I don't think the update-tick-count resynchronise is likely to help much with lag / judder.  If you are just dropping frames now and again, it might work if you can assume a fairly constant network latency that you can correct for.

You could perhaps improve it by buffering ahead with timestamps, for example the server runs its ticks strictly according to the timestamps compared to the system clock, and the same on the client;  but I think this would still rather sensitive to network delay.

 

I think the best answer will probably depend on exactly what you want to achieve with the playback and probably the best way to find the answer is to code it a few different ways and stress test it.

 

-TGG

 

 

 

[horsewithnoname]

Thanks for the answer TGG, by the way, you have really nice tutorials from which I learned a lot

 

However if you're actually aiming for a machinima rendering that doesn't need to be in real time, you might be better off locking the ticks between client and server eg with a handshaking, to ensure there is never any desynchronisation.

 

Well, for now, yes, I'm aiming for machinima rendering. With machinima rendering, it's actually pretty good, I haven't noticed any frame desynchronization since Minema does the synchronzation work for me, but for live stuff I'm not sure about it.

 

In the future, I would like to expand this recording/playback code to make cinematic stuff like where there are NPC's that do some pre-recorded stuff like in Borderlands 2 or Skyrim (just from the top of my head). I do not aim at multiplayer servers, though.

 

Your offer seems a little bit less expensive than option 1, and probably will have some issues during the lag (probably, although lag is the problem itself).

 

I don't think the update-tick-count resynchronise is likely to help much with lag / judder.  If you are just dropping frames now and again, it might work if you can assume a fairly constant network latency that you can correct for.

 

You're correct. Depending on the network latency it may drastically affect the playback. I tested everything only in single-player. By the way, I'm not aiming my mod toward public servers.

 

I think the best answer will probably depend on exactly what you want to achieve with the playback and probably the best way to find the answer is to code it a few different ways and stress test it.

 

Ok, I'll try different ways in the future, thank you

 

P.S.: What does stress test means?

[TheGreyGhost]

Hi

 

> Thanks for the answer TGG, by the way, you have really nice tutorials from which I learned a lot

Keen, glad you found them useful

 

What I mean by 'stress test' is testing your code with difficult conditions - like high latency, complicated animation, high server CPU load, that sort of thing.

 

Let me know how it goes?  It sounds very interesting.

 

-TGG

 

[horsewithnoname]

What I mean by 'stress test' is testing your code with difficult conditions - like high latency, complicated animation, high server CPU load, that sort of thing.

 

Oh, I get it now, thanks

 

Let me know how it goes?  It sounds very interesting.

 

Of course I think I'll rewrite in the next update to the option 1.