Packets. Large amounts of data. Breaking the 32kb cap

[Draco18s]

I have a need to send a packet from client to server (and back) dealing with data sizes larger than 32 kb.  A lot larger.

(Interestingly an array of 4096 integers is the max the current packets can handle, which should only be 16kb)

 

Based on my current handling of ~128kb, I'd like to increase this at least 4 fold (~512kb) without having to do the same thing I'm currently doing, where the server can only handle one player at a time sending the data (it sets a Lock flag until it receives the final packet from the original player).  This is reasonably acceptable for only 4 packets and the communication I'm performing, but for something a lot larger I'm not as certain.

 

Someone pointed me at FMLPacket, as it had a way to automatically split packet data into 32kb chunks, but that won't work, as the FMLPacket is only used for specific FML purposes: it requires a packet type, which is one of: mod_list_request/response, mod_ids, mod_missing, guiopen, entityspawn(adjustment), or mod_idmap.

 

Any other ideas?

[Draco18s]

I think I might just yoink that.

 

Thanks

[GotoLink]

Is there any reason you want to lock the data sent to one player at a time ? Is the data contained dependent on any player state ?

Asking because if the player disconnect before all packets are done, that could lock the entire thing.

[Draco18s]

Is there any reason you want to lock the data sent to one player at a time ? Is the data contained dependent on any player state ?

Asking because if the player disconnect before all packets are done, that could lock the entire thing.

 

It's only 4 packets sent off all at once.  I haven't tried to handle edge cases like that yet.  Which is part of the reason I was looking for another solution, because I knew that the one I have is untenable at larger scales.

[Draco18s]

Feel free to look at my code in SevenCommons, a coremod I am developing as a library/helper collection for my mods. It comes with a packet system which can handle any amount of data. It will automatically split it in appropriate chunks and put them back together on the other side without you having to do anything except mark your packet as being multipart.

It is not publicly available in a built form, but you can look at the github and get ideas: https://github.com/diesieben07/SevenCommons/tree/master/source/de/take_weiland/mods/commons/network

 

Hey, I'm trying to figure out how to use this, and I found the class I needed to extend in order to create my own multipart packet and send it to the server (or back).  But I'm not sure how I am supposed to handle the packets when received.

[rich1051414]

It would be fairly easy to implement your own. First output your data into a byte array, then split it up into chunks the size you want and send them, then after sending the last part, send a small final packet so the client can know the packet is done, then merge those parts back together client side and then extract whatever data it holds.

[Draco18s]

It would be fairly easy to implement your own. First output your data into a byte array, then split it up into chunks the size you want and send them, then after sending the last part, send a small final packet so the client can know the packet is done, then merge those parts back together client side and then extract whatever data it holds.

 

Except that I need to go both ways.