I need to split up a table

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
Post Reply
User avatar
togFox
Party member
Posts: 764
Joined: Sat Jan 30, 2021 9:46 am
Location: Brisbane, Oztralia

I need to split up a table

Post by togFox »

I have UDP sockets enabling co-op on my dungeon crawler and when a client connects to the host the first thing the host does is send the game world to the client. The host sends a range of layers that describe the game world:

- tile information (the maps)
- obstacle information (rocks, trees, etc that sit ON TOP of the tiles)
- tile weight information for each tile (for path-finding)
- object information (things on the tiles)

and I expect even more layers in the future like weather and environment effects. The UDP transfer is a simple bitser serialisation into a single string (datagram) for transmission to the client. Unfortunately, sending even one layer on a modest 100 x 100 map makes UDP go "MESSAGE TOO LARGE" (or something). My serialised string for one layer is about 11,000 characters and I'm guess that is exceeding the UDP limit for a single datagram (I assume to be just under 64k bytes).

So - what to do?

Most obvious thing is to send the world information in smaller chunks. I can send say, 20 rows at a time and do this 5 times to get the whole 100 rows to the client. I was hoping to have a much larger world or even an indefinite world so not sure this is a good idea.

I'm now leaning towards the client requesting just the 'chunk' that is visible on the screen (+ some reasonable margin). That would also be about 30 rows/columns of viewable tiles but it would need to be requested and sent repeatedly on demand as the players move around the world. Some of that can be cached but not much. Of all the layers I described above, nearly all of them can change or be impacted by player actions.

As a proof of concept, for demo purposes and to test the UDP/sockets, what is the correct way to break up a 100 x 100 table into 5 smaller tables? UDP can't send tables so each of those 5 messages MUST be serialised or textified in some way. I'm not too worried about unpacking and stitching the table back together. I assume that will be a simple thing.

And THEN - when I've sent those five messages for that ONE layer - I need to do the same for the other layers.

I guess I'm on a learning curve and I'll take good, bad and ugly suggestions. I'm an iterative guy (aren't most of us) where I'll take anything that works and then refactor it later.

I'm guessing my problem is not unique so thought I'd learn from past failures and successes. :)
Last edited by togFox on Fri May 14, 2021 12:40 am, edited 1 time in total.
Current project:
https://togfox.itch.io/backyard-gridiron-manager
American football manager/sim game - build and manage a roster and win season after season
User avatar
darkfrei
Party member
Posts: 1168
Joined: Sat Feb 08, 2020 11:09 pm

Re: I need to split up a table

Post by darkfrei »

Serialization to the binary data an send it as a list of strings?
:awesome: in Lua we Löve
:awesome: Platformer Guide
:awesome: freebies
User avatar
togFox
Party member
Posts: 764
Joined: Sat Jan 30, 2021 9:46 am
Location: Brisbane, Oztralia

Re: I need to split up a table

Post by togFox »

UDP will send text and only text so I guess I could serialize each row and then send each row as text then unpack it on the receiving end and table.insert(t, unpackeddata).

I guess.
Current project:
https://togfox.itch.io/backyard-gridiron-manager
American football manager/sim game - build and manage a roster and win season after season
User avatar
ivan
Party member
Posts: 1911
Joined: Fri Mar 07, 2008 1:39 pm
Contact:

Re: I need to split up a table

Post by ivan »

If your level generator is deterministic you can just send the initial random seed as well as the visible range and let the client reproduce the level from scratch. Note that the server would still need to provide information about the state of destructible tiles and so on.
User avatar
zorg
Party member
Posts: 3435
Joined: Thu Dec 13, 2012 2:55 pm
Location: Absurdistan, Hungary
Contact:

Re: I need to split up a table

Post by zorg »

togFox wrote: Fri May 14, 2021 1:48 am UDP will send text and only text so I guess I could serialize each row and then send each row as text then unpack it on the receiving end and table.insert(t, unpackeddata).

I guess.
you mean strings. strings as in 8-bit clean, arbitrary strings. binary data can be represented with such strings. love.data.pack and unpack might be helpful, but you already said that you're using bitser so it's probably compressed enough as it is.
Me and my stuff :3True Neutral Aspirant. Why, yes, i do indeed enjoy sarcastically correcting others when they make the most blatant of spelling mistakes. No bullying or trolling the innocent tho.
User avatar
togFox
Party member
Posts: 764
Joined: Sat Jan 30, 2021 9:46 am
Location: Brisbane, Oztralia

Re: I need to split up a table

Post by togFox »

Yes. Bitser so it should be already compact.

My map is not deterministic but I guess I can change it so it could be. It's not too late to make that change.

I'd have the same problem with the other layers but it just occurred to me that most of those 'cells' are empty because only a very small number of tiles will have an object or be destroyed or whatever.

Perhaps with a deterministic map doing the heavy lifting + other layers reporting by exception (I.e only transmit the table elements that are non-null) then data volumes can become really quite small.

:)
Current project:
https://togfox.itch.io/backyard-gridiron-manager
American football manager/sim game - build and manage a roster and win season after season
User avatar
pgimeno
Party member
Posts: 3541
Joined: Sun Oct 18, 2015 2:58 pm

Re: I need to split up a table

Post by pgimeno »

Maybe switch to lua-enet?

"ENet will send and deliver packets regardless of size. Large packets are fragmented into many smaller packets of suitable size, and reassembled on the foreign host to recover the original packet for delivery. The process is entirely transparent to the developer." http://enet.bespin.org/Features.html
User avatar
Gunroar:Cannon()
Party member
Posts: 1085
Joined: Thu Dec 10, 2020 1:57 am

Re: I need to split up a table

Post by Gunroar:Cannon() »

Yeah, and maybe use sock.lua or some other library to make it easier(unless it messes with your structure and vibe).
The risk I took was calculated,
but man, am I bad at math.

-How to be saved and born again :huh:
User avatar
kicknbritt
Citizen
Posts: 96
Joined: Sat May 30, 2015 2:15 am
Location: Chicago, IL/Anchorage,AK

Re: I need to split up a table

Post by kicknbritt »

+1 For enet. Alot of the information you are sending seems crucial so you wouldn't want to fragment and drop random pieces of data (which will happen with udp packets)
"I AM THE ARBITER!!!" *Pulls out Energy sword and kills everything*
Post Reply

Who is online

Users browsing this forum: No registered users and 11 guests