Memory Profiling?

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
User avatar
Sasha264
Party member
Posts: 131
Joined: Mon Sep 08, 2014 7:57 am

Re: Memory Profiling?

Post by Sasha264 »

slime wrote: Sat Aug 22, 2020 4:02 pm LuaJIT's hard memory limit is per process rather than per Lua instance within a process unfortunately, so spreading memory across multiple threads won't raise the limit.
Thank you! Saved some time for me trying that :megagrin:
That being said, most memory use in games tends to be things like textures and other large bits of data, most of which doesn't count towards LuaJIT's Lua-allocated memory limit when you use love objects. There are tons of ways to store large collections of data in memory allocated by love rather than by Lua, and doing so often comes with performance benefits as well.
In games — absolutely! Performance benefits as well, yes!
For example depending on how you structure string concatenation within a loop, it can use a lot more temporary memory than you might expect.
Yes, I solved that problem with something like this. But it was overhead troubles. I just wanted to parse bunch of json files, save them modified in another folder, and that's it, remove that script :3 One time operation, does not care about performance or memory usage, only about simplicity. But it turned out that I must care about memory usage :rofl:
love 12.
Yay! :cool: Great news :cool:
User avatar
Sasha264
Party member
Posts: 131
Joined: Mon Sep 08, 2014 7:57 am

Re: Memory Profiling?

Post by Sasha264 »

slime wrote: Sat Aug 22, 2020 4:02 pm LuaJIT's hard memory limit is per process rather than per Lua instance within a process unfortunately, so spreading memory across multiple threads won't raise the limit.
Oh, wait! Does this mean that separate love threads gets common lua garbage collection? So if one of my custom threads do a lot of garbage collection it will affect performance of other threads, and most important — main thread, that does graphic work? :o
User avatar
slime
Solid Snayke
Posts: 3131
Joined: Mon Aug 23, 2010 6:45 am
Location: Nova Scotia, Canada
Contact:

Re: Memory Profiling?

Post by slime »

Garbage collection is local to individual threads / Lua instances, it's just the internal memory address space used by LuaJIT 2.0 that is shared (so in practice the only Lua thing shared is the memory cap).
User avatar
Sasha264
Party member
Posts: 131
Joined: Mon Sep 08, 2014 7:57 am

Re: Memory Profiling?

Post by Sasha264 »

Hm... Interesting, thank you for enlightening, that was counter-intuitive for me =)
User avatar
pgimeno
Party member
Posts: 3544
Joined: Sun Oct 18, 2015 2:58 pm

Re: Memory Profiling?

Post by pgimeno »

Xii wrote: Fri Aug 21, 2020 1:19 am You are surely mistaken. C structs allocated with ffi.new definitely show up on my collectgarbage("count")
You're right. My recollection was wrong.

Sasha264 wrote: Sat Aug 22, 2020 3:15 pm For example I tried to parse some heavy json-s with lua on love. (but not so heavy really, just several files about 50 mb in filesize)
Parsing in Lua is best done avoiding generation of substrings, by using string.byte as much as possible. The resulting parser looks really awful, but it's fast and doesn't generate so much garbage.

See for example https://notabug.org/pgimeno/minetest/co ... 0077c9R111 which is a simple parser to escape Lua strings with a special escaping scheme.

It might also help to use an FFI buffer for building strings from parts. But in general, Lua is a horrible language for writing a parser.
User avatar
Sasha264
Party member
Posts: 131
Joined: Mon Sep 08, 2014 7:57 am

Re: Memory Profiling?

Post by Sasha264 »

Looks really as difficult example =)
I used some existing library https://github.com/rxi/json.lua
But if I will need to parse something on my own in Lua, I will remember your advise :cool:
rincewind
Prole
Posts: 4
Joined: Wed Dec 26, 2018 3:30 pm

Re: Memory Profiling?

Post by rincewind »

pgimeno wrote: Wed Aug 19, 2020 11:08 pm
The 1 GB limit in 64 bits applies to Linux.
https://stackoverflow.com/questions/351 ... -platforms
http://timetobleed.com/digging-out-the- ... egression/
Hello!
is it same for FreeBSD or just Linux specific?
User avatar
pgimeno
Party member
Posts: 3544
Joined: Sun Oct 18, 2015 2:58 pm

Re: Memory Profiling?

Post by pgimeno »

It's Linux specific. I don't know the limit in FreeBSD. In Windows I don't remember if it was 1.5 GB or 2 GB.
gcmartijn
Party member
Posts: 134
Joined: Sat Dec 28, 2019 6:35 pm

Re: Memory Profiling?

Post by gcmartijn »

pgimeno wrote: Wed Aug 19, 2020 11:08 pm
gcmartijn wrote: Wed Aug 19, 2020 7:22 pm Do we always need to set variable to nil for the garbage collector ?
No. Going out of scope is equivalent to setting it to nil.

Sometimes, Löve objects take a long time to be cleaned up by the GC, letting many of them to accumulate and take a lot of memory. In these cases, it's best to explicitly release them with Object:release. I wonder if that's what's happening here.

Jeeper wrote: Wed Aug 19, 2020 9:55 pm Did you miss type or does the 32bit version have support for more memory? That would make no sense?
The 1 GB limit in 64 bits applies to Linux.
https://stackoverflow.com/questions/351 ... -platforms
http://timetobleed.com/digging-out-the- ... egression/
@pgimeno
I was searching for memory fluctuations inside my code.
And I never set temp tables to nil (and other variables), after you did mention that leaving a function is automaticity (at some point) free up some memory. This make the code more readable (less lines).

But now I did test something, and want to know if this behaviour is oke.

Situation 1: a function create a huge temp table, and the garbage count is now 1024 big.
I did thought that is was not, after your comment.
But is this garbage a problem? at this point ? Or will new object inside the garbage replaces (overwritten), and is in my case 1024 the max garbage I can have.

Code: Select all

function bla()
    t = {}
    for i = 1, 100000000 do
        table.insert(t, "hello")
    end
end

function love.load()
    bla() -- garbage count == 1024 now
end

function love.draw()
    love.graphics.print(
        string.format(
            "Memory: gfx=%.1fMiB lua=%.1fMiB",
            love.graphics.getStats().texturememory / 1024 ^ 2,
            collectgarbage "count" / 1024
        ),10,25)
end
Situation 2: the same, but now I want to manual remove it, it don't work.

Code: Select all

function bla()
    t = {}
    for i = 1, 100000000 do
        table.insert(t, "hello")
    end
end

function love.load()
    bla() -- garbage count == 1024 now
    collectgarbage("collect") -- does nothing
end
Situation 3: works

Code: Select all

function bla()
    t = {}
    for i = 1, 100000000 do
        table.insert(t, "hello")
    end
    t = nil -- we don't need it as garbage
end

function love.load()
    bla() -- garbage count == 0.x now
end
I like situation 3 when searching for memory issues or fluctuations in the future.
But will give it other benefits? I have to add multiple lines to my code.

And I don't know why situation 2 did not work, but I don't really use that.
I was thinking about manual garbage collections (turn it off), and only 'collect'/remove the garbage on a frame update (to see what happens).
User avatar
pgimeno
Party member
Posts: 3544
Joined: Sun Oct 18, 2015 2:58 pm

Re: Memory Profiling?

Post by pgimeno »

gcmartijn wrote: Sun Aug 21, 2022 7:49 am

Code: Select all

function bla()
    t = {}
    for i = 1, 100000000 do
        table.insert(t, "hello")
    end
end
`t` does not go out of scope, because it's not local to the function: it's either global or an upvalue. So, it's still in scope when the function ends, therefore it can be read in other functions like love.update or love.draw, so the data must be kept in the variable and can't be cleared by the garbage collector. Only local variables go out of scope when the function ends.

Code: Select all

function bla()
    t = {}
    for i = 1, 100000000 do
        table.insert(t, "hello")
    end
end

function love.load()
    bla()
end

function love.draw()
    love.graphics.print(t[100000000])
        -- prints "hello" because t is visible from here
end

That's yet another case of "globals are evil". Get used to making everything local; it will make the code more readable and prevent some nasty bugs.
Last edited by pgimeno on Sun Aug 21, 2022 10:50 am, edited 1 time in total.
Post Reply

Who is online

Users browsing this forum: Ahrefs [Bot], Bing [Bot] and 55 guests