Memory Profiling?

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
User avatar
Madrayken
Party member
Posts: 110
Joined: Sun May 04, 2014 4:21 pm
Location: Oakland CA
Contact:

Memory Profiling?

Post by Madrayken » Tue Aug 18, 2020 4:15 pm

Hi folks,
I'm writing a fairly large RPG (for a Lua project, anyway) and have just started running into a peculiar out-of-memory error far earlier than I expected. As I dive into the reasons for the OOM error, I was thinking it'd be really handy to have a basic 'one number' memory-use stat. Has anyone managed to figure out how to get something like this to work? Is it already somewhere super-obvious that I've missed?

Thanks in advance for any help or advice!
Discord: https://discord.gg/tYfHgXc
Bandcamp: https://madrayken.bandcamp.com/
Twitter: @Fluttermind

User avatar
zorg
Party member
Posts: 3043
Joined: Thu Dec 13, 2012 2:55 pm
Location: Absurdistan, Hungary
Contact:

Re: Memory Profiling?

Post by zorg » Tue Aug 18, 2020 4:30 pm

Iirc this has been discussed before, unfortunately there's no simple way to count all the main memory used by the application as far as i'm aware.

That said, LuaJIT has a memory limit (from 1-2GB on 64bit to 4GB on 32bit or something like that), whether or not you're intending to use the 32 or 64 bit version of löve. One option for really large mem requirements would be to use 64bit and löve-owned C++-side objects (all the Data objects, like ImageData, SoundData, ByteData - the last one specially if you need misc mem areas for other purposes... you'll need to use the FFI though to write and read from them though.)

I did read about something regarding a luajit fork using a newer garbage collector that makes it possible to use a larger address space but i'm sure it's not included in löve yet... not sure if it can be. :o:
Me and my stuff :3True Neutral Aspirant. Why, yes, i do indeed enjoy sarcastically correcting others when they make the most blatant of spelling mistakes. No bullying or trolling the innocent tho.

User avatar
Madrayken
Party member
Posts: 110
Joined: Sun May 04, 2014 4:21 pm
Location: Oakland CA
Contact:

Re: Memory Profiling?

Post by Madrayken » Tue Aug 18, 2020 4:53 pm

Thanks. I feared that might be the case. I've seen external memory profilers (e.g. ones mentioned in https://www.lua.org/wshop15/Musa2.pdf) but I'd have no idea how to get them to work. For now I'm literally just watching the Activity Monitor app and watching my memory use suddenly jump to 1Gb making the same calls I've been using all over the place for a year. Fun day ahead... ack!
Discord: https://discord.gg/tYfHgXc
Bandcamp: https://madrayken.bandcamp.com/
Twitter: @Fluttermind

User avatar
pgimeno
Party member
Posts: 2416
Joined: Sun Oct 18, 2015 2:58 pm

Re: Memory Profiling?

Post by pgimeno » Tue Aug 18, 2020 10:32 pm

LuaJIT 2.1 (not a fork, the official one) supports much more memory. Circa 48 bits if my recollection is right. I don't think it's included with Löve yet, though.

User avatar
Xii
Prole
Posts: 47
Joined: Thu Aug 13, 2020 9:09 pm
Contact:

Re: Memory Profiling?

Post by Xii » Tue Aug 18, 2020 11:59 pm

If you are using Lua tables to store game information, like npc.health=100 or map[pos]=tile_id or whatever, you can significantly reduce your memory consumption by using FFI. Here is an article detailing how to do that. Using FFI you can define compact memory structures for values that simply do not need the full 64 bits range that regular Lua values have.


User avatar
ivan
Party member
Posts: 1663
Joined: Fri Mar 07, 2008 1:39 pm
Contact:

Re: Memory Profiling?

Post by ivan » Wed Aug 19, 2020 6:48 am

Running out of Lua memory shouldn't happen unless you are doing something terribly wrong.
Make sure you set any unused references to nil otherwise the garbage collector can't release that memory.

gcmartijn
Prole
Posts: 36
Joined: Sat Dec 28, 2019 6:35 pm

Re: Memory Profiling?

Post by gcmartijn » Wed Aug 19, 2020 7:22 pm

ivan wrote:
Wed Aug 19, 2020 6:48 am
Make sure you set any unused references to nil otherwise the garbage collector can't release that memory.
I need to test this, but what about something like this.

Code: Select all

function add (data)
      local rawdata = data
      --- do things
      return something
    end
    
local x = add({bla})
Do we always need to set variable to nil for the garbage collector ?

Code: Select all

function add (data)
      local rawdata = data
      --- do things
      
      rawdata = nil
      data = nil -- the function to ?
      
      return something
    end
 
 local x = add({bla})

User avatar
Jeeper
Party member
Posts: 585
Joined: Tue Mar 12, 2013 7:11 pm
Contact:

Re: Memory Profiling?

Post by Jeeper » Wed Aug 19, 2020 9:55 pm

zorg wrote:
Tue Aug 18, 2020 4:30 pm
Iirc this has been discussed before, unfortunately there's no simple way to count all the main memory used by the application as far as i'm aware.

That said, LuaJIT has a memory limit (from 1-2GB on 64bit to 4GB on 32bit or something like that), whether or not you're intending to use the 32 or 64 bit version of löve. One option for really large mem requirements would be to use 64bit and löve-owned C++-side objects (all the Data objects, like ImageData, SoundData, ByteData - the last one specially if you need misc mem areas for other purposes... you'll need to use the FFI though to write and read from them though.)

I did read about something regarding a luajit fork using a newer garbage collector that makes it possible to use a larger address space but i'm sure it's not included in löve yet... not sure if it can be. :o:
Did you miss type or does the 32bit version have support for more memory? That would make no sense?

User avatar
pgimeno
Party member
Posts: 2416
Joined: Sun Oct 18, 2015 2:58 pm

Re: Memory Profiling?

Post by pgimeno » Wed Aug 19, 2020 11:08 pm

gcmartijn wrote:
Wed Aug 19, 2020 7:22 pm
Do we always need to set variable to nil for the garbage collector ?
No. Going out of scope is equivalent to setting it to nil.

Sometimes, Löve objects take a long time to be cleaned up by the GC, letting many of them to accumulate and take a lot of memory. In these cases, it's best to explicitly release them with Object:release. I wonder if that's what's happening here.

Jeeper wrote:
Wed Aug 19, 2020 9:55 pm
Did you miss type or does the 32bit version have support for more memory? That would make no sense?
The 1 GB limit in 64 bits applies to Linux.
https://stackoverflow.com/questions/351 ... -platforms
http://timetobleed.com/digging-out-the- ... egression/
Last edited by pgimeno on Wed Aug 19, 2020 11:15 pm, edited 2 times in total.

Post Reply

Who is online

Users browsing this forum: No registered users and 48 guests