Memory Profiling?

[Madrayken]

Hi folks,
I'm writing a fairly large RPG (for a Lua project, anyway) and have just started running into a peculiar out-of-memory error far earlier than I expected. As I dive into the reasons for the OOM error, I was thinking it'd be really handy to have a basic 'one number' memory-use stat. Has anyone managed to figure out how to get something like this to work? Is it already somewhere super-obvious that I've missed?

Thanks in advance for any help or advice!

[zorg]

Iirc this has been discussed before, unfortunately there's no simple way to count all the main memory used by the application as far as i'm aware.

That said, LuaJIT has a memory limit (from 1-2GB on 64bit to 4GB on 32bit or something like that), whether or not you're intending to use the 32 or 64 bit version of löve. One option for really large mem requirements would be to use 64bit and löve-owned C++-side objects (all the Data objects, like ImageData, SoundData, ByteData - the last one specially if you need misc mem areas for other purposes... you'll need to use the FFI though to write and read from them though.)

I did read about something regarding a luajit fork using a newer garbage collector that makes it possible to use a larger address space but i'm sure it's not included in löve yet... not sure if it can be.

[Madrayken]

Thanks. I feared that might be the case. I've seen external memory profilers (e.g. ones mentioned in https://www.lua.org/wshop15/Musa2.pdf) but I'd have no idea how to get them to work. For now I'm literally just watching the Activity Monitor app and watching my memory use suddenly jump to 1Gb making the same calls I've been using all over the place for a year. Fun day ahead... ack!

[pgimeno]

LuaJIT 2.1 (not a fork, the official one) supports much more memory. Circa 48 bits if my recollection is right. I don't think it's included with Löve yet, though.

[Xii]

If you are using Lua tables to store game information, like npc.health=100 or map[pos]=tile_id or whatever, you can significantly reduce your memory consumption by using FFI. Here is an article detailing how to do that. Using FFI you can define compact memory structures for values that simply do not need the full 64 bits range that regular Lua values have.

[grump]

https://github.com/pfirsich/jprof

[ivan]

Running out of Lua memory shouldn't happen unless you are doing something terribly wrong.
Make sure you set any unused references to nil otherwise the garbage collector can't release that memory.

[gcmartijn]

Make sure you set any unused references to nil otherwise the garbage collector can't release that memory.

I need to test this, but what about something like this.

Code: Select all

function add (data)
      local rawdata = data
      --- do things
      return something
    end
    
local x = add({bla})

Do we always need to set variable to nil for the garbage collector ?

Code: Select all

function add (data)
      local rawdata = data
      --- do things
      
      rawdata = nil
      data = nil -- the function to ?
      
      return something
    end
 
 local x = add({bla})

[Jeeper]

Iirc this has been discussed before, unfortunately there's no simple way to count all the main memory used by the application as far as i'm aware.

That said, LuaJIT has a memory limit (from 1-2GB on 64bit to 4GB on 32bit or something like that), whether or not you're intending to use the 32 or 64 bit version of löve. One option for really large mem requirements would be to use 64bit and löve-owned C++-side objects (all the Data objects, like ImageData, SoundData, ByteData - the last one specially if you need misc mem areas for other purposes... you'll need to use the FFI though to write and read from them though.)

I did read about something regarding a luajit fork using a newer garbage collector that makes it possible to use a larger address space but i'm sure it's not included in löve yet... not sure if it can be.

Did you miss type or does the 32bit version have support for more memory? That would make no sense?

[pgimeno]

Do we always need to set variable to nil for the garbage collector ?

No. Going out of scope is equivalent to setting it to nil.

Sometimes, Löve objects take a long time to be cleaned up by the GC, letting many of them to accumulate and take a lot of memory. In these cases, it's best to explicitly release them with Object:release. I wonder if that's what's happening here.

Did you miss type or does the 32bit version have support for more memory? That would make no sense?

The 1 GB limit in 64 bits applies to Linux.
https://stackoverflow.com/questions/351 ... -platforms
http://timetobleed.com/digging-out-the- ... egression/