Managing memory for a ton of pictures

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
User avatar
Sasha264
Party member
Posts: 131
Joined: Mon Sep 08, 2014 7:57 am

Re: Managing memory for a ton of pictures

Post by Sasha264 »

BrotSagtMist wrote: Sun Feb 05, 2023 1:37 am The problem purely lies in that i do not know HOW MUCH i can preload.
Ohhh.... I feel that pain :cry:
I was struggling to solve that problem in 2019 viewtopic.php?f=4&t=86986
slime wrote: Sat Aug 10, 2019 12:11 pm Unfortunately you can't.
...aaand there was no solution except "use some predefined limit" like lets say 2GB. Which is bad. Bad bad bad.

One note: what you are looking for is the available VRAM amount, not RAM. Because images (or canvases) stored there. All of my projects with a lot of images uses a lot of VRAM, but only < 1GB RAM usually.
Back in the day I had gpu with 11 GB VRAM, now I have 16, and I can fill it all with generated images or some other sh*t in couple of seconds like it is nothing.
It should run on a 128mb potato. But at the same time uses a 128gb monster to its fullest.
Exactly! That is the proper way!

Sometime ago I followed some tutorials for vulkan. One of them: https://vulkan-tutorial.com/
And there are functions to get available VRAM amount, along with some other amounts, like RAM, like memory that can be accessed from cpu and from gpu and some other complicated stuff. And not only "there are", but this is basic need to make anything. You need that before you draw your first triangle. Because to draw something you clearly need some buffers, and how can you create any buffer without firstly check if it will fit in? (sarcasm) :rofl: But for simplicity that step is omitted in love2d. That is the case well known as happy case programming, like "of course it will fit in". That is reasonable default behavior for game engine, because it protects newcomers from these complicated problems. Imagine if before every love.graphics.newImage() you would need to check available VRAM :o: But not to have that mechanism entirely... That is pain.

There is a lot to be said about "Why do you need that?" "Are you sure you need that?" "Maybe you can optimize something?" But all that is secondary. The main question is "How much memory you have?" So you can juggle your stuff inside that limit.
It is really depressing that this testing till it fails, as darkfrei just wrote, is the only solution we can come up with.
It is depressing. I was depressed 4 years ago :emo: Now in love projects I use "better ask for forgiveness than permission" principle. As a workaround you can allocate VRAM with a step of 256 MB or something reasonable like that, and see when it fails. Do that once, when program starts for the first time on a new device and save it in some file. And after use only ~70% of that amount. Or something like that. That is not ideal, that have some caveats, that is not satisfying at all, but it will work at least somehow.
RNavega
Party member
Posts: 239
Joined: Sun Aug 16, 2020 1:28 pm

Re: Managing memory for a ton of pictures

Post by RNavega »

If you look at how AAA games get away with it, they support a range of hardware. They have a "minimum" requirement of some VRAM, and a "recommended/highest" requirement with lots of it.

Then in the game settings they have a Texture Quality slider, any lower quality will cause textures to be downsized before being uploaded to the GPU. The game is shipped with textures at their highest resolution, but that's on disk. Before sending them to the GPU they can be downsized depending on that setting.
User avatar
slime
Solid Snayke
Posts: 3131
Joined: Mon Aug 23, 2010 6:45 am
Location: Nova Scotia, Canada
Contact:

Re: Managing memory for a ton of pictures

Post by slime »

Sasha264 wrote: Thu Feb 09, 2023 10:18 am Sometime ago I followed some tutorials for vulkan. One of them: https://vulkan-tutorial.com/
And there are functions to get available VRAM amount, along with some other amounts, like RAM, like memory that can be accessed from cpu and from gpu and some other complicated stuff. And not only "there are", but this is basic need to make anything. You need that before you draw your first triangle. Because to draw something you clearly need some buffers, and how can you create any buffer without firstly check if it will fit in? (sarcasm) :rofl: But for simplicity that step is omitted in love2d.
LÖVE 11 doesn't use Vulkan, it uses OpenGL. :)

OpenGL and Vulkan have very different memory models. In OpenGL, graphics memory allocations are all handled by the graphics driver, individual apps just make requests. The driver may allocate more (or less) memory than requested, depending on what it thinks is best. It may allocate from whichever memory pool it thinks is best. It may temporarily shuffle some app-requested graphics memory over to main memory to prevent a real out-of-VRAM situation. The graphics driver is a single place that manages memory for all running OpenGL apps at the same time. Many systems have an integrated GPU which shares its memory with system RAM, in which case the graphics driver can dynamically expand and shrink the amount of memory available to the GPU at runtime, depending on how much system RAM is being used and other metrics.

OpenGL does not provide any API an app can use to query available or total graphics memory that will work on all systems, because it doesn't really make sense with OpenGL's memory model.

With Vulkan, each app needs to have its own memory allocation code - basically every single Vulkan program has to implement its own version of malloc() and free() for graphics memory. There isn't much information about allocations that gets shared between Vulkan apps, it's very easy for one app to have a big chunk of graphics memory it mostly isn't using but it prevents another running app from being able to allocate the memory it needs. Since Vulkan apps are expected to allocate big chunks from the system and then frequently sub-allocate from those (unlike OpenGL, and unlike love's Lua APIs), the Vulkan API needs to provide some memory metrics - but that doesn't mean those metrics are particularly accurate especially when there are multiple apps running at once on a user's system.

---

What RNavega said is generally the way to do it. The bigger games I've worked on at my day job all do something similar.

There's often a lot of graphics memory (texture memory especially) optimizations one can do to avoid using a lot more VRAM than is actually necessary. Everything from avoiding loading a texture that isn't being used, to making sure something's base size isn't bigger than the number of pixels it will take on-screen when it's being drawn, to using compressed texture formats like what people mentioned earlier in the thread. A DXT1 compressed texture will take 1/8 (!) of the space in VRAM as a PNG of the same dimensions, and it will also be more performant when rendering it.
User avatar
Sasha264
Party member
Posts: 131
Joined: Mon Sep 08, 2014 7:57 am

Re: Managing memory for a ton of pictures

Post by Sasha264 »

@slime, thank you for the excurce in OpenGL memory management. It is quite informative! I didn't know that with OpenGL graphic driver can shift something from vram to main memory on its own. Wow! :crazy: About several programs running simultaneously on the same device: yes, it is a problem. But the common assumption is that if the program is a game, then it can use all ok, not all, just 80% and only if it is focused available gpu-cpu rams-and-processing-powers while it is running. Simply because it is designed to use all of user's attention, so he would not need to run another game, for example :ehem: So, if a user wants to run several games, it is a problem, but it is his problem. Our problem is to make sure that our games run fine in the most common environments and use reasonably all available benefits.

About vulkan memory management with huge chunks and secondary allocations: there is a thing called VulkanMemoryAllocator. It is not part of vulkan itself but it is made by AMD and it does all the heavy lifting for you, so you don't have to manage insanely complex vulkan memory model by hands :nyu: But that has nothing to do with the current topic, sorry :rofl: The first reason why I mentioned vulkan was not because it has something to do with Love, but just as an example with what I was happy :3

As for the slider with something like "texture quality" or "vram usage" it is a good think, because only end user can know for sure how many other stuff he will run alongside with your game/program, and what combination of performance/quality he wants. But it should be fail safe around upper limit. So if user have potato device and his slider somehow ended up being on the high side I would prefer that the game somehow would know about the problem and do not chaw up more memory then device has. In my case from 2019 thread OpenGL managed to allocate ~13GB "video" memory when I only had 11 without any errors or warnings, and this led to unbearable consequences. It would be good to know how to detect this situation in advance.
User avatar
slime
Solid Snayke
Posts: 3131
Joined: Mon Aug 23, 2010 6:45 am
Location: Nova Scotia, Canada
Contact:

Re: Managing memory for a ton of pictures

Post by slime »

Yes, OpenGL does not provide that information to apps. Some specific graphics drivers have an extension capability to query total VRAM estimates, but many don't.

It gets even trickier with integrated GPUs, because you're also sharing that total amount of memory with... your own CPU-side code and whatever it allocates. In love 11 and older, in order for love.graphics.setMode to work without forcing you to manually recreate every image, every Image object holds onto a CPU-side copy of its memory. So there's much less total available "graphics memory" than a system-reported value would indicate when running on an integrated GPU.
Sasha264 wrote: Thu Feb 09, 2023 3:24 pm About vulkan memory management with huge chunks and secondary allocations: there is a thing called VulkanMemoryAllocator. It is not part of vulkan itself but it is made by AMD and it does all the heavy lifting for you, so you don't have to manage insanely complex vulkan memory model by hands :nyu:
Yeah, love 12's experimental work-in-progress Vulkan backend uses it. :)
User avatar
Sasha264
Party member
Posts: 131
Joined: Mon Sep 08, 2014 7:57 am

Re: Managing memory for a ton of pictures

Post by Sasha264 »

slime wrote: Thu Feb 09, 2023 4:01 pm Some specific graphics drivers have an extension capability to query total VRAM estimates, but many don't.
That would be a helpful feature, even though it gets nothing in half of the cases. Someone could write something like that:

Code: Select all

local vram = love.graphics.getStats().vramestimate or 1e9
(^-^)"
It gets even trickier with integrated GPUs...
I guess it's ok if at some point you have to treat integrated gpus differently. Not only in the engine but also in a particular game. Too many differences:
  • Floating amount of memory.
  • Faster copy between cpu and gpu memory. For example this SpriteBatchUsage can create up to 50 times performance difference between static and stream for a discrete gpu, and almost no difference for integrated gpu. Or this Canvas:newImageData should be faster on integrated gpu.
  • Slower shader performance in general.
  • All gpu memory is accessible to cpu. So again about vulkan -_- in vulkan you do not need that bunch of "staging" buffers.
Yeah, love 12's experimental work-in-progress Vulkan backend uses it. :)
That is soooo cool :3 Can not wait to test it :awesome:
User avatar
BrotSagtMist
Party member
Posts: 607
Joined: Fri Aug 06, 2021 10:30 pm

Re: Managing memory for a ton of pictures

Post by BrotSagtMist »

Ah thanks, at least someone with the same problem.
Thats helpful info.
slime wrote: Thu Feb 09, 2023 1:14 pm A DXT1 compressed texture will take 1/8 (!) of the space in VRAM as a PNG of the same dimensions, and it will also be more performant when rendering it.
Thats nice and all, but i still cant find info on how to work with that anywhere.
obey
User avatar
zorg
Party member
Posts: 3436
Joined: Thu Dec 13, 2012 2:55 pm
Location: Absurdistan, Hungary
Contact:

Re: Managing memory for a ton of pictures

Post by zorg »

BrotSagtMist wrote: Sun Feb 12, 2023 11:58 pm
slime wrote: Thu Feb 09, 2023 1:14 pm A DXT1 compressed texture will take 1/8 (!) of the space in VRAM as a PNG of the same dimensions, and it will also be more performant when rendering it.
Thats nice and all, but i still cant find info on how to work with that anywhere.
Work how?
How/with what application to create them / convert into them from other images?
Or do you mean format specification documentation so you can write code to do that within löve itself?
Me and my stuff :3True Neutral Aspirant. Why, yes, i do indeed enjoy sarcastically correcting others when they make the most blatant of spelling mistakes. No bullying or trolling the innocent tho.
User avatar
BrotSagtMist
Party member
Posts: 607
Joined: Fri Aug 06, 2021 10:30 pm

Re: Managing memory for a ton of pictures

Post by BrotSagtMist »

Booth maybe.
Are there tools to convert for example?
obey
User avatar
zorg
Party member
Posts: 3436
Joined: Thu Dec 13, 2012 2:55 pm
Location: Absurdistan, Hungary
Contact:

Re: Managing memory for a ton of pictures

Post by zorg »

BrotSagtMist wrote: Mon Feb 13, 2023 12:21 am Booth maybe.
Are there tools to convert for example?
Gimp and Photoshop (and apparently paint.net as well) support exporting as dds files, not sure how many of the dxt algorithms they support though.

Wikipedia has article ("S3 Texture Compression") that explains things, although no code examples are given.
Me and my stuff :3True Neutral Aspirant. Why, yes, i do indeed enjoy sarcastically correcting others when they make the most blatant of spelling mistakes. No bullying or trolling the innocent tho.
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 45 guests