simple threaded resource loader for next incarnation of love

Showcase your libraries, tools and other projects that help your fellow love users.
Post Reply
giniu
Party member
Posts: 221
Joined: Mon Nov 30, 2009 4:44 pm

simple threaded resource loader for next incarnation of love

Post by giniu »

As some of you noticed recently I was playing with threads. Was doing lots of different things to try break it, but wasn't able to :) Anyway, during the process, I created simple resource loading library I decided to share with others. It's a starting point so you might do with it whatever you like, probably try it and find bugs in it or in stuff it uses :)

I'd like to mention, that this library is for current development version (at least ee2a21493ea2 from 9 days ago is required) and might not work in released 0.7 version - it depends on multiple factors, especially on the very existence of love.thread in next release :)

But to description of what it is. I will describe api I decided to use and then some details maybe.

resource.init(name) - if name is omitted, name = "_resource_worker_". It created new worker thread, that waits for loading requests. It does not consume cpu when it is idle (request functions takes responsibility to wake the worker thread when needed)

resource.finish() - kills the worker thread. Not really needed usually, but it kills worker thread.

resource.requestResource(type, filename) - sends request to load userdata of given type from file with given filename. The requests are queued in FIFO queue, on first request to idle worker, it is restarted. Possible type are "image", "sound" and "file", which loads ImageData, SoundData and FileData. The requested data is loaded and kept in worker thread until it is probed (checked if file is loaded or request to return userdata). There are also three functions: resource.requestImage(filename), resource.requestSound(filename) and resource.requestFile(filename) which are special cases of above.

resource.ready(filename) - checks if resource is ready, returns true or false

resource.get(filename) - returns userdata if loaded, or nil otherwise

resource.getProgress() - returns number from 0 to 1 - it counts proportion of number of loaded and ready to be received with resource.get resources, to whole number of requested resources.

And that's all. About design - you cannot request same file to load as two different file types, but you can request same file few times. resource library ref-counts your request/get commands, and keeps the resource in table in main thread if it was requested more than once. When resource is recieved (resource.get) requested exactly same number of times as it was requested, reference to image is removed from main thread. This have some consequences. First of all, user must keep reference to loaded images, in one way or another, just as if they were loaded with simple NewImageData etc. Second, in case of multiple requests, we send image only once between threads. It's because threads do not share data - sending same thing over and over would lower the performance. The library can be extended to support "often requested resources", but it wasn't the point - it was made to see how threading works :)

Other design elements? Well, it's all with message passing. When worker thread is started, it awaits for message "get_ready" from main thread. When it gets it, it sends ("sleeping",false) and enters process_requests(). Then it checks for message "ask1", if it isn't available, it finishes process_requests, but if it is, it contains resource type - say "image" - then it loads "image1" where the filename for our image is. It uses love.image.newImageData to load it, and sends message (filename, data) to itself, then bumps last message by one. Not it checks for "ask2", etc. When all requests are processed, it sets last index to zero so next one will be one again, and informs main thread that it went to sleep again. Then it awaits for new "get_ready" request. The synchronization of id's is important part, but it's not as hard to do - i.e., if worker is sleeping we reset the id. Possible race condition would be when "ask1" is sent but "image1" isn't, that's why if worker process notices sending is in progress, it awaits for end of sending. It's also handled automatically. And finally - we the resource library can be used only from main thread. That's only to simplify things so multiple threads don't fight for resources.

I think I wrote enough. It's simple code with possible bugs, but still it proves that threads are working quite nice. Have fun :)
User avatar
bartbes
Sex machine
Posts: 4946
Joined: Fri Aug 29, 2008 10:35 am
Location: The Netherlands
Contact:

Re: simple threaded resource loader for next incarnation of

Post by bartbes »

giniu wrote:It's because threads do not share data - sending same thing over and over would lower the performance.
This is not entirely true, love's userdata is shared (as in, it's not copied, internally they point to the same), however, it holds true for some part, as for every time you send it a message is created, which on the other end is turned back into a 'proxy', which is basically a wrapper, so while the resource isn't loaded twice, it is 'introduced' to lua twice.
Last edited by bartbes on Fri Aug 13, 2010 1:28 pm, edited 1 time in total.
Reason: Quote fail
Post Reply

Who is online

Users browsing this forum: Ahrefs [Bot] and 62 guests