Unit testing

General discussion about LÖVE, Lua, game development, puns, and unicorns.
Post Reply
User avatar
togFox
Party member
Posts: 764
Joined: Sat Jan 30, 2021 9:46 am
Location: Brisbane, Oztralia

Unit testing

Post by togFox »

Thinking about unit testing and what it means in LUA and love2d - and I'm not talking about syntax checking and other code sanitisers. I'm talking about ensuring functions return an expected outcome and can be immensely useful for regression reading and refactoring code (and understanding).

Google shows oodles of modules for LUA but thought I'd ask what ppl's experience is?

Can I just put some ASSERTS at the end of each function and call that unit testing?
Current project:
https://togfox.itch.io/backyard-gridiron-manager
American football manager/sim game - build and manage a roster and win season after season
MrFariator
Party member
Posts: 509
Joined: Wed Oct 05, 2016 11:53 am

Re: Unit testing

Post by MrFariator »

Writing asserts at the end of each function would be a pretty poor attempt at unit testing, and a possible performance hog at worst. For any decently complex function, the assert, or multiples of them, would become so mangled (and possibly redundant) that it can get pretty hard to reason with, or verify that each given output is correct.

At the end of the day unit testing is about writing, well, tests - little pieces of code that you run to check that your functions return correct values for given inputs. A single test may be as simple as passing some values x,y to a given function, and expect to receive the value z back. The rest of your codebase or project does not have to be aware that these tests even exist, and so, creating and running tests does not differ much between programming languages (beyond maybe some of the tooling you might use). As such, if you're not fully committing to unit testing (which generally isn't necessary for smaller indie game endeavors), your time is better spent on ensuring that the inputs your functions receive are correct and within permitted limits, as opposed to trying to check that result is correct after the fact with a bunch of asserts. Of course you'll have to check the output for correctness, but more often than not it's the inputs that may cause issues, particularly in a weakly typed language like lua.

I have written many unit tests and even done test driven development in the past, and they can be great tools. However, they can also be great time sinks in a single man project, and as such are generally suited better for larger projects with multiple people involved, or projects where bug-freeness is a major goal (think banking or other sensitive applications).
User avatar
pgimeno
Party member
Posts: 3541
Joined: Sun Oct 18, 2015 2:58 pm

Re: Unit testing

Post by pgimeno »

I agree with MrFariator. However, using asserts is still a good idea, even with unit tests in place. Whenever you expect a function to only receive certain kinds or ranges of parameters, you can insert an assert() to verify that. I don't use asserts on the results of functions, only on the inputs of either functions or smaller blocks of code.

But in Lua you must use them carefully, because unlike in other languages where it's a special construct, assert() is a regular function. The worst danger performance wise is in the error message actually:

http://lua.space/general/assert-usage-caveat

You can add this at the top of main.lua when you consider that the version is sufficiently tested: function assert() end; however that won't help if the performance sink is in the preparation of parameters. The best option is to make assert a macro and use a preprocessor: https://love2d.org/forums/viewtopic.php ... 49#p240849 - that way the whole call can be removed in the release version, including the parameters.

And back to unit tests, as for writing them, there are fancy tools such as Busted, but I just create a Lua file with asserts in it. Here's the unit test file for my WIP linear algebra library, to serve as example:

Code: Select all

local alg = require 'alg'

local function within(v1, v2, tol)
  return math.abs(v1 - v2) <= tol
end

local m1 = alg.new_mat4()
local m2 = alg.new_mat4{2,3,5,7, 11,13,17,19, 23,29,31,37, 41,43,47,53}
local v1 = alg.new_vec4()
local v2 = alg.new_vec4{59,61,67,71}
local v3

-- Test eq, as we rely on it
m1[1] = m2[1] + 1
for i = 1, 2 do
  alg.set_vec2(m1, m2)
  assert(alg.eq_vec2(m1, m2))
  assert(alg.eq_vec2(m2, m1))
  m1[i] = m1[i] + 0.0000001
  assert(not alg.eq_vec2(m1, m2))
  assert(not alg.eq_vec2(m2, m1))
  m1[i] = math.floor(m1[i])
  assert(alg.eq_vec2(m1, m2))
  assert(alg.eq_vec2(m2, m1))
end
m1[1] = m2[1] + 1
for i = 1, 3 do
  alg.set_vec3(m1, m2)
  assert(alg.eq_vec3(m1, m2))
  assert(alg.eq_vec3(m2, m1))
  m1[i] = m1[i] + 0.0000001
  assert(not alg.eq_vec3(m1, m2))
  assert(not alg.eq_vec3(m2, m1))
  m1[i] = math.floor(m1[i])
  assert(alg.eq_vec3(m1, m2))
  assert(alg.eq_vec3(m2, m1))
end
m1[1] = m2[1] + 1
for i = 1, 4 do
  alg.set_vec4(m1, m2)
  assert(alg.eq_vec4(m1, m2))
  assert(alg.eq_vec4(m2, m1))
  m1[i] = m1[i] + 0.0000001
  assert(not alg.eq_vec4(m1, m2))
  assert(not alg.eq_vec4(m2, m1))
  m1[i] = math.floor(m1[i])
  assert(alg.eq_vec4(m1, m2))
  assert(alg.eq_vec4(m2, m1))
end
assert(alg.eq_mat2 == alg.eq_vec4)
assert(alg.set_mat2 == alg.set_vec4)
m1[1] = m2[1] + 1
for i = 1, 3*3 do
  alg.set_mat3(m1, m2)
  assert(alg.eq_mat3(m1, m2))
  assert(alg.eq_mat3(m2, m1))
  m1[i] = m1[i] + 0.0000001
  assert(not alg.eq_mat3(m1, m2))
  assert(not alg.eq_mat3(m2, m1))
  m1[i] = math.floor(m1[i])
  assert(alg.eq_mat3(m1, m2))
  assert(alg.eq_mat3(m2, m1))
end
m1[1] = m2[1] + 1
for i = 1, 4*4 do
  alg.set_mat4(m1, m2)
  assert(alg.eq_mat4(m1, m2))
  assert(alg.eq_mat4(m2, m1))
  m1[i] = m1[i] + 0.0000001
  assert(not alg.eq_mat4(m1, m2))
  assert(not alg.eq_mat4(m2, m1))
  m1[i] = math.floor(m1[i])
  assert(alg.eq_mat4(m1, m2))
  assert(alg.eq_mat4(m2, m1))
end

-- vec2/mat2 tests

-- Expected values were calculated with iSymPy and Octave

alg.mul_mat2mat2(m1, alg.id_mat2, m2)
assert(alg.eq_mat2(m1, {2,3,5,7}))

alg.mul_mat2mat2(m1, m1, {11,13,17,19})
assert(alg.eq_mat2(m1, {73,83,174,198}))

alg.inplace_transpose_mat2(m1)
assert(alg.eq_mat2(m1, {73,174,83,198}))

alg.set_vec2(v2, {23,29,31,37})
alg.mul_vec2mat2(v1, v2, m1)
assert(alg.eq_vec2(v1, {4086,9744}))

alg.neg_vec2(v1, v1)
assert(alg.eq_vec2(v1, {-4086,-9744}))

alg.clear_vec2(v1)
assert(alg.eq_vec2(v1, {0,0}))

alg.mul_mat2vec2(v1, m1, v2)
assert(alg.eq_vec2(v1, {6725,7651}))

assert(alg.dot_vec2(v1, v2) == 376554)
assert(alg.cross_vec2(v1, v2) == 19052)

alg.add_vec2(v1, v2, {5,7})
assert(alg.eq_vec2(v1, {28,36}))

alg.mul_vec2vec2(v1, v2, {5,7})
assert(alg.eq_vec2(v1, {115,203}))

alg.scale_vec2(v1, v2, 2)
alg.set_vec2(v2, {46,58})
assert(alg.eq_vec2(v1, v2))

alg.set_vec2(v1, {0.5490196078431373, 0.34509803921568627})
alg.set_vec2(v2, {0.8156862745098039, 0.19215686274509805})
alg.composite_vec2vec2(v1, v1, v2)
assert(within(v1[1], 0.62027759475928960, 1e-13)
   and within(v1[2], 0.47094194540561324, 1e-13))

alg.scale_mat2(m1, m1, 3)
assert(alg.eq_mat2(m1, {219,522,249,594}))

alg.add_mat2(m1, m1, {1,2,3,4})
assert(alg.eq_mat2(m1, {220,524,252,598}))

alg.rot_mat2(m1, m1, math.pi/2)
assert(within(m1[1], -252, 1e-13)
   and within(m1[2], -598, 1e-13)
   and within(m1[3],  220, 1e-13)
   and within(m1[4],  524, 1e-13))

alg.seti_mat2(m1)
assert(alg.eq_mat2(m1, alg.id_mat2))

-- TODO
-- seti_mat2
-- rot_mat2
-- det_mat2
-- invert_mat2


-- vec3/mat3 tests

m1 = alg.new_mat3()
m2 = alg.new_mat3({2,3,5,7,11,13,17,19,23})
alg.mul_mat3mat3(m1, m2, {29,31,37,41,43,47,53,59,61})
assert(alg.eq_mat3(m1, {446,486,520,1343,1457,1569,2491,2701,2925}))

alg.inplace_transpose_mat3(m1)
assert(alg.eq_mat3(m1, {446,1343,2491,486,1457,2701,520,1569,2925}))

v1 = alg.new_vec3(13, 29, 41)
assert(alg.eq_vec3(v1, {13, 29, 41}))
alg.neg_vec3(v1, v1)
assert(alg.eq_vec3(v1, {-13, -29, -41}))
alg.clear_vec3(v1)
assert(alg.eq_vec3(v1, {0,0,0}))

alg.set_vec3(v1, {13, 29, 41})
v2 = alg.new_vec3(5, 61, 97)
assert(alg.dot_vec3(v1, v2) == 5811)

alg.cross_vec3(v1, v1, v2)
assert(alg.eq_vec3(v1, {312, -1056, 648}))

alg.scale_vec3(v1, v1, 0.125)
assert(alg.eq_vec3(v1, {39, -132, 81})) 

alg.add_vec3(v1, v1, v2)
assert(alg.eq_vec3(v1, {44, -71, 178}))

alg.mul_vec3vec3(v1, v1, v2)
assert(alg.eq_vec3(v1, {44*5, 61*-71, 97*178}))

alg.cross_vec3(v1, alg.y_vec3, alg.x_vec3)
alg.neg_vec3(v2, alg.z_vec3)
assert(alg.eq_vec3(v1, v2))


-- vec4 tests

v1 = alg.new_vec4()
v2 = alg.new_vec4()
v3 = alg.new_vec4()
alg.set_vec4(v1, {0, 0, 0.5490196078431373, 0.34509803921568627})
alg.set_vec4(v2, {0, 0, 0.8156862745098039, 0.19215686274509805})
alg.composite_vec4vec4(v3, v1, v2)
assert(within(v3[3], 0.62027759475928960, 1e-13)
   and within(v3[4], 0.47094194540561324, 1e-13)
   and v3[1] == 0 and v3[2] == 0)
v1[2] = v1[3]; v1[3] = 0
v2[2] = v2[3]; v2[3] = 0
alg.composite_vec4vec4(v3, v1, v2)
assert(within(v3[2], 0.62027759475928960, 1e-13)
   and within(v3[4], 0.47094194540561324, 1e-13)
   and v3[1] == 0 and v3[3] == 0)
v1[1] = v1[2]; v1[2] = 0
v2[1] = v2[2]; v2[2] = 0
alg.composite_vec4vec4(v1, v1, v2)
assert(within(v1[1], 0.62027759475928960, 1e-13)
   and within(v1[4], 0.47094194540561324, 1e-13)
   and v1[2] == 0 and v1[3] == 0)

alg.clear_vec4(v1)
assert(v1[1] == 0 and v1[2] == 0 and v1[3] == 0 and v1[4] == 0)
v1 = alg.new_vec4({-2, 5, 7, -9})
alg.neg_vec4(v1, v1)
assert(v1[1] == 2 and v1[2] == -5 and v1[3] == -7 and v1[4] == 9)
alg.set_vec4(v2, {13, 19, 23, 29})

alg.mul_vec4vec4(v3, v1, v2)
assert(alg.eq_vec4(v3, {2*13,-5*19,-7*23,9*29}))
assert(alg.dot_vec4(v1, v2) == v3[1]+v3[2]+v3[3]+v3[4])

alg.add_vec4(v3, v1, v2)
assert(alg.eq_vec4(v3, {15,14,16,38}))

alg.scale_vec4(v1, v1, 2)
assert(alg.eq_vec4(v1, {4, -10, -14, 18}))


-- mat4 tests

m1 = alg.new_mat4()
m2 = alg.new_mat4{59,61,67,71,73,79,83,89,97,101,103,107,109,113,127,131}
alg.mul_mat4mat4(m1, {2,3,5,7,11,13,17,19,23,29,31,37,41,43,47,53}, m1)
alg.mul_mat4mat4(m1, m1, m2)

assert(alg.eq_mat4(m1, {1585,1655,1787,1861,5318,5562,5980,6246,
  10514,11006,11840,12378,15894,16634,17888,18710}))

alg.inplace_transpose_mat4(m1)
alg.set_mat4(m2, {1585,5318,10514,15894,1655,5562,11006,16634,
  1787,5980,11840,17888,1861,6246,12378,18710})
assert(alg.eq_mat4(m1, m2))

-- Invert in place
alg.invert_mat4(m1, m1)
assert(alg.eq_mat4(m1, {2371/19404, -4601/19404, 5/63, 151/4851,
  3191/129360, 12347/129360, -2/105, -2833/32340,
  -877/4410, 3149/17640, 59/630, -1399/17640,
  8629/77616, -9809/77616, -4/63, 3053/38808}))


m1, m2, v1, v2, v3 = nil
print("All passed")
I've caught a few typos with it. Unfortunately, writing them took long enough as to give up on continuing with the library, which supports what MrFariator said. Writing good tests is about knowing how the functions may fail, and try to write worst cases. I even caught a bug in Python while working on the unit tests for a standard C library I was writing. But they are also time-consuming.
User avatar
togFox
Party member
Posts: 764
Joined: Sat Jan 30, 2021 9:46 am
Location: Brisbane, Oztralia

Re: Unit testing

Post by togFox »

I've installed and got this one working:

https://github.com/hawkthorne/lovetest

and by working I mean working for love 11.3. I now have no idea how to actually use this. The github assumes ppl know - and fair enough - I can't blame them for that. Lovetest uses lunatest so I read about lunatest but that is too low level.

Lovetest wants me to write tests in a seperate test file (makes sense). Do I use that test file to REQUIRE my modules, invoke modules then somehow assert different values?
Current project:
https://togfox.itch.io/backyard-gridiron-manager
American football manager/sim game - build and manage a roster and win season after season
MrFariator
Party member
Posts: 509
Joined: Wed Oct 05, 2016 11:53 am

Re: Unit testing

Post by MrFariator »

If you read the github page, it gives you exact details on how to use it.

1. Download the repo, and put the contained "test" folder into your project's root (next to your main.lua)
2. Require lovetest as instructed in in the Usage section
3. Invoke the run() function at your leisure while your love2d project is running. You could put it at the end of your love.load, make it run whenever you press a specific key on your keyboard, or wherever else.
4. Once run() is executed, it will run the tests contained in the "test" folder (where lovetest also resides)

So, to make a simple test, you could create a test_myTest.lua file, and put these as its contents:

Code: Select all

function test_assert ()
	assert_equal ( 1 + 1, 2 )
end
local myLib = require "path/to/myLib"
function test_myLib ()
	assert_equal ( myLib.thisFunctionReturnsTheNumberTwo(), 2 )
end
For further examples, check lunatest samples.
User avatar
togFox
Party member
Posts: 764
Joined: Sat Jan 30, 2021 9:46 am
Location: Brisbane, Oztralia

Re: Unit testing

Post by togFox »

Thanks. How are those two examples useful and what do I learn from those examples that can make me write more meaningful tests?
Current project:
https://togfox.itch.io/backyard-gridiron-manager
American football manager/sim game - build and manage a roster and win season after season
MrFariator
Party member
Posts: 509
Joined: Wed Oct 05, 2016 11:53 am

Re: Unit testing

Post by MrFariator »

The first example function in my code isn't useful, just an example of an assert using lovetest. The second one is more useful in that you're checking that the function returns what it promises. And that latter is really what unit testing is about: You pass some values to the functions defined in your lua files, and check that they return correct values - or react accordingly if you pass incorrect parameters. You're trying to test that your functions are robust and don't break if you throw weird things at them, and try to fix any weird corner cases that might occur (rounding errors, weird boolean conversions, table structures, etc).

An example is a function that expects that it's given a number via its arguments (perhaps even within a specific range of numbers), that then adds to or substracts from that number, and then returns it. Your unit tests for that function could be along the lines of testing that it doesn't crash if you pass nothing, a string, a boolean, or an out-of-range number.

That's really all there is to it.
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 22 guests