Shader variables - documentaiton and get the transformed vertext position

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
Post Reply
zell2002
Citizen
Posts: 75
Joined: Sun Feb 23, 2014 9:22 pm

Shader variables - documentaiton and get the transformed vertext position

Post by zell2002 » Thu Sep 26, 2019 6:54 pm

Hey guys,
https://love2d.org/wiki/Shader_Variables
From there, I don't fully get how to access the global variables. Are there any examples anywhere ?
Also the variable types, there's no explicit write up? vec4 I assume has .x and .y, but I dont know what the 3rd and 4th are. I assume z and then....?

I looked up the mat4, and assume this is a matrix with 4 columns and 4 rows. But is it accessed as an array? And what are the objects/items/properties at each row/column ?

I apologise if I've missed some documentation somewhere! Just very eager to have a play with the vertex shader stuff but not sure what all the variables hold and how to access it.


*Edit*
Sorry I should say what I'm trying to do : the vertex_position is the position before translate etc, but I need the transformed position - which i assume is in the TransformMatrix ?

Cheers

User avatar
pgimeno
Party member
Posts: 1885
Joined: Sun Oct 18, 2015 2:58 pm
Location: Valencia, ES

Re: Shader variables - documentaiton and get the transformed vertext position

Post by pgimeno » Fri Sep 27, 2019 12:34 am

zell2002 wrote:
Thu Sep 26, 2019 6:54 pm
Hey guys,
https://love2d.org/wiki/Shader_Variables
From there, I don't fully get how to access the global variables. Are there any examples anywhere ?
Also the variable types, there's no explicit write up? vec4 I assume has .x and .y, but I dont know what the 3rd and 4th are. I assume z and then....?

I looked up the mat4, and assume this is a matrix with 4 columns and 4 rows. But is it accessed as an array? And what are the objects/items/properties at each row/column ?

I apologise if I've missed some documentation somewhere! Just very eager to have a play with the vertex shader stuff but not sure what all the variables hold and how to access it.
There's no explicit write up, because the shader language is GLSL (the only variations are that it defines a few macros and requires the code to be in the effect() and position() functions instead of main()), so the GLSL documentation applies to LÖVE just fine.

You have documentation covering your questions about vec4's and mat4's here:

https://www.khronos.org/opengl/wiki/Data_Type_(GLSL)

zell2002 wrote:
Thu Sep 26, 2019 6:54 pm
*Edit*
Sorry I should say what I'm trying to do : the vertex_position is the position before translate etc, but I need the transformed position - which i assume is in the TransformMatrix ?
From the wiki (love.graphics.newShader):
If no vertex shader code is used, LÖVE uses a default. This is its code:

Code: Select all

vec4 position(mat4 transform_projection, vec4 vertex_position)
{
    // The order of operations matters when doing matrix multiplication.
    return transform_projection * vertex_position;
}
So the transformed position is the result of multiplying the transform_projection matrix argument with the vertex_position vector argument (in that order).

zell2002
Citizen
Posts: 75
Joined: Sun Feb 23, 2014 9:22 pm

Re: Shader variables - documentaiton and get the transformed vertext position

Post by zell2002 » Fri Sep 27, 2019 6:53 am

Cheers for that. I did look over that wiki page before posting, as I said I'm unsure what the items are in mat4 list - as there's a possible 4 different objects in there?

Thanks for explaining the transformation, seems very obvious now,

User avatar
pgimeno
Party member
Posts: 1885
Joined: Sun Oct 18, 2015 2:58 pm
Location: Valencia, ES

Re: Shader variables - documentaiton and get the transformed vertext position

Post by pgimeno » Fri Sep 27, 2019 10:37 am

zell2002 wrote:
Fri Sep 27, 2019 6:53 am
Cheers for that. I did look over that wiki page before posting, as I said I'm unsure what the items are in mat4 list - as there's a possible 4 different objects in there?
When it comes to addressing components, a mat4 behaves effectively as if it were an array of four column vectors.

Vectors in turn can be addressed with array syntax, or with .x, .y, .z and .w, or equivalently with .r, .g, .b, .a, or equivalently with .s, .t, .p, .q; you can also form vectors of 2, 3 or 4 components by concatenating letters of the components (without mixing types). For example, if myVec4 is a vec4, myVec4.xyz is a vec3 with the components x, y and z of myVec4, and myVec4.rgb is the same thing, but myVec4.xgs is not valid because you can't mix them. Order matters: myVec4.zyx is a vec3 with the components in reverse order. You can duplicate components: myVec4.xxxx is another vec4 made with four copies of the first component of myVec4.

For example:

Code: Select all

mat4 myMat4;
if (myMat4[0].x == myMat4[0][0]) /* always true */ ;
vec4 myVec4 = myMat4[0] /* sets a 4D vector to the first column in the matrix */ ;
if (myVec4.grab == myMat4[0].yxwz) /* always true */ ;
vec2 myVec2 = myMat4[1].wz /* sets a 2D vector to components fourth and third of second column */ ;
if (myVec2.x == myMat4[1][3]) /* always true */ ;
if (myVec2.y == myMat4[1].z) /* always true */ ;

zell2002
Citizen
Posts: 75
Joined: Sun Feb 23, 2014 9:22 pm

Re: Shader variables - documentaiton and get the transformed vertext position

Post by zell2002 » Fri Sep 27, 2019 5:42 pm

Thanks for all that, I had no idea thats how it behaved.
Its pretty neat being able to do:

Code: Select all

	vec4 bar = vec4(0,1,0,1);
	vec2 foo = bar.xy;
mat4 TransformProjectionMatrix
The combined transform and projection matrices. Used as the transform_projection argument to the vertex shader position function.
So I know a mat4 is an array, but I'm unsure of what each item is in there for this VERTEX position() function:
transform_projection[0] = ?
transform_projection[1] = ?
transform_projection[2] = ?
transform_projection[3] = ?

I assume the first is the original position before transform..?
This is where I'm not following what each item is (in the array).

I aplogise if Ive misunderstood what you said.

User avatar
pgimeno
Party member
Posts: 1885
Joined: Sun Oct 18, 2015 2:58 pm
Location: Valencia, ES

Re: Shader variables - documentaiton and get the transformed vertext position

Post by pgimeno » Sat Sep 28, 2019 12:18 am

zell2002 wrote:
Fri Sep 27, 2019 5:42 pm
So I know a mat4 is an array,
Ummm, no. A mat4 can be accessed as if it were an array, but it is a matrix. As such, it can transform vectors. You can't multiply an array of four vec4's and a vec4, but you can multiply a mat4 and a vec4.

zell2002 wrote:
Fri Sep 27, 2019 5:42 pm
but I'm unsure of what each item is in there for this VERTEX position() function:
transform_projection[0] = ?
transform_projection[1] = ?
transform_projection[2] = ?
transform_projection[3] = ?
It's a bit complicated. It combines the transformation matrix that transforms LÖVE vertices according to the given scale, rotation, translation and shear (with the caveat about automatic batching mentioned in Shader_Variables), with the projection matrix that transforms LÖVE coordinates to OpenGL screen coordinates. If you really want a description, be ready for a long explanation.

So, it all depends on what you want to obtain. If you want the OpenGL screen coordinates of the vertices, note that these are numbers between -1.0 and 1.0 when they lie in the screen, and it's uncommon for these to be useful directly. If you want them expressed in LÖVE's coordinate system, you can use the TransformMatrix alone to transform the vertex:

Code: Select all

vec4 position(mat4 transform_projection, vec4 vertex_position)
{
  vec4 v = TransformMatrix * vertex_position;
  // v is now in LÖVE screen coordinates with (0, 0) at top left and (Width, Height) at bottom right.

  // Use v here for something (not sure what you want to do with it).

  // Convert v to OpenGL screen coordinates where (-1, -1) is bottom left and (1, 1) is top right,
  // so it can be displayed:
  return ProjectionMatrix * v;
}

zell2002
Citizen
Posts: 75
Joined: Sun Feb 23, 2014 9:22 pm

Re: Shader variables - documentaiton and get the transformed vertext position

Post by zell2002 » Sun Oct 13, 2019 9:46 pm

Thank you for that explanation! Makes a lot more sense.

The transform_projectio variable passed in, that is a mat4, are you saying each item that you mention "It combines the transformation matrix that transforms LÖVE vertices according to the given scale, rotation, translation and shear" as a seperate item in there ?

Like if i could just print(transform_projection); - what would it show me?
Is it possible to have this kind of debugging here ?

It seems so interesting, what you can do with shaders, (pixel and vertex), its really helped me get a better understanding of how people make waves (what im doing now) and lighting systems (something il never be smart enough to do lol)

User avatar
pgimeno
Party member
Posts: 1885
Joined: Sun Oct 18, 2015 2:58 pm
Location: Valencia, ES

Re: Shader variables - documentaiton and get the transformed vertext position

Post by pgimeno » Mon Oct 14, 2019 3:59 am

zell2002 wrote:
Sun Oct 13, 2019 9:46 pm
The transform_projectio variable passed in, that is a mat4, are you saying each item that you mention "It combines the transformation matrix that transforms LÖVE vertices according to the given scale, rotation, translation and shear" as a seperate item in there ?
No. OpenGL matrices are 4x4 (that's 16 floats, which is what you'd get if you could print them). But for 2d operations, LÖVE is really only using 6 elements of each matrix. Two of them are an offset; the other four form a 2x2 sub-matrix. Some of the underlying principles are easier to understand with 2x2 matrices, so I'll try to explain using only 2x2 matrices, disregarding translation for now. Here comes some basic linear algebra, so fasten your seatbelt. You may need to read this slowly, as there's a lot of concepts explained here, but it's the only way I know of explaining what the components are.

In 2D, the vector (x, y) can be decomposed using the formula x * (1, 0) + y * (0, 1), where (1, 0) and (0, 1) are vectors as well. This formula obviously produces (x*1 + y*0, x*0 + y*1) which simplifies to (x, y). The vectors (1, 0) and (0, 1) form what is called a basis, meaning that you can represent any vector by just these two vectors in that formula, and appropriate x and y coordinates. This may sound obvious, but then any other pair of vectors also allows you to represent any point, as long as they aren't aligned and none of them is the vector (0, 0). These two vectors will form a different basis. The basis formed by the vectors (1, 0) and (0, 1) is called the canonical basis (or standard basis or natural basis).

It is important to understand that without a basis, a vector's coordinates don't really mean anything. We always assume the canonical basis (1, 0) and (0, 1) when unspecified, but vectors are always relative to a certain basis.

Let's take the basis formed by the vectors (5, 3) and (3, 5), and the vector (2, 1) relative to that basis. How will the coordinates of that vector look like when seen as a vector relative to the canonical basis (1, 0), (0, 1)? To find out, just apply the above formula with the correct basis vectors: 2 * (5, 3) + 1 * (3, 5) = (13, 11).

Image

Now let's say for example that we have the vector v = (0.75, 0.65) relative to the basis (0, 1), (-1, 0). How will the coordinates of that vector look like when seen as a vector relative to the canonical basis (1, 0), (0, 1)? Applying the formula, it turns out to be the vector (0.75 * 0 + 0.65 * 1, 0.75 * (-1) + 0.65 * 0) = (0.65, -0.75).

This operation may sound like "bah, just another useless formula", but let's examine more carefully what just happened in the last example. The vector (0, 1) is the vector obtained by rotating the first vector of the canonical basis, namely (1, 0), by 90°. And the vector (-1, 0) is the vector obtained by rotating the second vector of the canonical basis, i.e. (0, 1), by 90° as well. Therefore, the basis (0, 1), (-1, 0) is a basis obtained by rotating the canonical basis by 90°. And what happened to the vector (0.75, 0.65) in the example above? It turns out, the vector (-0.65, 0.75) is the result of rotating the vector (0.75, 0.65) by 90°! So we've effectively transformed a vector into another one that is rotated by 90° with respect to the original. Of course it works for any vector, not just (0.75, 0.65). And it can work for any angle too, by taking the basis formed by the vectors (cos(angle), sin(angle)) and (-sin(angle), cos(angle)).

Does it work for other transformations? Sure it does. Let's try with scaling. If the vector (0.75, 0.65) is taken as being relative to the basis (2, 0), (0, 2) and we want to see what its coordinates will be if it was relative to the canonical basis, we get (1.5, 1.3), which is indeed the result of duplicating the vector, because both basis vectors were double the original ones. We could scale differently in x and y too if we wish. Another possible transformation is shearing, where the second vector of the basis is no longer at a 90° angle from the first (see the image above for an example).

Can you combine, say, a scaling and a rotation? Yes you can. For example, the basis (0, 2), (-3, 0) is the one obtained by scaling 2 units in x and 3 units in y, and then rotating the result 90°. In general, any sequence of such transformations can be described by a single transformation. But how do you easily get the result of accumulating them?

Enter matrices. Matrices generalize all these calculations, allowing them to be extended to any number of dimensions, and to combine as many such transformations as desired into a single operation. They also allow you to find out what the inverse transformation of a given one is, i.e. given the transformed vector, calculate the original one. That needs you to find the inverse of a matrix, which by the way is a costly operation.

To use matrices for transformation purposes, you proceed as follows. There are two conventions that obtain the same result, but I'll explain the most usual one. You place the basis vectors as columns of a 2x2 matrix, in order, and place the vector to transform as the elements of a matrix with 1 column and 2 rows (a column matrix; that's confusingly called a 2x1 matrix because rows are usually specified before columns). Then you perform an operation called matrix multiplication (google it for more info) of the 2x2 matrix by the 2x1 matrix; that results in another 2x1 matrix whose elements are the transformed vector.

Matrix multiplication is not commutative, i.e. you can't multiply a 2x1 matrix by a 2x2 matrix, but you can multiply a 2x2 matrix by a 2x1 matrix. And while you can multiply two 2x2 matrices, if you swap them, nothing guarantees you that the result will be the same, i.e. A*B is not generally the same as B*A. When accumulating transformations, you have to multiply them in reverse order of operation, i.e. B*A is the combination of applying first A and then B. (EDIT: Note that if you use CPML, you may run into a bug where it inverts the multiplication order, see https://github.com/excessive/cpml/issues/33)

In the 90° rotation example above, we would write:

Code: Select all

[0   1] * [0.75] = [0.65 ]
[-1  0]   [0.65]   [-0.75]
In OpenGL terms, you can multiply a mat2 and a vec2 to obtain another vec2, which is the result of transforming the first vec2 according to the basis given by the columns of the mat2.

Now let's take translations into account. You can translate any vector by adding a displacement vector to it, so if you're working with the components directly, that part is easy, but how does that work if you're performing the operations in matrix form?

What you do, is add a third element to the vectors and to the matrix, as follows.

For the vector to transform, instead of a 2x1 matrix, you create a 3x1 matrix where the first two elements are the components of the vector, and the last element is the constant 1.

For the transformation matrix, instead of a 2x2 matrix, you use a 3x3 one. You place the components of the first vector of the basis in the first two elements of the first column of the matrix, and make the last one a constant 0. You do the same with the second vector of the basis, in the second column. In the last column, you place the translation vector in the first two components, and make the last one a constant 1. This means that the last row of the matrix will always contain the constant elements 0, 0, 1.

Thanks to the way matrix multiplication works, this ensures that the first two elements of the result will be the transformed vector with the translation vector added to it, and the last element will always be the constant 1.

Using the same example above, if we wanted to translate the resulting vector by the vector (5, -3), we would write:

Code: Select all

[0   1   5]   [0.75]   [5.65 ]
[-1  0  -3] * [0.65] = [-3.75]
[0   0   1]   [  1 ]   [  1  ]
Finally, OpenGL works in 3D, even though LÖVE is only using two of the three dimensions it provides, therefore the vectors and matrices OpenGL deals with are 4x1 and 4x4 respectively (three components and that extra element). For 3D projection, it uses this thing called homogeneous coordinates which makes the last row not be all zeros and a final 1, but since LÖVE does not use that, and it's a concept that I still have some trouble understanding, we won't go there.

And with that, we're finally ready to explain what the elements of TransformMatrix mean:

- The first two columns of the TransformMatrix are the vectors of the basis that transforms the vectors passed in the draw functions, by the transformations performed by love.graphics.rotate, love.graphics.scale, etc. (the third component is z, which is 0 because LÖVE doesn't use it, and the fourth component is 0 to meet the requisites for translation).
- The third column equals vec4(0, 0, 1, 0), because the third vector of the basis is (0, 0, 1). This third vector isn't used in 2D, because nothing extends beyond the Z=0 plane.
- The fourth column contains the translation vector (with the Z component always 0) and the final 1.

So the matrix has this form:

Code: Select all

[x1  x2   0  tx]
[y1  y2   0  ty]
[0    0   1   0]
[0    0   0   1]
where x1, y1 are the components of the first vector of the basis; x2, y2 are the components of the second, and tx, ty are the translation vector's components.

What about ProjectionMatrix? They are the same idea, but the basis and translation are set in such way that the top left corner (0, 0) is transformed to coordinate (-1, 1) and the bottom right corner (w, h) is transformed to coordinate (1, -1), as required by OpenGL.

Finally, what about transform_projection? It's the transformation that combines the two transformations above, i.e. a basis and translation such that it performs the LÖVE transformation (love.graphics.scale etc.) followed by the OpenGL projection (the transformation to coordinates between -1 and 1), all in one go. It's obtained as the product of ProjectionMatrix * TransformMatrix.

zell2002 wrote:
Sun Oct 13, 2019 9:46 pm
Is it possible to have this kind of debugging here ?
Unfortunately, I'm not aware of any mechanism for importing values from GPU shaders to the CPU. You could perhaps modify the LÖVE sources so it can give you the matrix that it passes to the shader.

zell2002
Citizen
Posts: 75
Joined: Sun Feb 23, 2014 9:22 pm

Re: Shader variables - documentaiton and get the transformed vertext position

Post by zell2002 » Wed Oct 16, 2019 8:05 am

Thanks for such an indepth write up!
I'm going to have to find a note book like im in class lol
Will give this a proper read over weekend - when work code isnt in my brain trying to push out game code

Post Reply

Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 10 guests