This draw engine can do 24 bit colour, but it also can use compressed colour spaces like 12bit, 10bit, or 8bit mono tints. The idea is that images you draw are not photo realistic, but have a retro feel to them. After the video, I would like to include a sound generator with parametric controls and compressed audio including 8 bit audio for retro sound.
Anyway, this is the basic spec of the drawing virtual machine. I still have a few spare opcodes to fill, so maybe someone can think of useful extra ones. I might use the extra op codes to extend drawing to the canvas directly.
Code: Select all
Bit16 - Draw Engine and Media Player
====================================
The Bit16 engine digests a string of byte pairs and executes each one. All instructions are
16 bit long and control a Virtual Machine with registers and instructions that impliment basic
drawing primitives in a 1024x1024 pixel map in CPU space. There is a 1024x1024 Canvas which
matches the VM pixel map and is updated by draw instructions.
A string of byte pairs is presented to Bit16 in an update (~60fps) and the Bit16.draw is done
in the draw callback (Love2d), which draws part of the canvas to the display.
Each string is a "draw fragment" and executes atomically. Most instructions will draw shapes,
set pixels, or change the color of the draw operation. A draw fragment may also execute
commands like "publish" that flushes changes up to VRAM and the final 1024x1024 image
that the ui program uses. Since a draw fragment is locked to a frame, it is possible to
stagger the VRAM flushes over several frames and only publish every (say) 6th frame
(for 10fps video).
A "clip" is a data structure (table or json) that has an array of draw fragments plus
metadata describing how to draw them, including loops and pans, zooms. The array of draw
fragments is 1 .. n.
The Bit16 engine also has a "snapshot" command that copies the final 1024x1024 canvas to a
second canvas. This image persists for viewing separate to the normal output, or it could
be used for generating dynamic textures or tile maps.
16 bit Drawing Op-Codes
-----------------------
|| 4 bit op code || 2 bits op/data || 10 bits ||
In general, the 16 bit instruction is divided up into a 4bit op code and 12bit data.
For many instructions that only need 10 bits data, the opcode is divided into 4
Also, op code 0000 subdivides into smaller instructions with less or no data
To simplify decode, we have the top 4 bits as an instruction group, then the
following bits decoded as required - 12 bit data, 2 op bits + 10 bit data, or
4 op bits + 8 bit data. The 0 opcode is divided into 16 groups. 1-15 are 8 bit
parameters. Group 0 (0x000X) is a block of up to 256 commands that have no data.
the opcode 0x0000 is the NOP - it always executes nothing. A NOP instruction (2 bytes)
can be presented to the engine and it will just consume a draw tick.
X,Y position can be set as 2 10b values. In addition there is a second XX,YY pair
for destinations of block copy, line draw etc
Colour can be 24 bit, 12 bit, 10 bit or 8 bit mono/tint. 24 bit requires three instructions
to set RGB independently.
Once a colour is set, runs can be done with a draw/skip command that draw 1-32 pixels
then skips 0-31, or block, rect, circle, ellipse, line.
ImageData and canvas
--------------------
Note:- the pixel map workspace is 1024x1024 matching 10bit data co-ordinates
the published area is set by commands in the data stream, but may be drawn
at any size in the user application. The user view port is
set by the user program and is not controlled by the instruction stream.
Publish and refresh are now a lot more complex. Partial updates of VRAM images from
a rect of the bitmap are not supported in love from 0.11, and not at all in other engines
like godot or unity. The safe way to do this is to divide the 1024x1024 bitmap into 8x8 tiles
of 128x128 pixels and selectively refresh these. There is no longer any point in having
a 1024x1024 Image, so instead we have a 1024x1024 canvas and let the gpu blit from the
image tiles to the canvas.
So - we get the rect of tiles that includes the publish/refresh area, paste from the
main bitmap into the tile bitmaps, refresh the selected tile images, then blit the
tile images to the destination canvas. It is assumed that this is done as separate
operations as the GPU commands will be sent in batches which may be more efficient than
interleaving a replacePixels() with a draw for each tile.
The engine may be extended to include some draw commands that operate on the canvas in VRAM.
Useful would be rect fills and clears to save overhead of useless RAM->VRAM copies.
ViewPort
--------
Commands that change the ViewPort don't need any VRAM update. For example, to simulate video
noise (snow) you randomise a section of the draw bitmap, then every frame just issues a
command to change the ViewPort position. That will look like a new random noise every frame,
but actually use zero RAM->VRAM bandwidth. Same with pan and scroll effects. Load the bitmap
with an image (which might take hundreds of frames), then scroll, pan, and zoom around the
image with no draw overhead. Zooming is for free as well, since the draw fragments set the
published ViewPort size (nominally 360 x 240) and the external user program sets the display
ViewPort independently. The UI might be a window 720x480 and the published VP might be
128x85 which will be automatically scale up in the draw command. That gives a lot of
opportunity for playing with image resolution. In 1024x1024 you can fit 96 128x85 images
which gives a 10sec clip if played at 10fps. Once loaded, playing sequences of these frames
only requires a ViewPort change per frame - which is 3 16bit instructions (6 bytes).
Image Compression
-----------------
This Draw Engine knows nothing about .png or .jpeg. Any image has to be composed
from primitive draw commands. For simple logos and shapes it should do a lot better than
png and jpeg. For random images it will depend on the image and the amount of compute
time encoding.
The basic method for a general image is first to reduce the color space
to 12 or 10 bits - in games you often don't want photo realistic. Reducing the color
space means more blocks and runs of the same color. Next, you divide your image into
blocks and look for the most common color in that block. Issue a block fill command to
set each of the blocks (2 bytes per block). As you generate primitive drawing commands,
you keep a bitmap of wrong pixel colors. Then, on a block by block basis, you search for
bad pixels that are adjacent and matching and fill them with run instructions which are
quite efficient. Finally, to clear the remaining bad pixels in each block, you find sets
of same color pixels in the block and then set them with a mov_relative_xy_setpixel
instruction. This only uses 1 instruction (2 bytes) per pixel. Assuming that typically
about half the pixels can be covered by efficient block fills and runs, then that would
put the encoded image size at about 70k-80k for a 360x240 general image, which is similar
to a low loss jpeg and about 50% of a png. For lossy complex images, jpeg is better,
however for any cartoon style images for games, this encoding systems is very good.
Effects
--------
Random noise patterns are built in. You can specify a rectangle, block fill it with
a background color, then set a random threshold and change pixels above that threshold
to a random color, bound by color ranges. Or, you can set a block size and make a mosaic
of random color tiles in the rectangle - all in one instruction (2 bytes). You can build
small images somewhere on your 1024x1024 bitmap and then block copy them to build an image
in your main ViewPort. This draw engine is like a cut'n'paste worksheet of bits of images.
OP CODE definition
------------------
[0] 0000.0000.[0] -- nop, special case (256 no param commands)
.[1] -- reset - clear the bitmap and the canvas
.[2] -- reset tick - set back to zero
.[3] -- tick - put this at the end every draw fragment
.[4] -- reset frame
.[5] -- frame - increment with a publish or VP change
.[6] --
.[10] -- change viewport only (image is assumed correct)
.[11] -- refresh (force update of image from VP2)
.[12] -- publish (force update of image from publish area and vp)
.[13] -- snapshot - grab the canvas and the VP2 window as well
.[50] -- noise mono rectangle on XY rectangle (speckle)
.[51] -- noise color rectangle on XY rectangle
.[52] -- noise mono block draw a rnd blk color or not
.[53] -- noise color block draw a rnd blk color or not
.[54] --
.[55] -- rect copy with mono noise
.[56] -- rect copy with color noise
.[57] -- rect copy stencil black
.[58] -- rect copy color squish (noise ranges)
.[59] -- rect copy mono squish (noise range)
.[60] -- rect mono (convert to mono)
.[61] -- rect mono negative (convert to mono and invert)
.[62] -- rect color invert (color negative)
.[63] -- rect color squish (use noise range params)
.[64] -- rect XY, XXYY fill with random blocks mono
.[65] -- rect XY, XXYY fill with random blocks color
.[100] -- set dwg mode pixel (default)
.[101] -- set dwg mode block
.[102] -- set dwg mode rectangle
.[103] -- set dwg mode circle
.[104] -- set dwg mode ellipse
.[105] -- set dwg mode line
.[200] -- rect copy (straight copy src->dest)
[1] .0001.[8b] -- undefined
[2] .0010.[8b] --
[3] .0011.[8b] --
[4] .0100.[8b] --
[5] .0101.[8b] --
[6] .0001.[8b] --
[7] .0010.[8b] --
[8] .0011.[8b] --
[9] .0100.[8b] --
[a] .0101.[8b] --
[b] .0001.[8b] --
[c] .0010.[8b] --
[d] .0011.[8b] --
[e] .0100.[8b] --
[f] .0101.[8b] --
[1] 0001.xx.[10b value] -- publish parameters commands
0.X_pos -- X TL corner of published area
1.Y_pos -- Y TL corner of published area
2.X_size -- width of published area
3.Y_size -- height of published area
[2] 0010.xx.[10b value] -- VP2 parameters commands
0.X_pos -- X TL corner of 2nd viewport
1.Y_pos -- Y TL corner of 2nd viewport
2.X_size -- width of VP2 area
3.Y_size -- height of VP2 area
[3] 0011.xx.[10b value] -- position commands
0.X_pos -- X move absolute
1.Y_pos -- Y move absolute
2.XX_pos -- secondary X (destination)
3.YX_pos -- secondary Y (destination)
[4] 0100.xx.[10b value] -- block values
0.X_size -- X size of rect (also ellipse)
1.Y_size -- Y size of rect
2.B_size -- side of block to draw ind of rect
3.C_size -- dia of circle to draw ind as well
[5] 0101.xx.[10b value] -- relative instructions
0.XY_mov -- move +/-16 X +/-16 Y
1.XY_movblk -- move +/-16 X +/-16 Y in blocks
2.XY_pix -- mov,set pixel at rel position
3.XY_blk -- mov,set block at rel position in blocks
[6] 0110.xx.[10b value] -- runs of pixels
0.X_run -- draw hor run of pixels
1.Y_run -- draw vert run of pixels
2.X_runskp -- Hor fill 1-32 then skip 0-31
3.Y_runskp -- Vert fill 1-32 then skip 0-31
[7] 0111.xx.[10b value] -- draw immediate with 10 bit color
0.pix -- set pixel 10b color and mov right
1.blk -- draw block 10b color and move 1px right
2.rect -- draw rect 10b color and mov 1 px right
3.line -- draw a line diag in immediate rect
[8] 1000.xx..[10b value] -- draw immediate with 10 bit color
0.circ -- draw circle 10b color and mov right
1.ellipse -- draw ellipse 10b color and mov right
2.circ band -- circle band
3.ellipse band -- ellipse band
[9] 1001.xx.[10b value] -- draw immediate object with 10 bit parameter
0.blk -- draw a sq block - side 10b
1.circ -- draw a circle - dia 10b
2.rect -- draw a rect 1-32,1-32
3.
[a] 1010. -- not defined
[b] 1011.xxxx.[8b value] -- noise and color parameters
.0.[8b value] -- set R lower limit
.1.[8b value] -- set R upper limit
.2.[8b value] -- set G lower limit
.3.[8b value] -- set G upper limit
.4.[8b value] -- set B lower limit
.5.[8b value] -- set B upper limit
.6.[8b value] -- set Y lower limit (mono)
.7.[8b value] -- set Y upper limit
.8.[8b value] -- set prob of noise 0-255
.9.[8b value] -- set mono (tint) color
.10.[8b value] -- set R component (set for 24 bit color)
.11.[8b value] -- set G component
.12.[8b value] -- set B component
.13.[8b value] -- set tint R
.14.[8b value] -- set tint G
.15.[8b value] -- set tint B
[c] 1100.xxxx.[8b value] -- more 8 bit parameters and mono draw commands
.0.[8b value] -- set circle band x/255
.1.[8b value] -- set rand X min (4block boundary)
.2.[8b value] -- set rand X max
.3.[8b value] -- set rand Y min
.4.[8b value] -- set rand Y max
.5.[8b value] -- set rand X grain
.6.[8b value] -- set rand Y grain
.8.[8b value] -- draw pixel mono
.9.[8b value] -- draw block mono
.10.[8b value] -- draw circle mono
.11.[8b value] -- draw rect mono
.12.[8b value] -- draw ellipse mono
.13.[8b value] -- draw circle band mono
.14.[8b value] -- draw ellipse band mono
.15.[8b value] -- draw line mono
[d] 1101. -- not defined
[e] 1110.[12b value] -- set 12bit colour
[f] 1111.[12b value] -- draw object 12b colour