Binary File to String of 0s and 1s

[rok]

I'm reading a ~500kB binary file via io.open and io.read. This is very fast. But now I need to get every character that was read from the file converted to a string of zeros and ones.

I wrote my own function for that, but it is very slow.

Code: Select all

function Converter:UnsignedCharToBits(number)
  local output = ""
  number = number + 1
  
  for i=8, 1, -1 do
    if (number - math.pow(2,i-1)) > 0 then
      number = number - math.pow(2,i-1)
      output = output.."1"
    else
      output = output.."0"
    end
  end
  
  return output
end

A guy from #love IRC suggested I should try the basen function from here http://stackoverflow.com/questions/3554 ... -converter but its still very slow. Does anyone know a faster way?

I'm writing a compression/decompression algorithm. For the decompression I probably have to use this method because I am running through all bits and checking for certain combinations.

Would it be faster If I'd try to somehow do it in smaller chunks?

[bakpakin]

Just a thought, but maybe you could create a lookup table for every byte with your current code. That would be a lot faster, and your lookup table would only have 256 elements, which is not too big at all. Also, don't use the concatenation operator in a loop like that; its a huge memory hog. Instead use table.concat.

If your making a compression alogorithm, though, love 0.10.0 will have compression and decompression built in.

[kikito]

Just a thought: LuaJIT (included in LÖVE by default) has a bit operations module. You can probably use it to increase the speed of your algorithm significantly.

[jjmafiae]

a quick little optimization would be:

Code: Select all

local math.pow = math_pow

and then call the math_pow

[rok]

@bakpakin I swapped all of the .. with table.concat and made a lookup table like you suggested. It's a bit faster and the code is cleaner. Thank you

@kikito I'll take a look, thanks

@jjmafiae Ok

[bobbyjones]

a quick little optimization would be:

Code: Select all

local math.pow = math_pow

and then call the math_pow

Dude you have it backwards that would not work.

Code: Select all

local pow = math.pow 

that will work.

[undef]

Aye you sure you have to check every bit?
At 500kb you'd have 4 million bits to check, maybe it suffices to check each byte?

You could dump the binary data in hex like this:

Code: Select all

local file = io.open( filename, "rb" ) -- I think you need the "b" if you're on Windows
local data = file:read"*a"
local hexDump = data:gsub( ".", function( byte )
	return string.format( "%02X ", string.byte( byte ) )
end )
print( hexDump )

This should be fast enough, if not you can try and cache the string functions.

[rok]

Yeah I have to run over every bit

Now you've guys helped me a lot already This part is very fast now.

Code: Select all

local lookup = require "lookuptable8bits"

-- ...

local t = {}

for i=1, #file_content, 1 do
  table.insert(t, lookup[string.byte(file_content, i)+1])
end

local all_content = table.concat(t) -- doing only one concat at the end

-- ...

Time to optimize the other stuff now and it will be perfect, and I think I already know what to do

[jjmafiae]

a quick little optimization would be:

Code: Select all

local math.pow = math_pow

and then call the math_pow

Dude you have it backwards that would not work.

Code: Select all

local pow = math.pow 

that will work.

Oh yeah sorry my brain switched off for a second xD

[T-Bone]

a quick little optimization would be:

Code: Select all

local math.pow = math_pow

and then call the math_pow

Dude you have it backwards that would not work.

Code: Select all

local pow = math.pow 

that will work.

Oh yeah sorry my brain switched off for a second xD

Technically, the original one will be faster. The first time you try to call math.pow, the game will crash because you're trying to call a nil value. This is faster than doing tons of pow calculations