Computer Vision using Ruby and libJIT
19 Nov 2009
The RubyConf 2009 took place near San Francisco. I gave a talk about Computer vision using Ruby and libJIT. Here is the abstract of the talk:
Ruby originated in Japan, the country which is world-leading in robotic research. It suggests itself to put the two together and to start using Ruby as a language to program robots. However at the moment the performance of available Ruby interpreters is not sufficient. It is hard to achieve performance comparable to compiled C++-code since manipulation of Ruby-integers and Ruby-arrays requires frequent bounds-checking. It can be shown that universal bounds-check elimination is actually impossible.
This talk presents HornetsEye which is a Ruby-extension facilitating the development of real-time machine vision algorithms for future robotic applications. HornetsEye offers I/O facilities to capture and display videos. HornetsEye also can be integrated into GUI-applications developed with Qt4-QtRuby. Furthermore there is a set of Ruby classes which provides the means to compose native datatypes and specify operations on them. The libJIT just-in-time compiler is used to achieve real-time performance. The project was inspired by NArray and ruby-libjit.
The software is called HornetsEye and it is available under the terms and conditions of the GPL.
Feel free to leave comments below.
Discussion
Lazy computation
Future work on HornetsEye could be about using lazy computation to avoid the creation of intermediate results when doing array operations. Here’s a small prototype implementation. Let me know if you have any suggestions.
class Promise
def initialize( size, &action )
@size, @action = size, action
end
def inspect
"Promise(#{@size})"
end
def []( index )
@action.call index
end
def demand
Seq[ *( 0 ... @size ).collect { |i| @action.call i } ]
end
def -@
Promise.new( @size ) { |i| -@action.call( i ) }
end
def fill!( value = 0 )
@action = proc { |i| value }
self
end
end
class Seq
class << self
def []( *args )
retval = new args.size
retval.data = args
retval
end
def new( size )
unless Thread.current[ :lazy ]
super size
else
Promise.new( size ) { 0 }
end
end
end
attr_accessor :data
def initialize( size )
@data = [ 0 ] * size
end
def inspect
"Seq#{@data.inspect}"
end
def []( index )
@data[ index ]
end
def -@
Seq[ *@data.collect { |x| -x } ]
end
def fill!( value = 0 )
@data = [ value ] * @data.size
self
end
end
def lazy
previous_state = Thread.current[ :lazy ]
Thread.current[ :lazy ] = true
retval = yield
Thread.current[ :lazy ] = previous_state
if retval.is_a? Array
retval.collect { |x| x.demand }
else
retval.demand
end
end
# Ordinary computation
a = -Seq.new( 3 ).fill!( 1 )
# Simple lazy computation
b = lazy { -Seq.new( 3 ).fill!( 1 ) }
# ...Reflection
Using the following code one can use reflection to interpret Ruby code in order to for example translate it to machine code later.
class Const
attr_accessor :inspect
alias_method :to_s, :inspect
def initialize( s )
@inspect = s.to_s
end
def method_missing( name, *args )
str = "#{ self }.#{ name }"
unless args.empty?
str += "( #{args.join ', '} )"
end
Const.new str
end
def coerce( y )
return Const.new( y ), self
end
end
a = Const.new 'a'
# a
b = Const.new 'b'
# b
-a
# a.-@
a + b
# a.+( b )
a[ 2 ]
# a[]( 2 )
2 * a
# 2.*( a )
2 * a + b
# 2.*( a ).+( b )
2 * ( a + b )
# 2.*( a.+( b ) )Unfortunately Ruby reflection does not capture creation of variables, assignment of values, and control structures such as while or if-then-else.
GPU
One could use “monkey patching” to add GPU support (using OpenCL) in a transparent way. Furthermore it would be desirable to make use of multi-core computers. Ruby 1.9 supports native threads but there is a global interpreter lock on extension calls. So multi-threading has to be implemented explicitely if it is to be used in such a Ruby extension.
Technical Issues
It was suggested to me to
- make the array classes available separately
- to use Rubygems for packaging the software and Gemcutter for publishing the software
- to use Git for version control and to host the repositories on Github
- to use ffi so that the extension works with different Ruby VMs (JRuby, Rubinius, …)