Matlab day

Well, yesterday after Larry Ewing, my mentor, helped me out with the scaling problem I had, we reached a point where we have the detections located correctly in f-spot, screenshot here.

Today I'm working on the Matlab code of the algorithm, trying to glue something, but I don't feel like I get the whole thing properly. I'll try to pseudocode the algorithm for people to comment on.
First, a bit abstract description.
The algorithm uses a cascade of classifiers, where a classifier is described by:
* a rectangular feature (f)
* parity (p)
* threshold (theta)
* error (alpha)
The cascade is built of several stages, which in turn are made of a number of classifiers. The property of a classifier is that it has a very low false negative rate, so if it considers the tested region not face, then we can immediately stop processing that region and follow to the next one.
When a classifier classifies the region as a face it will be tested by the following classifiers and stages until it reaches the end of the cascade or some classifier along the way reports that it's not a face after all. That way computation time is greatly reduced for most of the regions.
Now, to be able to detect faces that are in various scales and locations Viola and Jones propose a shifting window mechanism. The detector is applied at every location and every scale. The window is shifted each time by 1 pixel. It is common that the same face is detected several times, at slightly different locations or scales - V-J propose a simple method of summing these up.
The classifier itself is a simple binary function returning true if a feature over the examined image window exceeds a given threshold. The classifier's error has to be accounted here as well.
As to the feature calculation - I found several implementations, all not very educative, maybe the best one being the one in Torch. It seems that the feature is a simple sum of the pixel intesities (of the so called integral image), although the features vary by having different weight on different regions (i.e. 1 or -1 ). If the rectangular sum exceeds the given threshold - the classifier reports a face.

I'll try to post some pseudocode later, for discussion. As for now, please point out any misunderstandings if you find them.

Edit :
I posted the Matlab code here : http://wytyczak.com/andrzej/upload/MatlabVJDetector/
I'll check for Octave compliance in a minute and post the code in http://wytyczak.com/andrzej/upload/OctaveVJDetector/

Octave is not compliant because it seems not to understand Matlab's sparse arrays,
which are used in the feature set I'm using (reusing actually).

Face detection status

Tuesday I finished hooking up f-spot with the opencv library. This integrates the face detection into f-spot and we're able to mark the face candidates returned by opencv (although not very precisely, due to some bug that is yet to be located). It looks kind of funny, but here it is :

This shows both the detection integrated into f-spot and the original opencv detection (the one blending in, with the circles).
It seems like some problem with the coords, but don't really know. The hack is mostly located in the f-facedetect.c which is linked with the libfspot. You can check it out of the svn at http://code.google.com/p/facedetect-f-spot/

Anyway, I spent the last 2 days digging through the various Viola-Jones face detector articles and code. I still feel uncomplete though, so if anyone has any experience in this matter and is
willing to answer a couple of questions - please let me know :)

Also in the news is that I pretty much have to finish this until July 18th, due to my earlier unplanned trip to the US, where I'm not allowed to work on the SoC project, and where I'll be until the end of september :/ If (which is highly probable) I won't make it until the 18th I'll continue working on the project, but outside of SoC.


edit: I actually have the algorithm pretty much figured out right now, it didn't take as much as I thought it would ;) although, still, I'll probably have a couple of questions, so please let me know if you know anything about the V-J algorithm!

f-spot face detection

I'm a bit off the schedule with my project, but nothing to be worried about. I took the last couple of days to get familiar with f-spot's code, and I'm still in the process, but there are already some visible effects. I managed to register a little extension, initially in the Tools menu, but due to some problems (see below) had to switch to the Export submenu.

The code is registered as a Mono.Addin and it just passes the filename of the picture being viewed to a facedetect sample program from the opencv library to get the following result.
Ok, now to the details.
The wiki at http://f-spot.org/Extend_F-Spot is outdated. Stephane has already coded an extension point for the Tools menu, the extension has to use the ITool interface. The code for that is in the trunk/src/Extensions in the ITool.cs and ToolMenuItemNode.cs files.
I'd love to use that, but...

I have no idea how to call anything inside the main program from the addin, except for the little example in the export extension, that passes the list of currently selected images to the extension. I tried transferring a similar functionality to the ITool interface, but I have no way of knowing if it works because...

Right now my environment got screwed up by an apt-get upgrade of my ubuntu (gutsy) and for some reason make all in the svn trunk builds an unusable binary that crashes with some nullpointer exception. What's weird - the binary compiled before the system upgrade works fine.

So, while waiting on some updates of the mono libs in gutsy I'll be hacking the face detection algorithm in octave :) I'll probably take some parts from the opencv, it seems to do a pretty good job :

It turns out, that it wasn't a bug in gutsy's libraries, but some mess I've created in the f-spot's glade file ;-) works fine now. Thanks to Sebastian Dröge for his help :)