Sunday, June 10, 2007

f-spot face detection


I'm a bit off the schedule with my project, but nothing to be worried about. I took the last couple of days to get familiar with f-spot's code, and I'm still in the process, but there are already some visible effects. I managed to register a little extension, initially in the Tools menu, but due to some problems (see below) had to switch to the Export submenu.
The code is registered as a Mono.Addin and it just passes the filename of the picture being viewed to a facedetect sample program from the opencv library to get the following result.
Ok, now to the details.
The wiki at http://f-spot.org/Extend_F-Spot is outdated. Stephane has already coded an extension point for the Tools menu, the extension has to use the ITool interface. The code for that is in the trunk/src/Extensions in the ITool.cs and ToolMenuItemNode.cs files.
I'd love to use that, but...

I have no idea how to call anything inside the main program from the addin, except for the little example in the export extension, that passes the list of currently selected images to the extension. I tried transferring a similar functionality to the ITool interface, but I have no way of knowing if it works because...

Right now my environment got screwed up by an apt-get upgrade of my ubuntu (gutsy) and for some reason make all in the svn trunk builds an unusable binary that crashes with some nullpointer exception. What's weird - the binary compiled before the system upgrade works fine.

So, while waiting on some updates of the mono libs in gutsy I'll be hacking the face detection algorithm in octave :) I'll probably take some parts from the opencv, it seems to do a pretty good job :

It turns out, that it wasn't a bug in gutsy's libraries, but some mess I've created in the f-spot's glade file ;-) works fine now. Thanks to Sebastian Dröge for his help :)

10 comments:

Joachim Beckers said...

a lot of packages in gutsy are affected by a bug in mono. symptoms are crashing programs and failing builds. installing libmono-cairo2.0-cil fixes the problem. maybe you can try that.

apart said...

I already seem to have that one :/ Thanks for input though.

mdi said...

Pretty amazing.

Congratulations!

Miguel.

apart said...

this is just the out-of-the-box performance of the opencv library, nothing I can take credit for! I must admit it's cool though :)

Tom McMahon said...

Hey There,

Let me just say that this looks really cool, I love F-Spot, and this face recognition stuff seems awesome.

Tom

Jan Moren said...

Haar-feature detection works fairly well. But remember first that you need a really diverse set of training examples (various kinds of lighting, facial hair, glasses and so on). Also, you really need a bottom-up selective process that preselects areas likely to contain faces as well, to constrain the face detector, or you'll en up with quite a lot of false positives.

And don't use skin color as the sole (or at all) bottom-up filter. Yes, it's an attractive, easy-to-implement idea. And it works fine if your camera and environment is known. But color varies too much across environments to be much use for a general method (and that's before you consider that a fair amount of people like black and white photography).

Benjamin Carlyle said...

How long until my photos can be automatically tagged according to the people in them? :)

Benjamin

apart said...

jannem - you're right about the haar detection, but that holds to every (me thinks) algorithm. When I'm ready I'll ask the community to train the algorithm on people's libraries, I hope it will give us enough diversity for the algorithm to work properly. As to the false positives - I thought about that, but I think I'll leave that for future work, and for now maybe just stick to some good skin detection or something. You know - you'll never satisfy all of the needs ;)
As to when autotagging might be available - I'd be really carefull with giving any estimates on that. Hopefully when I finish the detection algorithm I'll find some time to hookup my face recognition to that, but that would be more like a proof of concept.

Unknown said...

Any development by now...?

Unknown said...

Yes, are you or anyone else working on this anymore?