Andres Campanella participated in the spring 2011 TechCrunch Disrupt Hackathon, one of hundreds of hackers who took part in the event. His hack, Doach, uses computer vision to help the user talk to your date by tracking body language, posture, volume of speech, heart rate and perspiration as well as suggesting topics of conversation based on your date’s Hunch and Twitter profiles. Andres was one of the winners selected by a panel of judges to present their hacks during the TechCrunch Disrupt conference.

Watch Andres’ final presentation at Disrupt here:

How did the idea progress? When I first started I wanted to try something using Open CV. I came up with an idea where I wanted to leverage the “Beanbag API” by setting up beanbags to mimic the first stage to the right of Yoshi’s house in Super Mario World. I had taken the entire level, laid it out and scaled in in terms of beanbag placement. I wanted to have the camera and use a tracking token for an actor to have him jump across beanbags as if he was Mario and track it in an emulator in a hack-y sort of way. (Class of 2011 hackNY Fellow) Abe Stanway and I were working on it and then someone told us something similar had been done before at last year’s hackathon.

So then I started wandering around like a lost soul for hours, looking over people’s shoulders to see what they were working on. This happened to me at the hackNY Spring Hackathon, too. I was outside looking at the night sky, and then I just kind of thought of the idea. It’s not really something that I need, but it’s definitely something that a lot of people need. I just did it because it’s kind of funny. I can’t explain what I was looking for, just something humanistic, which is what I usually do. It was too late to be second guessing at 10pm. Abe was excited about the idea, but by that time had already started working on his own project, so I stared working on it on my own. I drank a surprisingly small amount of coffee for my own stamina.

The end product would be a service that communicates with when you have a persistent connection, like a web cam connection. The service would be in the cloud and could call your phone so you’d have an ear piece connected with the device, guiding you. You would enter the Twitter handle or Hunch username of the person you’re trying to date and, assuming you’re malicious enough to place a camera in a neatly unobscured region in the bar, it could also perform augmented reality tracking. The input of the person’s twitter and hunch usernames would build a profile of likes and dislikes of the person you want to date and it would track your body language info of how close you are and how slanted your posture is, and could maybe track how loud you were and how few or many words you say per second. It would take all that info and make a strategy plan. That was the big vision but it ended up being huge joke, just tracking your distance and telling you “you’re too far away and looking like a big loser, so why don’t you get closer.” The end product only tracks proximity to your partner and your posture in a not-at-all-real way. It can also recommend topics of conversation every certain number of seconds.

An interview with Andres at Disrupt:

I still don’t know how I managed to build it – I coded my ass off. Abe did help too, and did the Hunch interaction for me.

There’s not a lot of long term vision for the product here. It’s just following primitive ideas based on the little bit I’ve read about body language, proximity, and finding the sweet spot between two people. If the point was creating long-lasting relationships maybe the approach would be different and would encourage you to be a little gentler in your approach.

I used Open CV to track clothing color on two people. Open CV has libraries that do color tracking for you with handful of exciting matrix operations. I had never worked with Open CV before but a bunch of people very helpful, especially the Aviary guys, who were very supportive and helped me find resources to operate it.

This is what the first version of the hack looked like:

After the hackathon, I got help from with my friends Tengchao Zhou and Joe Saavedra to polish it up more before the final presentation during the conference. Joe is good with electrical engineering and helped add a sweat sensor, breath rate sensor and heart sensor. We also added tracking for loudness of voice using a microphone. Tengchao helped to make it look better. We also added a “kill switch.” If your date is beyond recovery, it gives you a phone call pretending to be your mom with medical emergency so can excuse yourself.

How was presenting?
The first presentation was very scary. I’m not much of a performer in that way and have terrible stage fright. I was shaking before I went on stage. I curse a lot when I’m nervous and this was only my second time presenting at a hackathon. I guess it can be charming sometimes if everyone is well-spoken and you try to be vulgar – it can move people around sometimes.

I was very scared. I didn’t want to memorize anything since I was so nervous and feared that if I got off track I’d have to start from the beginning and would look like a dick. I improvised and opened by saying that I know you geeks spend too much time playing Counter-Strike and using Microsoft Paint, which is why you don’t know how to approach women. And they responded very well and laughed. I was afraid they would think it was an insult, but it takes a geek to know a geek. If I was insulting anyone, I was insulting myself.

What did you win?
I won a little Statue of Liberty thing, a TechCrunch sticker and an Android phone that moot awarded me. That’s my moot award, and I will treasure it forever and ever.

Will you continue to work on it?
i don’t really know if I’ll follow up. I’ve got myself tied up with some research at NYU right now so I’m dedicating myself to other projects right now.

What advice would you give to others participating in hackathons?
Never surrender. There will be a ton of discouragement and your hack won’t work or you’ll find out someone’s already done the hack you did, so you have to switch up. Just keep trying and you’ll have something at the end.

Congrats, Andres!