Rajiv Mehta

January 18, 2012

This is the eight post in the “Toolmaker Talks” series. The QS blog features intrepid self-quantifiers and their stories: what did they do? how did they do it? and what have they learned?  In Toolmaker Talks we hear from QS enablers, those observing this QS activity and developing self-quantifying tools: what needs have they observed? what tools have they developed in response? and what have they learned from users’ experiences?

For me, some of the most interesting QS talks have been by those creatively repurposing existing sensor technologies for novel self-tracking applications — such as Mikolaj Habryn’s Noisebridge, at an early QS meetup, and Hind Hobeika’s ButterflEye goggles, at the QS Amsterdam conference. It’s fascinating to hear what the inventors are thinking long before their product is in the market. Here, Eric Gradman, master hardware hacker, tells how he is applying his skills to a focused life-logging application.

Q: How do you describe Facelogger? What is it?

Gradman: The Facelogger is a passive lifelogger that helps me remember every person I meet by creating flashcards of their face, name, where we met, and our conversation. Facelogger consists of an always-on videocamera necklace, a software suite to process the video, and a smartphone interface for reviewing the flashcards.

The camera is a commercially available Looxcie camera, which was modified with a prism so it hangs around the neck. This camera continuously captures activity, and has a button that allows you to save the preceding 30 seconds of footage (footage that’s not saved is automatically discarded). When I meet someone for the first time and they introduce themselves, I press the button. The camera preserves the previous 30 seconds of footage which hopefully includes a good video frame of the person, their name, and what they said about themselves.

When I next plug the camera into the computer, all the captured video clips are automatically uploaded to a server, and sent to Amazon Mechanical Turk. There, human beings identify the most representative faces from the video, determine their names, and even transcribe the conversation.

Facelogger gathers all the information and creates a Facecard, which can be reviewed later on a smartphone. A Facecard is like a flashcard, but it shows someone’s smiling face, their name, a map of where you met, a link to the video of the conversation, and often even a transcript of the introduction. I  can search the text of the Facecards, sort them chronologically, or by geographic proximity.

Q: What’s the back story? What led to it?

Gradman: Like any self-respecting geek, I’ve always tried to stay a technological step ahead of my peers and a technological leap ahead of my parents. But when I discovered that my parents use the same model smartphone I do, I realized I was beginning to lose my edge. To me, the next frontier for personal electronics is wearable technology, and the natural application is self quantification.

But what to quantify? As a hardware hacker and artist, my first foray into QS wearable technology was definitely more for entertainment purposes. Called the Narcisystem, it was a biosensor suit featuring sensors for pulse, heading, EEG, pedometry, and breath alcohol level. I used the output of these sensors to drive the lights, sounds, and ambiance at a party venue. Fun, but not really a form of human augmentation.

I have terrible trouble remembering the names and faces of people I meet. Its hard to say which is worse: my face-blindness or my memory for names. I’ll meet someone, shake their hand, and we’ll introduce ourselves. Moments later I realize with panic that I’ve already forgotten their name! And hours later, if they’ve changed clothes, altered their hair, or removed their glasses, I’ll blithely reintroduce myself like we’ve never met. At least I’m not shy!

I’ve always wanted to offload the mental burden of remembering people. When I was in school and I needed to remember something I used flashcards. Why couldn’t that technique work for people too?

Q: What impact has it had? What reactions have you had?

Gradman: Because the Facelogger is a first-stage prototype I am its only user. Has it helped me remember people I meet? You bet it has. I’m amazed by the quality of the Facecards and by how effective they are at jogging my memory. I get the general sense that reviewing Facecards a day or two after meeting someone gives me an opportunity to properly commit someone’s name and face to memory at my own pace … something I simply cannot do “on the fly” as we meet.

There’s another purely psychological effect: because I’m confident that my technology is taking care of remembering for me, I can relax into the conversation. I was never shy about saying “hi” to people before, but I did experience stress over the fact that I immediately forgot their name and face. Now with that interaction captured and searchable, I’m not bothered at all.

I’m sensitive to the ethical concerns with capturing someone on video without their consent. When asked what I’m wearing around my neck—and as you might expect, that happens a lot—I never lie. I explain that I’m wearing a video camera to help me remember people I meet. Invariably, I’m asked “is it recording me now?” I’ve been asked to turn it off, and I always comply. But a surprising number of people tell me they want their own Facelogger. It turns out there’s demand for a device to help remember people’s faces and names!

Some have questioned the legality of wearing a video camera. But there are already cameras trained on us wherever we go. You can buy a video camera hidden in a pen, or a pair of sunglasses. Will our social mores (or our laws) surrounding cameras trail so far behind the technology?

Very few have actually questioned the morality of wearing a video camera. In the age of pervasive social networking we’re living highly examined lives. For anyone with a camera on their mobile phone, its not such a stretch to imagine wearing the camera around their neck.

Also, I’m careful to remove the Facelogger when I’m not likely to meet new people:at home, in a business meeting, etc. I do this because the purpose of this device is not to have a record of every conversation I have.

Q: What makes it different, sets it apart?

Gradman: Life-logging is always something that fascinated me, but I felt that an ever growing cache of unsearchable video of my life would just be a huge burden. Facelogger is an experiment in constrained lifelogging. By only capturing moments that share a particular characteristic and have common features Facelogger allows for a well-defined process of data extraction and collation that address a specific shortcoming.

Gordon Bell, the pioneer of life-logging described his always-on MyLifeBits image recorder as “write-once, read-never.” For me, the decline in storage costs is not sufficient reason to record my entire life on video. Huge amounts of unprocessed video is just video I’ll have to review someday! That’s why I find it so easy to resist the temptation to press the “capture button” more often. Unless I have automatic tools to convert video into a compact searchable representation—in this case, a Facecard representing a person I’ve met—the video just isn’t worth saving.

There are other tools out there designed to help remember names and faces. Evernote recently released Hello, a mobile app to record people. What distinguishes Facelogger is it’s passive form of information capture.

Q: What are you doing next? How do you see Facelogger evolving?

Gradman: Currently, a Facecard only expresses information captured in the 30-second clip. But APIs for face identification are getting really good. Soon the Facelogger will dig through my social network, and connect a Facecard to the social profile of the person it represents.

Next I will passively capture my meals, and use Mechanical Turk to help catalog my meals.

Face logging and food logging are only two well-defined applications of life-logging. I intend to identify others, and make them available as software for anyone wearing a compatible life-logging rig.

Q: Anything else you’d like to say?

Gradman: Face-blindness and poor memory for names are widespread problems! I designed the Facelogger with my own shortcomings in mind, but I’m now examining how I can make these tools more widely available, perhaps as a subscription service.

If you’re interested in updates on this project, have ideas to improve the system, or want to be contacted when the Facelogger service is available for beta-testing, please join the mailing list.

Product: Facelogger
Website: http://www.gradman.com/facelogger
Platform: Currently, iOS.  Coming soon to any HTML5 enabled smartphone.
Price: not yet for sale; to be contacted for beta-testing, please join the mailing list

(If you are a “toolmaker” and want to participate in this series, contact Rajiv Mehta at rajivzume@gmail.com)