Gary Wolf

February 26, 2008

The New York Times magazine published a story last weekend about a special kind of baby monitor, the LENA, a $400 device that is tucked into a child’s clothing and evaluates the “language environment” throughout the day.

After recording all the sounds in a child’s environment, at the end of each day special software evaluates both the amount of exposure the child has had to verbal stimulation as well as the child’s own utterances.

Ultimately, the device generates percentile rankings that help assess a child’s language development, just as doctors provide such rankings for a child’s height, weight and head circumference.
The value of the LENA obviously depends on the quality of the analysis.

Speech recognition software has come a long way. Security agencies have long lusted after it. But whatever tricks the CIA might be deploying, at the consumer level our devices are not yet able to efficiently recognize and transcribe every utterance, especially in natural environments, where there is high background noise and multiple speakers.

In fact, we’re not even close. So the inventors the LENA took a shortcut. As Yudhiji Bhattacharjee reports: The best solution, it seemed, was to eschew the identification of particular words and focus on a recording’s acoustic features.

Modeling every conceivable sound in a household, they designed a system that distinguishes different voices from one another, gives a rough count of the number of words directed at a child and counts also the number of conversational “turns” that are taken as child and interlocutor exchange words.

The use of sound “signatures” to model behavior is allowing rapid progress in life-monitoring systems. This is the same general tactic used by the inventors of the e-watch, who can track and identify almost any common activity using just three measurements with small sensors that fit on a strap on the wrist: ambient sound; ambient light; and motion, as measured by a small accelerometer.

Technovelgy.com, which tracks the emergence of science-fiction ideas into real life, has a very good post that connects the LENA to other life-logging phenomenon, such as the SenseCam. The SenseCam has long been criticized as a solution in search of a problem. And so the notion of a ubiquitous baby monitor might seem, at first, more a symptom of neurosis than a useful tool.

But both these types of systems will eventually prove their worth, and the first stage of this will be in palliating chronic conditions or aiding in the diagnosis of subtle illnesses and disabilities. A story late last year in the MIT technology review described the proven value of the SensCam in helping people with dementia.

There are set of technical papers on the LENA site that explain the system and compare its effectiveness at analyzing a child’s verbal environment with the effectiveness of human observer-listeners. The results are impressive. However, these are not refereed scientific papers; they are the detailed technical claims of the inventors. But, on the reasonable assumption of good faith, this report shows that computer observation is capable of subtle and effective analysis of natural environments.