Smartphone App Identifies Trees from Camera phone Photo

By: Ian Mansfield | 12th Jun 2011: 3:56pm

­A new smartphone app has been developed that helps users identify trees simply by taking a photo of a leaf and scanning it with an internal database taken from the Smithsonian Institute.

William Belhumeur, a professor of computer science at the engineering school and director of Columbia's Laboratory for the Study of Visual Appearance had worked on face recognition software since the mid-1990s, and realised that the same same algorithms that can process the curve of an eyebrow or the angle of a cheekbone could be applied to the shape of a leaf.

With the help of computer scientist David Jacobs at the University of Maryland and John Kress, research botanist and curator at the Smithsonian Institution, Belhumeur developed LeafSnap, an electronic field guide that is now available on the iPhone and iPad, and will be released for Android phones later this year.

The team started by photographing leaves from the Smithsonian's vast library. But they soon realized a viable application would have to be able to recognize leaves in the wild, not just museum specimens. So Belhumeur's student volunteers collected thousands of leaves from New York's Central Park - up to 50 samples each from the park's 145 species.

A leaf's shape is its least variable feature and easiest to capture in a photo, so the team focused on characteristics like smooth versus jagged, many-lobed or single-lobed. They then programmed the computer to perform a sort of process of elimination. "The computer basically ranks images by most similar to least similar," says Neeraj Kumar, a Ph.D. candidate in computer science who manages LeafSnap's software coding and is in charge of the volunteer leaf identifying team.

Back in a Schapiro Hall lab, the team trained the computer to distinguish one species from another. "We pick one feature we extract from the leaf, and using that we can say, 'This looks more like all of these maples I've seen and less like something else,'" says Kumar.

The app, which is free, allows a user to photograph a leaf, upload it and see a list of possible matches within seconds. There is also a complementary website with profiles of each species. Initial interest is high; the app has been installed 150,000 times.

They hope to eventually map species across the United States and use a crowd-sourcing element to let users add their own images to the database.

On the web: Leaf Snap - via Colombia University