By Henri Bernhard

Simplifying plant identification: that was the basic idea of ​​the app, explains biologist Jana Wäldchen. (Unsplash / Pascal Debrunner)

An app that recognizes plants in seconds: with “Flora Incognita” anyone can make a real impression on their next walk – without their knowledge. At the same time, searching is also supported while using the app.

The bees are doing well in the meadow on the grounds of the Technical University of Ilmenau. Purple and yellow flowers stand in the tall grass. The names of the plants are unknown to the reporter.

Jana Wäldchen, PhD in Biology, can help. Of course, she would recognize the flowers, but she is interested in the app on her cell phone – Flora Incognita, the “unknown plant”.

“We now have a starting image with three large, eye-catching flowers. And when you point to a plant, you tap the blue flower plus. Then you choose the life forms that the plant has in front of you,” she explains. .

“So, is it a flower? Is it a tree, a grass or a fern? And now we choose the flower here. And then the app asks for a typical photo of a flower. And now I’m going to take it. in picture .” She photographs the purple flower and selects the “Use photo” option.

“Now the photo is sent to the servers of the TU Ilmenau”, explains the biologist, “and within seconds the user receives the result. And here in our example the common vetch of birds.”

The reporter wants to know if she can confirm the result. “I can confirm that it is also correct”, answers the expert.

Simplify plant identification

Jana Wäldchen works at the Max Planck Institute for Biogeochemistry in Jena. But she spends a good part of her working time in Ilmenau, with the IT specialists at TU. “My interest is that I had the idea at the time: it would be nice to simplify the identification of plants and thus make it accessible to all who are interested.”

By identifying plants, people who use the app can get a better view of the diversity of what grows around them, says the biologist.

Jana Wäldchen hoped for help from TU Ilmenau for her idea: software capable of recognizing plants in their natural environment. This is where Patrick Mäder came in nine years ago, today he heads the data intensive systems and visualization department.

“So we want to recognize plants. This means that we need as many images as possible that show the species we want to recognize in the most varied perspectives, representations, different lighting situations and different seasons. “, he explains.

“Then we use what’s called a deep learning neural network and show these images one after another to that neural network and expect the network to be able to decide which species has been presented. in each case.”

Software that can learn

The software learns to recognize the typical characteristics of a plant – regardless of season, location or lighting conditions. This requires a lot of photos, which the software analyzes. Researchers are currently using around two million different images of thousands of plant species.

The software is trained in numerous runs to improve the success rate of photos taken with a cell phone. After about three months, the program recognizes up to 85 percent of the plants.

After years of development work, it works for an impressive biodiversity, says Jana Wäldchen: “In Germany there are more than 3000 plants with wild flowers. We have currently implemented around 4,800 species in the app. These are of course also species that are not naturally occurring in Germany, but can also be planted in parks and gardens. “

To do this, the software needs immense amounts of images to learn beforehand. It is based on an archive of images created over decades and made available free of charge by amateur botanists. Capturing and learning is not a trivial process and sometimes still requires human expertise and support.

A second application for the training database

In order to obtain the required data, researchers in Thuringia developed a second app: with it, users can send photos of plants from different angles – biologists then determine the species.

This increases the training database, which also includes the location and time of year. A rose in bloom in January in the Alps could therefore exclude the application. However, the data obtained is more than a gadget to identify plants using a smartphone.

“Our app is not just a pure identification app, we biologists also work with observational data. By storing the observation, we can understand where the plant grows, ”explains Jana Wäldchen.

“This way, we can also do long-term analyzes on how plants spread, for example, how plants may decline or how plants respond to certain climate changes. This is a long-term goal that we are also pursuing with our Flora Incognita app. . “

In the medium term, they want to transmit the data collected to authorities or environmental organizations, for example. Besides discoveries for biology, computer scientist Patrick Mäder also sees his team as a winner.

Up to 30 image requests per second

“We currently have over four million installations of this software. On strong Sunday afternoons we have up to 370,000 determinations, 30 frames per second requests. It’s an incredibly powerful system that’s in demand. “, explains the computer scientist.

“It has developed us enormously. We have learned a lot from it. Finally, a whole series of publications has emerged. We have developed new methods to process this image data, how to do it efficiently. “

Mäder also became an amateur botanist. He has already identified nearly 500 species with the app. Biologist Jana Wäldchen does not need an app for this – but she too has benefited from the collaboration with another research discipline.

“Well, I definitely learned how to deal with huge amounts of data. I hadn’t done that before. And we really noticed that ecology and botany have happened one way or another. in the age of big data. I’ve learned that over the years, “says the biologist.