Taking pictures of your food may soon be more than just a thing you do for Instagram. Researchers at MIT trained an AI system called Pic2Recipe to generate a list of ingredients based off of an image of food. It even recommends recipes. So, basically, it's Julia Child in AI form.
When tested, the system used machine learning to come up with the correct ingredients around 65 percent of the time. That's a pretty good success rate for an AI, considering it can be hard for some humans to tell what's in a dish by actually tasting it.
Nicholas Hynes, a graduate student at MIT's Computer Science and Artificial Intelligence lab, compiled data from Food.com, All Recipes, and other food sites to create Recipe1M, a database that stores more than a million recipes.
It's a tool that could eventually help us learn to cook, count calories, and track our eating habits.
"That the program recognizes food is really just a side effect of how we used the available data to learn deep representations of recipes and images," Hynes told Gizmodo. "What we're really exploring are the latent concepts captured by the model. For instance, has the model discovered the meaning of 'fried' and how it relates to 'steamed?' We believe that it has and we're now trying to extract the knowledge from the model to enable downstream applications that include improving peoples' health."
Here's how it works:
The algorithm had a hard time with complex dishes like sushi and smoothies, but nailed the ingredients of desserts. Clearly the tool is a work in progress, but it's a positive step towards one day having an app on your phone that can tell you how to make your favorite dish from your favorite restaurant.