Description:
Reference #: 01587
The University of South Carolina is offering licensing opportunities for Multimodal Retrieval and Execution Monitoring Using Rich Recipe Representation
Background:
The content of recipes like for preparing food or assembling furniture exist as textual or image documents which makes it difficult for machines to read, reason and handle ambiguity. We consider the problem of reasoning with data in recipes in multiple representations.
Invention Description:
Our innovative representation is used in a web-based decision support system that helps users perform constrained queries using multiple modalities and monitor an agent executing (cooking) based on the recipes. This technology overcomes shortcomings with our recipe representation, which is enhanced with additional knowledge such as outcomes like allergen information, possible failures, and solutions for each atomic cooking step.
Potential Applications:
Recipes
Complex instructions such as furniture assembly
Advantages and Benefits:
Ability to perform expressive and constrained queries about
– the outcome of recipe
– the process of recipe preparation
Ability to monitor progress of an executor automatically following the instruction.