Keywords: Deep Learning, Computer Vision, Neural Networks, Probabilistic Programming, Tensorflow, PyTorch
Intelligent reality applications assist users with tasks in the real world. Examples include machine vision assisted inspections, checklists, troubleshooting, and assembly. These applications fuse inputs from different sensors and models to create a realtime understanding of the states of objects in the world in order to assist the user in achieving a goal. We demonstrate LayerJot, a framework for creating intelligent reality applications and discuss how LayerJot trains and combines models for visual-inertial odometry, detection, classification, tracking, and audio with a knowledge graph expressing the states of objects in the world. Furthermore, we show the importance that quantified uncertainty and model critiquing has on the user acceptance using Tensorflow Edward. The talk will then explain how to use LayerJot to capture training data and then train models using PyTorch and Tensorflow. This will then lead a demonstration of how to compose models together to create an intelligent reality application for safety inspection.