IS20: SP2: Best Paper Prize Lecture - Plug and Play: A General Approach to AI & Sensor Model Fusion

Friday, July 10 12:00 PM - 12:45 PM The overarching goal of computational imaging (CI) is to form images from noisy, incomplete, and heterogeneous sensor data. Approaches such as maximum a posteriori (MAP) estimation became the dominant approach to CI because they integrate sensor and prior models by representing each as a penalty function to be minimized. More recently, AI methods have resulted in an explosion of amazing CI algorithms. However, AI inference methods such as convolutional neural networks (CNN) are purely algorithmic and are not formulated using a penalty function minimization, hence cannot be easily integrated with more traditional physics-based sensor models in the MAP framework. This talk presents Plug-and-Play (PnP), a general framework for the integration of AI and traditional physics-based models in CI problems. In its most basic form, PnP allows a large class of AI algorithms to be used as prior models by formulating the algorithms as denoisers. When the denoiser is an ideal MAP estimator, then PnP results in the MAP estimate to the CI problem. More generally, the PnP framework can be used to integrate multiple heterogeneous models by solving the consensus equilibrium (CE) equations. Importantly, the CE equations define the solution in terms of an equilibrium criteria rather than an optimization function to be minimized. Examples are presented from a variety of CI problems including tomography and holographic imaging. Gregery Buzzard Purdue University, U.S. Charles Bouman Purdue University, U.S. Lawrence Drummy Air Force Research Laboratory, U.S. Jeff Simmons Air Force Research Laboratory, U.S. Suhas Sreehari Wells Fargo, U.S. Singanallur V. Venkatakrishnan Oak Ridge National Laboratory, U.S. Brendt Wohlberg Los Alamos National Laboratory, U.S.