TC AgRA – Webinar 37 – 4D Crop Analysis for Plant Geometry Estimation in Precision Agriculture

Luca Carlone is a postdoctoral associate in the Laboratory for Information and Decision Systems at the Massachusetts Institute of Technology. Before joining MIT, he was a postdoctoral fellow at Georgia Tech (2013-2015), a visiting researcher at the University of California Santa Barbara (2011), and a visiting researcher at the University of Zaragoza (2010). He got his Ph.D. from Politecnico di Torino, Italy, in 2012. His research interests include nonlinear estimation, optimization, and control applied to robotics, with special focus on localization, mapping, and decision making for navigation in single and multi robot systems. For more information, please visit www.lucacarlone.com

Abstract:
The reconstruction of the 3D geometry (height, crown radius) of plants in an agricultural field and the estimation of how the geometry evolves over time is crucial for early disease detection, stress evaluation, and phenotyping. In this talk, I discuss a complete pipeline for 3D crop reconstruction from monocular camera images and inertial sensors. Our pipeline leverages state-of-the-art techniques for Simultaneous Localization and Mapping (SLAM) in robotics, and multi-view stereo in computer vision. After discussing 3D reconstruction, I focus on the problem of estimating the evolution of the geometry of the plants over time. This is what we call 4D crop analysis. In this case, one is given a set of 3D reconstructions, picturing the status of the crop during many consecutive weeks, and the goal is estimate how the geometry and the appearance of each plant evolves over time. The problem is challenging since the 3D reconstructions may contain very partial views of each plant. Moreover, the presence of multiple plants (and background) requires solving the data association problem. We propose a general probabilistic model to estimate plants’ geometry and appearance using factor graphs. Then, we show that the choice of a suitable parameterization, and the use of expectation maximization enable fast inference on plant growth. We demonstrate our approach on data collected in a test field in Tifton, Georgia, showing the potential of our technique to enhance situational awareness in precision agriculture