Detailed Reconstruction of 3D Plant Root Shape∗Ying Zheng†Steve Gu‡Herbert Edelsbrunner§Carlo Tomasi¶Philip BenfeykAbstractWe study the 3D reconstruction of plant roots from multi-ple 2D images. To meet the challenge caused by the delicatenature of thin branches, we make three innovations to copewith the sensitivity to image quality and calibration. First,we model the background as a harmonic function to improvethe segmentation of the root in each 2D image. Second, wedevelop the concept of the regularized visual hull which re-duces the effect of jittering and refraction by ensuring con-sistency with one 2D image. Third, we guarantee connect-edness through adjustments to the 3D reconstruction thatminimize global error. Our software is part of a biologicalphenotype/genotype study of agricultural root systems. Ithas been tested on more than 40 plant roots and results arepromising in terms of reconstruction quality and efficiency.1. IntroductionAs the primary site of nutrient and water uptake, rootsplay a critical role in plant growth. Recent research [15, 22]highlights the role of genes in regulating root branching, akey component of overall root architecture. A better under-standing of root architecture could lead to the production ofplants that sequester larger amounts of carbon dioxide, thushelping to reduce one of the causes of climate change. Inaddition, improved root systems can aid in food productionparticularly in marginal soils.To better understand roots, it is important to be able tocompare the complex 3D structure of root systems betweenplants with different genotypes. In contrast to simple shapesof large volume, plant roots have delicate, fine geometricstructures with thin branches; see Figures 1 and 2 for theplant root imaging system and a sample image. This pos-es challenges for the image-based 3D reconstruction, whichis exacerpated by the inaccuracies caused by unavoidable∗This research is supported by the National Science Foundation (NSF)under grant DBI-0820624.†Dept. of Computer Science, Duke Univ., Email: [email protected]‡Dept. of Computer Science, Duke Univ., Email: [email protected]§IST Austria, Email: [email protected]¶Dept. of Computer Science, Duke Univ., Email: [email protected]. of Biology, Duke Univ., Email: [email protected] 1. Plant root imaging system.Figure 2. Close up image of two roots growing side by side in agel container.small refractions and the jittering inherent in the imagingsystem. Furthermore, there are requirements that originatefrom the embedding of the software in a larger work pro-cess, which include the need to have connected 3D recon-structions and software that is efficient and works withoutuser intervention. A sample 3D reconstruction is shown inFigure 3 and additional results can be seen in Figure 9.We make three main technical innovations to achieve thedetailed 3D reconstruction of plant roots. First, we modelthe background of each 2D image as a harmonic function,which facilitates the extraction of the silhouette by adaptivethresholding. Second, we formulate the 3D reconstructionFigure 3. Five views of the reconstruction of a pair of root systems growing in a common container. Here and in the rest of the paper, thecolor corresponds to the height on the root.step as a compromise between two objectives: satisfyingall images and one particular image. The former objec-tive guarantees for a good global approximation and cor-responds to the traditional visual hull algorithm. Addingthe latter objective, we call this the regularized visual hullalgorithm, which reconstructs otherwise lost delicate struc-tures. Third, we develop an algorithm inspired by persistenthomology [5] that guarantees the connectedness of the 3Dreconstruction. Our algorithm is efficient and runs fast inpractice. For example, given a set of forty images, eachconsisting of 1, 600 × 1, 200 pixels, we can reconstruct the3D root structure in seconds on a dual core laptop with only2 GB memory.This paper is organized as follows: Section 2 reviewsprior work and explains why our problem has not been welladdressed in the literature. Section 3 presents a method forextracting the binary silhouette using harmonic backgroundsubtraction. Section 4 describes the regularized visual hul-l that follows two optimization criteria. Section 5 presentsan algorithm for ensuring the 3D reconstruction is connect-ed. Section 6 shows and compares results obtained with oursoftware. Section 7 concludes the paper.2. Literature ReviewThe problem of reconstructing a 3D shape from 2D im-ages has been studied for decades. The general purpose al-gorithm referred to as visual hull, or volumetric carving,finds the largest shape consistent with the input silhouettesor color images [1, 4, 8, 9, 13, 14, 19, 20, 10]. However, dueto its sensitivity to calibration errors, thin features of theshape are likely to be lost. A joint optimization approach[7] has been proposed to cope with the segmentation andcalibration errors in the moving camera environment. It issimilar to our regularized visual hull but different becauseit relies on the texture and color information as matchingcues, which are not available in our setting. A new imagingsystem working with coplanar shadowgrams has been in-troduced in [24], in which the object and the camera remainstill while the light source moves. This reduces the com-plexity in the calibration step from six degrees of freedom(position and orientation of the camera) to three (positionof the light source), and leads to improved reconstructionresults. While this method is promising, it cannot be ap-plied in our lab setting in which the opacity of the gel poseschallenges to collecting the root shadows.Complementing the general purpose methods, there hasrecently been progress using prior knowledge on the shapeto be reconstructed. In [6, 11, 17], shapes are reconstructedby optimizing objectives that guarantee a continuous and ifpossible smooth surface. However, these methods assumeaccurate calibration and cannot deal with jittering or oth-er movements during the image process. Moreover, thesemethods are not designed for thin and delicate shapes suchas plant roots. Model-based reconstruction of shapes in arestricted class, such as trees, buildings, and human bodies,has also been studied in the past decade [12, 16, 18, 21, 23].Among this work, image-based tree modeling is the mostrelevant to our problem. However, this work is geared to-ward computer