Romanesco Broccoli – A Tale of Two Halves in Photogrammetry
I recently took part in a challenge set by the Sketchfab community to use photogrammetry to scan in a piece of fruit or vegetable to put in a 3D Cornucopia, more comonly known as the Horn of Plenty. I chose to scan a Romanesco Broccoli because it is complex, and interesting while still an easy shape for photogrammetry to pick up – at least so I thought!
I wanted to capture the whole veg which meant taking 2 sets of photos, one from the top and the other with the broccoli turned over to show the underside.
I used a newspaper base to rest the broccoli on which is extremely helpful to the photogrammetry software in working out camera alignment. The broccoli itself, while detailed, is also very regular which makes it hard to pick out unique features.
For this scan, I was able to borrow a Canon 7D camera which gave me a whopping 24 Megapixels to play with rather than my usual 6! I used the same lens though (my kit 18-55mm), so the gains were not as great as could be with decent glass.
No focus stacking was done for this shoot and I used natural light and a ring flash. The camera was also mounted on a sturdy tripod, rather than my usual ghetto lego balancing act.
The resulting photos were very detailed and clean but on close inspection a little soft which I mainly blame the lens for.
ISO was set to 100, Shutter speed was 1/5 sec. and F-stop was f/13.
I have made the photos available to the public for on Dropbox here .. Dropbox Download Be warned, it’s a fairly large 448MB download!
With all my 95 shots taken, it’s time to go into Agisoft Photoscan and see how it gets on…
Firstly, I tried without any masking to see how it gets on with camera alignment straight from the can. The resulting alignment was a little off and I could see it straying in the point cloud. This was probably down to the background throwing off the alignment.
Next, I manually masked out the background in every shot to tell Photoscan to focus on the foreground and include the good bits of newspaper print.
This gave me decent alignment of all cameras and an accurate looking cloud of points…
Once all the dense cloud and mesh generation was complete, my first thought was to do the mesh alignment in Zbrush and combine them there. I don’t know why I didn’t try it in Agisoft first, perhaps it was unfamiliarity of how best to go about it and I was at least familiar with Zbrush.
Unfortunately, the route into Zbrush proved no good. I spent a lot of time cleaning and aligning each half manually, but when it came to dynameshing, Zbrush crashed constantly and I couldn’t get a clean re-projection. I abandoned the Zbrush route and researched how I should have done it in the first place.. inside Agisoft.
Normally, you would let the software try to align the point clouds itself, but due to the fractal nature of the broccoli and the fact I didn’t have a lot of overlap in my meshes, it could not get an alignment without the newspaper. I decide to use a 2 step process using markers. First I picked out 4 points along the edge to allow the software to align the 2 chunks pretty closely. Then, by flipping between the 2, I was able to more accurately plot some more markers at various points around the rim…
From here, alignment was more accurate and I was able to re-generate a mesh from the 2 dense point clouds with a fair amount of accuracy. I then decimated the mesh to a more CPU friendly 700,000 faces rather than the original 14 million!
I wanted to try some nice rendering techniques, so took the model into Maya by Autodesk. I’ve been using this package professionally since it was Alias Wavefront, so am totally happy using it. The decimated geometry was fine to use as it was. The place I wanted to work a little further was in the texture. I wanted to use a sub surface scattering shader in my render, so needed to do a little work on the texture in Photoshop first. I used the colour to create 2 additional textures to represent the deeper layers of colour..
The resulting render looked like this ..
I did a turntable render too …
.. and you can see the final model and download it in Sketchfab here …
Overall I was pretty pleased with how it turned out. I learned loads about combining two sets of scans into one mesh and the pitfalls involved. In hindsight, I’d allow for more overlap in the meshes and paint some small bright dots on the broccoli, so I could align more easily. A Romanesco Broccoli is too fractally (made up word) to be able to pick out unique features.
All the best – Stu