Well that title is a mouthful! I’d like to talk about Photogrammetry. This is a technique to construct 3D models from a series of photographs. My particular take on it is to do this with small objects like a Hamster Skull! Hence the phrase Macro Photogrammetry. Now I don’t have a lot of money for this, so to do cheap Macro Photography, you can put the lens on your camera back to front with the help of an adaptor ring, now we get Reverse Lens Macro Photogrammetry. The problem with macro photography is you get a very shallow depth of field, so here comes the technique of Focus Stacking! Focus stacking means taking a series of photos in exactly the same place, but changing the point of focus slightly in each photo. You can then combine the photos in a program which takes the sharp bits of each photo and blends them together into one image and that’s how we get to Focus Stacked Reverse Lens Macro Photogrammetry!!
Hammie’s head is tiny! His skull measures a mere 28mm long. He also has some really challenging details such as thin pieces and deep sockets, so he will be a good test of the process.
Let’s look at each aspect in detail..
The Photogrammetry Process
Photogrammetry at its core is making 3D objects from 2D photos. It uses complex maths that I won’t even pretend to understand, but if you can get your head around what the program needs, you can supply it with good quality photos that yield accurate results (and look awesome).
How does it work?
You give the photogrammetry program a photo of something. It looks at the entire image not knowing what it is at all, what the program is looking for are details it can match with other photos. These will be areas of pixels which have high contrast or unique patterns.
It will look for thousands of details like the ones above ..
As you can see, it will look for points even in blurred areas and this is where errors can creep in. It is a good idea to either mask off the blurred areas or eliminate them in the first place.
Once it’s analyzed all the photos for matching details, the program then tries to work out where the camera was when each photo was taken. If will use EXIF data such as focal length from the photo to help with the calculation if it can. You can even supply calibrated camera data that has been pre-calculated beforehand.
Here is what a camera aligned screenshot looks like …
Once you have the aligned cameras, the program can then use this information to go back through the photos and look at finer details and cross-compare with others to generate a dense point cloud in 3D space. This cloud of points can then be used to generate a 3D model comprised of many vertices and faces (in the order of millions for some objects). Then you can project the photos back onto the model to give it texture (colour detail). I’ll go into more detail on the steps within Agisoft Photoscan later.
The end result is a 3D model you can edit, rotate, analyse from every angle or even 3D print it out!
So what do you need? – a camera, obvs. That’s it really, you can use any camera, your phone etc., but the end result is entirely dependent on the photos you take, so there is a little more equipment that will go a long way to helping the process..
I’m doing this on the cheap, so my choices are based on what I have already or can get at a reasonable price..
Camera – I’m using my old Canon 300D DSLR I’ve had for 13 years. It cost £700 at the time, but you can pick one up second hand for less than £50 these days. I wouldn’t recommend it as it’s only 6 megapixels, but as a DSLR I have a lot of control over shot settings and lens choice. For photogrammetry, megapixels do matter but so does sensor size, so go for the best you can afford. These things get pretty expensive fast!
Lens – I’m using the 18-55mm kit lens that came with my camera. For sharpness, I’d recommend a prime lens (fixed focal length) a 50mm prime would work for most. For macro, I’d go for Canon’s 100mm Macro lens. Again, money is a big factor here.
Remote Trigger – In order to minimize vibrations, it’s really handy to have some sort of means to be able to press the trigger without touching the camera. A cable release is simple and effective if your camera supports it. You will be taking a LOT of photos, so it needs to be accessible. I bought a cheap one for £4.48 from Amazon, here is a link (opens in new tab) – Amazon – Remote Shutter Cable
Reverse Lens Adapter – For cheap macro, you need an adapter ring that screws onto the filter thread of your lens and allows you to mount it in reverse I got mine from Amazon for £7.19, here is a link to it – Amazon – Reverse Lens Adaptor Ring
Lighting – The more even, diffuse lighting you can get around your subject the better as it will avoid harsh changes in shadows as you rotate the subject which will throw off the photogrammetry. I used a couple of desk lamps to set the main lighting either side of the subject. What really helps here is a ring flash as that will boost the light straight from the lens and helps get rid of shadows. Again, I’ve gone for the el cheapo option which works well enough for my purposes. Amazon to the rescue again! – I got mine for £21.11 – Amazon – Macro Ring Flash
Note: All these Amazon links above are affiliate links, which means I get a little bit of money if you buy anything from Amazon after clicking the links. Better you know up front.
Turntable – You need to be able to smoothly rotate your subject without knocking it in order to cover every angle. I’ve gone for two options here. The first is a Lazy Susan we had in the house already which works great for larger specimens. The second is a bit more custom as I made it out of my sons Lego. The Lego version rotates sideways like a lathe which is really handy for getting all angles with the camera. You have to remember to rotate the final photos by 90 degrees before putting them through the photogrammetry software though as it assumes you are moving around the subject in the up/down axis.
Tripod – It is essential your camera is steady while taking shots, so a worthwhile investment is a sturdy tripod. However, as I’m going cheap again. I built a base from more stolen Lego (sorry son) and an elastic band to hold the camera steady. It works well for the lathe orientation as the camera only needs to move from side to side. If using the Lazy Susan rotation, a tripod would be preferable (although I did manage it with a Lego tilt mechanism)
Total cost so far (at todays price for camera) – £82.78
There are a few free photogrammetry programs out there (VisualSFM + Meshlab, Autodesk 123D Catch). However, none come close to Agisoft Photoscan – http://www.agisoft.com/ It really is the best and most easy to use for the price. It has a 30 day trial and currently costs – £144 at Brexit exchange prices.
If you are going to do Focus Stacking, then I’d recommend Helicon Focus – http://www.heliconsoft.com/ It does a great job, better than Photoshop and is really fast too. Current price is – £24 for a 1-year license or £92 for unlimited
I’m also assuming you have access to a computer to do this all on. A fast CPU, lots of memory and a good graphics card will all help speed up the process. I’ve tried it on a £300 home built PC and a £7k beast, the more you spend, the faster things get processed.
Right, so you’ve got this far. You have your camera, computer and software. Now what? Well you need something to capture. I’m focusing on bones at the moment because my eventual aim is to scan in an entire skeleton and re-articulate it in 3D and animate it. Bones are also ideal because they are not shiny or hairy or leafy or made of glass or metal or plastic or alive and moving – all of which are hard to capture using photogrammetry. You also want something with a lot of surface detail or colour variation as smooth surfaces are also difficult to capture.
Once you’ve decided on an object, you want to mount it so it rotates about it’s longest axis showing as much of the entire object as you can without it falling over. I use Blu-tak to hold non-precious (and light) subjects in place. Bear in mind you may need to capture the object in two or more orientations and merge the meshes together later.
Think also about the background, it will ideally be plain and a different colour to the subject. Also useful is some sort of base plate like newspaper or irregularly patterned matt surface which can provide some really helpful key points for the software to lock on to.
Taking the pictures
Once you’ve got your subject chosen and ready, it’s time to go through a few settings in readiness for your photo session..
So, your camera needs to capture the image with as little noise or blurriness as possible. In photography, there is the holy triangle of ISO, F-Stop and Shutter speed which result in a perfectly exposed picture. We need to control each of these to get to that lovely exposed picture with minimal blur or noise. Everything will be manual, so expect a few test shots to get the right settings.
ISO – set to the lowest you can for your camera, usually 50 or 100. Lower means less noise = good for photogrammetry.
F-Stop / Aperture – The higher the number, the greater the range of depth of field, so the more of the subject is in focus. Sounds great to go for the highest you can, right? Unfortunately, there is also this thing called diffraction. The smaller the aperture (higher F-Stop number), the more like a pinhole the lens gets until it gets to a point where everything is a little bit blurry like trying to look through scrunched up eyes. A good general rule of thumb to avoid diffraction is about F11 to F16
Shutter speed – The last thing in the holy triangle you have some control over to get the right exposure! Unfortunately as the other two are at their worst setting for getting light onto the sensor in favour of sharpness, the shutter speed is going to be quite long. This is why everything must be solid and still. You can help this by getting more light on the subject, but more on that later.
Try some test shots at different angles to get the right shutter speed for best exposure. Use the histogram feature! Avoid too much contrast or burnt out highlights.
White Balance – If your lighting and subject matter colour is consistent, you can get away with leaving this on automatic. Otherwise fix it to a sensible temperature setting that works for most angles.
RAW or Jpeg – Normally, I’d recommend shooting RAW because Jpeg will introduce compression artifacts. I chose to shoot Jpeg for practical reasons of space and speed of capture, just make sure it’s on it’s highest setting. Shoot RAW if you can.
Focus – Set to manual and adjust for each shot to get as much in focus as you can. If you are going to focus stack, you’ll be taking several shots at different focus points.
Ideally you’ll have some good natural white photography lights shone through a diffuser to give your scene a bright, even light. However, I’m using what I have to hand so a couple of desk lights will have to do. Position one on each side of your subject, if you have a third you can shine overhead, even better. If the lights are bright enough, shine them through a piece of tracing paper or tissue paper which will help diffuse the direct light. Or you can point the light in the opposite direction and bounce it off some white card. There are many ways you can set this up, but the minimum is two light sources either side and preferably a ring flash.
The ring flash will be your key light source as it will give a much needed boost at picture taking time. If you have the battery power, some ring flashes can be set to continuous light mode. The great thing about the ring flash is the light is even all the way around the lens and really helps remove shadows. As my lens is reverse mounted, the ring flash can’t screw onto the lens filter, so I had to stick it on with blu-tak.
Another option is a bright, overcast day and shoot outdoors, but bear in mind clouds can cause subtle variations you cannot control.
Any knocks, vibrations or movement are going to result in blurry photos which are useless for photogrammetry. Make sure your camera is on a sturdy base (even if it’s made of Lego). Use the cable release. If your camera has a mirror lock up feature, use that to reduce shutter vibrations. Don’t leave your hand on the turntable or anywhere where you breathing is going to cause things to move (the subject or the camera). Remember your shutter speeds are likely to be long, so vibrations get magnified.
Think about the process of taking the photos, it’s best done in one big session with the lights consistent. First, position the camera side on and square to the subject. Take your shot. Rotate subject 5-10 degrees, take another shot. Repeat until you’ve done a whole 360 degree rotation. Then move the camera to a new angle and repeat. Do this for as many angles as to cover all nooks and crannies
If you are going to the added effort of focus stacking, when you set up each shot start with your point of focus at the farthest point behind the subject then after each shot, focus slightly further forward. Try to overlap the points of focus to get as much coverage as possible. Once, you’ve done a full set of focus point, then rotate and do it all again! And again! And again!
Here is an example of an entire photo shoot including focus stacking …
Processing the Photos
This technique really helps gets maximum detail out of your macro shots, but at the cost of adding another layer of processing which risks losing valuable lens information that can be used by the photogrammetry program.
I’m using Helicon focus stack on default settings as it seems to work well with minimal fuss. All my source photos are in one folder, I manually select each batch of photos from the folder and drag it into Helicon for processing. Any photos which are under exposed are not included for processing, you can see them appear in the video above where the flash has failed to fire.
They have a great tutorial on the workflow on their site – http://www.heliconsoft.com/helicon-focus-workflow/
Once stacked, you should have a clean set of photos ready for the next step.
Image adjustment / Rotation
The way I rotated this particular skull is set perpendicular to the ground like a lathe. While this makes it more convenient for camera placement and accessibility, it breaks some basic assumptions that the photogrammetry program makes about orientation. In order to fix this, you need to rotate all your images 90 degrees to pretend the object is static and you are moving around the subject with your camera at 90 degrees (portrait) which is the basic assumption the photogrammetry program is making.
If your photos are slightly under exposed, you can lift out some of the details by applying some careful basic adjustments like levels. However, bear in mind you are adding another layer of processing that might confuse the photogrammetry program. If you do decide to make adjustments, make sure the same amount is done on every photo to maintain consistency.
Another useful step is to mask out any background that you don’t want included. This can be done from inside Agisoft Photoscan, but if you have access to a program like Photoshop, you can automate the process especially if your background was a consistent colour not used elsewhere like a green screen.
For my example, I chose not to mask the background.
Agisoft Photoscan Workflow
Now that we have done the hard work of getting the best source photos we can, here is where the nitty gritty of generating a mesh comes in and (hopefully) the hard work will pay off!
First step is to import all our clean photos using Add Photos in the Workflow menu.
Then we need to mask the background of each photo using the in-built selection tools to define areas to ignore. This step can sometimes be skipped if you have a clean background.
Next is to align the photos, this is where PhotoScan works out where each camera is and creates a sparse cloud of points to represent the mesh. This stage is where you can tell if your source photos are any good for the program. The default settings should be sufficient, but if you have the PC power, it can be useful to up them by a factor of 10. Turn on constrain features by mask if you have masked out the background in each photo.
Once done, have a good look around the mesh cloud to see if it looks like the object you captured. You can tell if it’s getting mis-aligned as it will look less defined than you expected (or even inside out or overlapping). You can also look at the camera placement and you should have a clear idea if it looks right or not. There is not a great deal you can do about it other than take photos again using a different base plate, orientation or lighting or it could be down to the object itself. Sometimes changing settings or masking can help, but not always.
You can clean up an alignment after by selecting any points that are clearly in the wrong place and deleting them. It can also help to do a Gradual Selection from the Edit menu using default Reprojection Error settings and delete the selected points. Then do an Optimize Cameras in the Tools menu to refine stray points and resulting calibration.
Once you are happy that everything looks ok, it is time to build the Dense Cloud. This process looks in finer detail at every aligned photo and uses the newly worked out camera data to calculate what the actual mesh looks like. Before you build the Dense Cloud, check the position and size of the bounding box, so that incorporates the whole mesh. If you have a fast enough PC with enough memory (try it and see to guage if you have), use Ultra High settings as it will take the maximum resolution of photos as reference.
The remaining steps are to build the mesh and generate the texture. These are fairly self explanatory. Once the mesh is generated, you get a real sense of which parts worked and which didn’t …
After all that, we can view the mesh and examine all the lovely detail with any luck. I am somewhat happy with the results of my example skull. You can see the overall shape and detail. There are holes in the mesh where there were not enough overlapping photos, especially near the base. If I were to re-shoot, I’d use a slightly smaller base plate, so I can get in closer.
The grain you see all over the mesh is most likely due to my camera being so old and low resolution, I’d love to try it on a better camera next, but the expense is going to escalate fast.
I’ve uploaded my model to Sketchfab, so you can take a closer look yourself …
Click link to go to Sketchfab Model -> https://skfb.ly/XH7O
On the whole, I’ve learned a lot just to get this far with my example. Hopefully, my ramblings will help others take their own models further.
The only things I’ve not done with this example is to create a second version capturing the skull from a different base, which I can then combine the two meshes and remove both bases. I’d also take this into a 3D modelling package like Zbrush to clean up and optimize further, but maybe another time!
Future work will involve trying to get hold of a better camera and decent macro lens. These I believe would vastly increase my detail and help eliminate noise.
If you’d like to learn more from various other sites on the web, here are a few I have visited to gain my knowledge so far…
This fantastic video series on using Agisoft Photoscan and merging 2 meshes together…
Great PDF and blog on techniques for Photogrammetry – https://dinosaurpalaeo.wordpress.com/
Posts by Stu Pond on Photogrammetry – http://paleoillustrata.blogspot.co.uk/
Some great articles by Dr. Peter Falkingham on free Photogrammetry software – https://pfalkingham.wordpress.com/2016/09/14/trying-all-the-free-photogrammetry/
Paper about using Focus Stacking and Photogrammetry – http://zookeys.pensoft.net/articles.php?id=4346&display_type=element&element_type=4&element_id=411&element_name=
Paper about using automated system for capturing photos – http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0094346
That’s it and have fun! If you have any questions or comments, please do get in touch.