An image is a two dimensional projection of a three dimensional world. The big problem with this projection is that big distant objects appear the same size as small close objects. For people, and robots, it’s important to distinguish these different situations. Let’s look at how humans and robots can determine the scale of objects and estimate the 3D structure of the world based on 2D images.
We can derive a linear relationship between the coordinates of points on an arbitrary plane in the scene and the coordinate of that point in the image. This is the planar homography and it has a number of everyday uses which might surprise you.
Light field cameras are now commercially available and capture much more information about the rays of light reflected from the scene. This enables us to perform functions like changing the focus of an image after it has been captured.