We address visual correspondence problems without assuming that
scene points have similar intensities in different views. This situation is
common, usually due to non-lambertian scenes or to differences between
cameras. We use maximization of mutual information, which is a powerful
technique for registering images that requires no a priori model of the
relationship between scene intensities in different views.
However, it has proven difficult to use mutual information to compute dense
visual correspondence. Using mutual information to compare fixed-size windows
suffers from the well-known problems of fixed windows, namely poor performance
at discontinuities and in low-texture regions. In this paper, we show how to
compute visual correspondence using mutual information without suffering from
these problems. Using a simple approximation, we show how to incorporate
mutual information into the standard energy minimization framework used in
early vision. The energy can then be efficiently minimized using graph cuts,
which preserves discontinuities and handles low-texture regions. Experiments
demonstrate that our algorithm is effective for many situations, including real
scenes that have had their intensities artificially altered.