Harvard University

Inferring Shape and Materials under Real-World Lighting

A vision system is tasked with inferring the observable properties of a scene---shape, materials, and so on---from one or more of its images. The task is made hard by the fact that the mapping from scene properties to images is many-to-one: For any given image, there are infinite scenes to explain it.
A viable approach for dealing with this ambiguity is designing systems that combine prior visual experience with loose, redundant constraints induced by texture, shading, and various other aspects of optical stimulation. The basic idea is that each cue reduces the set of interpretations in some way, and by combining them, systems will be better equipped to sift through the infinite set of possibilities and arrive at a reasonable result.
To succeed with this approach, we need to understand the various ways that shape and materials are encoded in image data, and in this talk I will describe two that remain poorly understood. Each of these exists in the presence of complex "real-world" lighting, and for each I will summarize our recent progress in: 1) characterizing the constraints induced on a scene, and 2) creating algorithms for inference.

Slides <link