Photometric image formation is a fundamental concept in computer vision that explains how light interacts with objects in a scene and how this interaction results in the brightness and color values captured in an image. While geometric image formation deals with the projection of 3D points onto a 2D image plane, photometric image formation focuses on how radiance, reflectance, shading, and optics contribute to the pixel intensities observed in images.
1. Concept of Photometric Image Formation
Photometric image formation seeks to answer questions such as:
- Why do objects in an image appear bright or dark?
- What causes colors and shading in images?
- How do the properties of light, material, and camera affect image appearance?
Basic Process
Light from one or more sources strikes the surface of an object. This light is:
- Absorbed,
- Reflected (either diffusely or specularly), or
- Transmitted through the object.
A portion of the reflected light enters the camera lens, passes through the optics, and finally strikes the image sensor, producing a response based on the intensity and color of the incoming light. This process is illustrated in Figure 2.14, which presents a simplified model of photometric image formation, excluding multiple reflections and complex light interactions.
2. Key Components and Effects in Photometric Image Formation
Photometric effects are modeled by analyzing the following components:
a) Lighting
- Types of Light Sources:
- Point light sources: Emit light from a single location (e.g., bulb, sun).
- Area light sources: Emit light from a surface (e.g., LED panels, diffuse lights).
- Fall-off behavior: Light intensity decreases with the square of the distance from the source.
- Color spectrum: Light sources emit varying wavelengths, influencing the perceived color of surfaces.
- Environment maps: Model complex lighting environments (e.g., sunlight, reflections from the sky) using spherical or cubemap representations.
b) Reflectance and Shading
Light interacts with surfaces in complex ways:
- Diffuse reflection (Lambertian): Scatters light uniformly in all directions. It depends on the angle between the incoming light and the surface normal and contributes to soft shading (e.g., matte objects).
- Specular reflection: Occurs on shiny surfaces where light reflects in a specific direction. It causes highlights (glossy spots).
- BRDF (Bidirectional Reflectance Distribution Function): A general function describing how light is reflected based on incoming and outgoing angles. It incorporates both diffuse and specular effects and varies for different materials.
- The amount of light LrL_r reflected from a surface in a direction v^r\hat{v}_r is given by:
- Lr(v^r;λ)=∫Li(v^i;λ)⋅fr(v^i,v^r,n^;λ)⋅cos+θi dv^i
- L_r(\hat{v}_r; \lambda) = \int L_i(\hat{v}_i; \lambda) \cdot f_r(\hat{v}_i, \hat{v}_r, \hat{n}; \lambda) \cdot \cos^+ \theta_i \, d\hat{v}_i
- where frf_r is the BRDF,
- θi\theta_i is the incident angle, and cos+θi=max(0,cosθi)\cos^+ \theta_i = \max(0, \cos \theta_i).
- Phong Shading Model: Combines ambient, diffuse, and specular terms
- Lr(v^r;λ)=ka(λ)La(λ)+kd(λ)∑iLi(λ)[v^i⋅n^]++ks(λ)∑iLi(λ)(v^r⋅s^i)ke
- L_r(\hat{v}_r; \lambda) = k_a(\lambda) L_a(\lambda) + k_d(\lambda) \sum_i L_i(\lambda) [\hat{v}_i \cdot \hat{n}]^+ + k_s(\lambda) \sum_i L_i(\lambda) (\hat{v}_r \cdot \hat{s}_i)^{k_e}
- ka,kd,ksk_a, k_d, k_s: ambient, diffuse, and specular reflection coefficients.
- s^i\hat{s}_i: direction of perfect specular reflection.
- kek_e: shininess exponent.
c) Optics
- Models how light is focused through the lens onto the sensor.
- Real lenses introduce effects like:
- Focus and depth of field: Determined by focal length and aperture.Vignetting: Light fall-off at image edges due to angle and aperture shape.Chromatic aberration: Different wavelengths focus at different depths, causing color fringes (Figure 2.21).
- 1zo+1zi=1f
- \frac{1}{z_o} + \frac{1}{z_i} = \frac{1}{f} where:
- zoz_o: object distance
- ziz_i: image distance
- ff: focal length.
3. Important Effects in Photometric Image Formation
a) Shading Variations
- Created by changing surface orientation w.r.t. light.
- Affects depth perception and surface texture recognition.
b) Shadows and Occlusion
- Light blocked by objects creates shadows, adding realism.
- Hard shadows come from point sources; soft shadows from area sources.
c) Highlights and Gloss
- Result of specular reflection.
- Help in identifying material properties (e.g., plastic vs metal).
d) Interreflections
- Light bounces between surfaces before reaching the camera.
- Leads to color bleeding and complex illumination effects.
e) Global Illumination
- Includes both direct and indirect light contributions.
- Modeled using methods like ray tracing and radiosity.