Biological systems, from simple to complex, all demonstrate the capacity for fusing multiple sensing modalities for object learning and recognition. Unique insights obtained from biological system design yield distinct advantages in the development of adaptive sensor fusion systems for real-world applications. Using lessons learned from reptilian and mammalian (both animal and human) retina and cortex, AIT has developed an architecture and prototype system for fusion and exploitation of multisensor surveillance data.

Analyst Tools That Learn
Imagery and signals collected by multiple platforms over a common geospatial area can be brought into registration by means of a 3-D site model (i.e., terrain data and building models). Forming a layered database of multiple registered modalities, opponent-sensor image fusion and spatial feature extraction support interactive 3-D visualization and also augment the database. This layered data is well organized for interactive pattern learning and search, whereas each pixel now corresponds to a complete feature vector. The interaction of multi-modal data layers with learning structures is motivated by the organization of superior colliculus and its connections to cortex in mammalian brain.

Cognitive architecture for fusion and exploitation of multisensor surveillance data

With the aid of a graphical user interface, an analyst selects examples and counter-examples (context) of the object of interest, allowing the learning system to discover features that are salient to the object and to create an efficient search capability. This process supports construction of maps and intelligence products, and provides inputs to information networks for situational assessment.

Applications

• Geospatial intelligence
• Environmental monitoring
• Assisted feature extraction
• Semi-automated mapping
• Terrain characterization and trafficability

Image mining learns to find ore piles and detects material
transports in this satellite image of a steel plant

Real-time Fused Night Vision
Most image fusion methods employed are based on global statistical methods (e.g., principal components analysis) and false color overlay of separate modalities. These approaches do not derive or benefit from biological approaches to sensor fusion. Our approach, based on opponent-color processing in the retina and visual cortex, has led to a technology for color fused night vision. Complementary modalities of wide dynamic range imagery, low-light visible (VIS), short-wave infrared (SWIR), mid-wave infrared (MWIR), and thermally emitted long-wave infrared (LWIR) provide different views into the night. AIT's fusion processors combine any or all of these modalities in real-time color. This improves situational awareness and the detection of targets and obstacles under low visibility conditions. AIT's image fusion technology is being integrated into fused night vision goggles, airborne turrets, and an attached processor for a laptop computer.

Laptop computer with attached single board, real-time fusion processor, supports visible and infrared image fusion, capture, editing, and search for targets in the dark.

"NOTE: These video files are very large (31,030k and 68,242k)
and require considerable time to download."

Applications

• Color fused night vision goggles
• Multisensor fused weapon sites
• Multesensor airborne turret systems