Multimodal AI for Extreme Sensing

April 8, 2025 (2:00pm - 3:30pm)
E14 6th Floor Lecture Hall



Where are sensors failing you?


Tell us here to brainstorm solutions with us

Building AI systems that learn from multiple sensory inputs such as text, speech, video, real-world sensors, wearable devices, and medical data holds great promise for impact in many scientific areas with practical benefits, such as in supporting human health and well-being, enabling multimedia content processing, and enhancing real-world autonomous agents. This workshop will cover practical methods for designing and deploying AI systems that can sense, learn, and interact with the real world through many sensory modalities. It will cover a spectrum of methods, from lightweight, efficient sensing tools to modern large-scale multimodal foundation models. The workshop will also explore the applications of next-generation AI for extreme sensing, revealing hidden visual information beyond human perception, such as seeing around corners and turning objects into cameras.

Faculty

Faculty

Ramesh Raskar

Camera Culture

Associate Director and Associate Professor at MIT Media Lab

Paul Pu Liang

Multisensory Intelligence

Assistant Professor at MIT Media Lab and MIT EECS

Speakers

Speakers

Tzofi Klinghoffer

Camera Culture

Graduate Student at MIT Media Lab

Nikhil Behari

Camera Culture

Graduate Student at MIT Media Lab

Kushagra Tiwary

Camera Culture

Graduate Student at MIT Media Lab

Multimodal AI for Extreme Sensing

April 8, 2025 (2:00 PM - 3:30 PM), E14 - 6th Floor Lecture Hall

Part 1 (45 min): Foundations of Multi-Sensory AI

Paul Liang

2:00 PM - 2:10 PM

Introduction to Multi-Sensory AI

2:10 PM - 2:20 PM

Core Challenges and Solutions

2:20 PM - 2:35 PM

Latest Advances and Open Problems

2:35 PM - 2:45 PM

Q&A and Open Discussion

Part 2 (45 min): Applications of Multimodal AI for Extreme Sensing

2:45 PM - 2:55 PM

Computer Vision at the Speed of Light for AV and AR

Tzofi Klinghoffer

2:55 PM - 3:05 PM

What’s in an Image? Unlocking Hidden Cues in Imaging

Nikhil Behari

3:05 PM - 3:15 PM

AI-Driven Design of Vision Systems

Kushagra Tiwary

2:15 PM - 2:30 PM

Open Q&A and Discussion