Deep Reinforcement Learning and Multimodality

Combining deep reinforcement learning (DRL) with multimodality allows autonomous agents to process and learn from diverse data types—such as visual images, text instructions, structured sensor telemetry, and audio—to make sequential decisions.

Transcript

1. AI · MULTIMODALITY · DECISION MAKING Multimodality and Deep Reinforcement Learning Learn how AI combines different kinds of information and improves decisions through trial and error.
2. THE STARTING POINT AI rarely sees the world through one signal Multimodality Human reasoning Applications Learning from more than one information type, such as text, images, audio, speech, depth, or sensor signals. Humans combine sight, sound, and touch; real applications often need the same kind of joint reasoning. Video understanding, speech plus lip movement, and camera plus LIDAR for driving.
3. A FIVE-PART LENS Five questions organize multimodal learning 1 Representation — how do we encode different signals? 2 Translation — how do we change one modality into another? 3 Alignment — which words, regions, or moments correspond? 4 Fusion — how do we combine evidence for one prediction? 5 Co-learning — how can one modality teach another? The questions organize the design space.
4. QUESTION 3 · ALIGNMENT Alignment connects language to what we see “dog” — Matches the animal region. “tennis ball” — Matches the small object. “murky water” — Matches the surrounding scene.
5. A DIFFERENT LEARNING SIGNAL Reinforcement learning learns from consequences The agent updates future behavior to maximize longterm reward. Agent Action Chooses an action. Changes the situation. Reward Environment Tells how useful it was. Returns a new state.
6. RL FOUNDATIONS Three ways to learn what to do Different algorithms learn different parts of the decision process. Value-based Learn how good each action is, then choose the best one. Example: Q-learning picks the move with the highest expected reward. Policy-searchbased Search directly for a policy: a rule for choosing actions. Example: Try different robot controllers and keep the one that earns more reward. Actor–critic An actor chooses actions; a critic evaluates how well they are working. Example: The actor moves a game character while the critic scores the move.
7. SCALING THE STATE Deep reinforcement learning handles huge state spaces 1 Traditional Q-learning struggles when the number of possible states explodes. 2 A deep network learns a compact representation of a high-dimensional state, such as game-screen pixels. 3 In a Deep Q-Network, a CNN maps the current state to a Q-value for each possible action.
8. MAKING DQN PRACTICAL DQN becomes more stable with a few key tricks 01 02 Experience replay Target network Learn from stored past events in a less correlated order. Keep the target calculation fixed for a while, then update it. 03 04 Reward clipping Double DQN Normalize positive rewards to +1 and negative rewards to −1. Separate action selection from action evaluation to reduce overestimation.
9. THREE WAYS TO IMPROVE DQN DQN, Double DQN, and Dueling DQN DQN Double DQN Dueling DQN Learns which action gives the highest expected reward from each situation. Uses separate steps to choose and score actions, reducing overconfident estimates. Learns the value of a situation separately from the advantage of each action. Takeaway: Start with DQN, use Double DQN for more reliable values, and use Dueling DQN when state value matters even before an action is chosen.
10. PUTTING THE PIECES TOGETHER From mixed signals to better decisions Multimodality Reinforcement learning Deep learning Joins complementary evidence across data types. Joins actions with feedback over time. Supplies compact representations when the raw problem is too large. EXIT QUESTION What signals would an intelligent robot need, and what reward would guide its behavior?
11. A DELIVERY ROBOT LEARNS Sense → decide → learn The robot combines signals, acts, and uses the result to improve its next trip. 1 · SENSE 2 · DECIDE + ACT 3 · OUTCOME Camera + microphone + distance sensors Slow down, turn, or stop Delivery or obstacle See the path, hear people, and detect nearby obstacles. → Choose the safest next move. → Did the action help? POSITIVE REWARD NEGATIVE REWARD Safe delivery Obstacle or delay The robot remembers: this action worked. The robot learns to try a safer action next time. In short: multimodality tells the robot what is happening; reinforcement learning helps it choose actions that earn more safe-delivery rewards.