GRIDi - Deep Q-Learning Neural Network Demo

Interactive Deep Q-Learning Demonstration

Watch an AI agent learn to navigate complex environments using reinforcement learning. This real-time visualization shows how neural networks adapt and improve through trial and error.

What You'll Learn:

Neural Networks

See how layers process information

Reinforcement Learning

Understand reward-based learning

Real-time Training

Watch learning happen live

Experimentation

Modify parameters and observe

Under The Hood: Advanced AI Architecture

Input Neurons

Multi-sensor perception system

128

Hidden Neurons

Pattern recognition layer

10K

Memory Buffer

Experience replay capacity

95%

Discount Factor

Future reward weighting

Deep Neural Architecture

Input Layer (14 neurons) ReLU

Vision System (8 neurons):

8-directional obstacle detection
Normalized distance values (0-1)
Adaptive sensing radius

Action Memory (4 neurons):

Last 4 actions encoded
One-hot vector representation
Prevents immediate backtracking

Goal Vector (2 neurons):

Normalized X,Y direction to goal
Euclidean distance-based
Range: [-1, 1] for each axis

Hidden Layer (128 neurons) ReLU

Activation: Rectified Linear Unit (ReLU)

Purpose: Complex pattern recognition & spatial reasoning

Capacity: ~3,584 learnable parameters

Function: Maps high-dimensional state to action values

Output Layer (4 neurons) Linear

Q-Values: [Up, Right, Down, Left]

Range: Unbounded real numbers

Selection: ε-greedy policy (exploration vs exploitation)

Total Params: ~4,100 trainable weights + biases

Deep Q-Learning Algorithm

State Observation

Agent perceives 14-dimensional state vector from environment

• Sensor fusion from vision, memory, and navigation systems • Normalized input range [0,1] or [-1,1]

Action Selection

ε-greedy policy balances exploration vs exploitation

• 80% exploration rate initially • argmax(Q(s,a)) for exploitation • Random action for exploration

Environment Interaction

Execute action and receive reward + next state

• Immediate reward calculation • State transition dynamics • Terminal state detection

Experience Storage

Store (s, a, r, s') tuple in replay buffer

• Circular buffer (10,000 capacity) • FIFO replacement policy • Experience diversity preservation

Batch Learning

Sample random mini-batch for network training

• Batch size: 32 experiences • Bellman equation: Q(s,a) = r + γ·max(Q(s',a')) • Adam optimizer with MSE loss

Target Network Update

Periodic soft update for training stability

• Target network prevents moving targets • Update frequency: every 100 steps • Reduces correlation in Q-value updates

Interactive Deep Q-Learning Demonstration

What You'll Learn:

Neural Network

Speed Control

Analytics

Training Arena

Neural Network Parameters

Learning Rate

Exploration Rate

Hidden Layer Size

Batch Size

Memory Size

Discount Factor

Data Management

Training Controls

Under The Hood: Advanced AI Architecture

Deep Neural Architecture

Deep Q-Learning Algorithm