Neural networks and reinforcement learning (RL) are fundamental to machine learning (ML) and AI. Given the widespread adoption of AI algorithms in the industrial sectors, ensuring students understand these concepts will prepare them for a technology-driven job market. In this experience report, we introduce NeuRL, a free and accessible no-code web-based application that allows innovative real-time exploration of RL and neural networks. NeuRL provides interactive 3D WebGL environments, enabling students to experiment with multiple popular RL algorithms and observe the evolution of agents and neural networks as agents learn to accomplish tasks. To ensure NeuRL runs smoothly on low-performance computers, we created a custom neural network and RL library written in the OpenGL Shading Language (GLSL). This enables students to teach 256 independent agents how to complete tasks in parallel at their display screen’s refresh rate. To evaluate NeuRL’s effectiveness, we introduced it to teach RL fundamentals to 96 students enrolled in an ML course. After the lesson, students completed a survey that assessed NeuRL’s usability and learning effectiveness. Students found NeuRL easy to use and enjoyed its inclusion during the lesson . To the best of our knowledge, NeuRL is the first tool that enables students from any background to explore RL and observe both neural networks and agent behaviors in real-time. NeuRL demonstrates the feasibility and value of providing accessible web-based tools that empower students to explore AI concepts in a manner that transcends conventional teaching methodologies.