GPU programming is a critical component in AI system courses, which is notoriously difficult to learn and teach, given its different programming models from the CPU, such as massive parallelism and data movement across memory hierarchies. This paper presents Tensor-Viz, an innovative visualization toolkit that helps students learn GPU programming using Triton, one of the most widely used programming languages to develop AI applications. Tensor-Viz offers an intuitive interface with interactive visualizations of GPU operations from multiple perspectives, including parallelism, memory access, and performance metrics. When integrated into educational materials, Tensor-Viz enhances hands-on learning and improves comprehension of GPU programming concepts and AI algorithms in real applications, as demonstrated in a user study conducted with Computer Science students. The positive feedback from this study highlights the utility of Tensor-Viz in educational settings and its potential to bridge the gap between the theoretical algorithm and practical implementations in AI courses.