Reinforcement learning (RL) is emerging as a powerful tool in materials science, delivering a paradigm shift in how we find and optimize high-dimensional chemical and structural spaces. Unlike traditional methods, RL agents are able to learn to explore complex energy landscapes in an adaptive manner, instantaneously making decisions that guide the discovery of novel materials with certain properties. However, the application of RL to materials discovery faces unique challenges, including data scarcity, computationally expensive, and the challenge of designing reward functions that can balance multiple material objectives optimally. In this review, the current challenges and difficulties in applying RLtechniques in materials science and recent advances combining RL with machine learning, generative models, and domain knowledge are emphasized. We also outline promising future directions, such as transfer learning, hybrid models, and the creation of collaborative, open-access data infrastructures. By addressing these challenges, RL has the potential to transform the discovery and design of functional materials for catalysis, energy storage, and sustainability applications.