Path planning in swarm robotics exploration using SARSA and ACO algorithms

Swarm robotics is a particularly promising approach for autonomous exploration in complex and uncertain environments, with applications ranging from environmental monitoring to hazardous-area inspection. A major challenge lies in optimising robot trajectories to minimise travel distance while ensuring comprehensive and effective coverage of the exploration area. In this context, we propose a hybrid path-planning framework that combines the SARSA Reinforcement Learning algorithm with the ACO approach, drawing inspiration from collective coordination mechanisms in nature, particularly the use of pheromones as a medium for self-organisation. This framework leverages both individual learning and swarm intelligence in a complementary manner, thereby enabling more robust, scalable, and efficient exploration. A comparative analysis of the two methods was conducted to identify the most effective approach for optimising robot trajectories while minimising energy consumption. In this process, robots take into account obstacle avoidance, whether obstacles are traversable, using either pheromone-based environmental marking or reinforcement learning strategies. Simulation results demonstrate the effectiveness of a hybrid model that integrates SARSA with ACO, significantly enhancing trajectory quality and exploration coverage. However, they also reveal that increasing the environment size substantially increases the total travel distance and slows SARSA convergence due to the expansion of the state space. To overcome this limitation, future work will explore neural network–based value function approximation, which is expected to improve generalisation and accelerate convergence in large-scale scenarios.