Mobile Robot Navigation in Dynamic Environments

Author: Ahmed Yesuf Nurye
Advisor: Prof. Elżbieta Jarzębowska

Example simulation.

Abstract

We present a framework for mobile robot navigation in dynamic environments using Deep Reinforcement Learning (DRL) and the Robot Operating System (ROS). Traditional navigation methods often lack the real-time adaptability required in highly dynamic settings. To address this, we leverage the TD7 algorithm—an extension of the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm incorporating state and state-action embeddings—to directly map raw sensor inputs to control actions. These embeddings, trained to minimize the mean squared error (MSE) between the encoded state-action representation and the transition-predicted next state, enhance the system’s ability to model environment dynamics and improve navigation performance.

Extensive simulations were conducted in custom Gazebo environments of increasing complexity, ranging from open spaces to scenarios with static obstacles and moving actors. Performance was evaluated based on navigation success rate, time to goal, path efficiency, and collision rate. Results indicate that this approach consistently improves navigation performance, particularly in highly dynamic environments.

Network Architecture

To effectively capture dynamic actors in the environment, enabling the policy to make more informed actions, it is essential to predict the next state of the environment accurately. For this purpose, a pair of encoders (state and state-action encoders) are used.

Simulation Environment

Upon reset, the positions of the obstacles are randomly altered to enhance generalization, and new starting and target positions are generated randomly.

The framework was tested in Gazebo simulation environments with increasing level of complexity.

We have tested our system in a range of environments that vary in complexity to assess its robustness and effectiveness.

One of the potential application of this framework is exploration. We can use this framework to autonomously navigate in unknown environment and use SLAM frameworks to generate a map of that environment for further application.

@mastersthesis{Nurye-2024,
  author = {Nurye, Ahmed Y.},
  title = {Mobile Robot Navigation in Dynamic Environments},
  year = {2024},
  month = oct,
  school = {Warsaw University of Technology},
  address = {Warsaw, Poland},
  number = {WUT4f18e5c2cd214a9cb555f730fa440901},
  keywords = {Mobile Robot Navigation, Deep Reinforcement Learning, ROS2, Gazebo},
}