Mobile Robot Navigation in Dynamic Environments

Author: Ahmed Yesuf Nurye
Advisor: Prof. Elżbieta Jarzębowska

Example simulation.

Abstract

This project presents a novel framework for mobile robot navigation in dynamic environments using Deep Reinforcement Learning (DRL). The framework employs the TD7 algorithm, an augmentation of the TD3 algorithm, with state-action embeddings to predict the next environment state and better model the environemnt dynamics. Simulated in Gazebo and implemented with ROS2, the system was validated across various environments, demonstrating superior adaptability and performance compared to the baseline method.

Network Architecture

To effectively capture dynamic actors in the environment, enabling the policy to make more informed actions, it is essential to predict the next state of the environment accurately. For this purpose, we employ a pair of encoders (state and state-action encoders).

Simulation Environment

Upon reset, the positions of the obstacles are randomly altered to enhance generalization, and new starting and target positions are generated randomly.

The framework was tested in Gazebo simulation environments with varying complexity.

We have tested our system in a range of environments that vary in complexity to assess its robustness and effectiveness.

One of the potential application of this framework is exploration. We can use this framework to autonomously navigate in unknown environment and use SLAM frameworks to generate a map of that environment for further application.

@mastersthesis{Nurye-2024,
  author = {Nurye, Ahmed Y.},
  title = {Mobile Robot Navigation in Dynamic Environments},
  year = {2024},
  month = oct,
  school = {Warsaw University of Technology},
  address = {Warsaw, Poland},
  number = {WUT4f18e5c2cd214a9cb555f730fa440901},
  keywords = {Mobile Robot Navigation, Deep Reinforcement Learning, ROS2, Gazebo},
}