publications

2024

1. ICRA

   Enhancing Visual Inertial SLAM with Magnetic Measurements

   Bharat Joshi, and Ioannis Rekleitis

   In 2024 IEEE International Conference on Robotics and Automation (ICRA), May 2024

   Abs DOI Bib

   This paper presents an extension to visual inertial odometry (VIO) by introducing tightly-coupled fusion of magnetometer measurements. A sliding window of keyframes is optimized by minimizing re-projection errors, relative inertial errors, and relative magnetometer orientation errors. The results of IMU orientation propagation are used to efficiently transform magnetometer measurements between frames producing relative orientation constraints between consecutive frames. The soft and hard iron effects are calibrated using an ellipsoid fitting algorithm. The introduction of magnetometer data results in significant reductions in the orientation error and also in recovery of the true yaw orientation with respect to the magnetic north. The proposed framework operates in all environments with slow-varying magnetic fields, mainly outdoors and underwater. We have focused our work on the underwater domain, especially in underwater caves, as the narrow passage and turbulent flow make it difficult to perform loop closures and reset the localization drift. The underwater caves present challenges to VIO due to the absence of ambient light and the confined nature of the environment, while also being a crucial source of fresh water and providing valuable historical records. Experimental results from underwater caves demonstrate the improvements in accuracy and robustness introduced by the proposed VIO extension.

   @inproceedings{joshi_magnetometer_icra_2024,
     author = {Joshi, Bharat and Rekleitis, Ioannis},
     booktitle = {2024 IEEE International Conference on Robotics and Automation (ICRA)},
     title = {Enhancing Visual Inertial SLAM with Magnetic Measurements},
     year = {2024},
     pages = {10012-10019},
     doi = {10.1109/ICRA57147.2024.10611341},
     month = may,
   }

2023

1. OCEANS

   Hybrid Visual Inertial Odometry for Robust Underwater Estimation

   Bharat Joshi, Chanaka Bandara, Ioannis Poulakakis, and 2 more authors

   In OCEANS 2023 - MTS/IEEE U.S. Gulf Coast, Sep 2023

   Abs DOI Bib

   Vision-based state estimation is challenging in underwater environments due to color attenuation, low visibility and floating particulates. All visual-inertial estimators are prone to failure due to degradation in image quality. However, underwater robots are required to keep track of their pose during field deployments. We propose robust estimator fusing the robot’s dynamic and kinematic model with proprioceptive sensors to propagate the pose whenever visual-inertial odometry (VIO) fails. To detect the VIO failures, health tracking is used, which enables switching between pose estimates from VIO and a kinematic estimator. Loop closure implemented on weighted posegraph for global trajectory optimization. Experimental results from an Aqua2 Autonomous Underwater Vehicle field deployments demonstrates the robustness of our approach over different underwater environments such as over shipwrecks and coral reefs. The proposed hybrid approach is robust to VIO failures producing consistent trajectories even in harsh conditions.

   @inproceedings{joshi_hybrid_estimator_oceans_2023,
     author = {Joshi, Bharat and Bandara, Chanaka and Poulakakis, Ioannis and Tanner, Herbert G. and Rekleitis, Ioannis},
     booktitle = {OCEANS 2023 - MTS/IEEE U.S. Gulf Coast},
     title = {Hybrid Visual Inertial Odometry for Robust Underwater Estimation},
     year = {2023},
     pages = {1-7},
     doi = {10.23919/OCEANS52994.2023.10336994},
     issn = {0197-7385},
     month = sep,
   }

2. ICRA

   SM/VIO: Robust Underwater State Estimation Switching Between Model-based and Visual Inertial Odometry

   Bharat Joshi, Hunter Damron, Sharmin Rahman, and 1 more author

   In IEEE International Conference on Robotics and Automation (ICRA), Sep 2023

   Abs Bib

   This paper addresses the robustness problem of visual-inertial state estimation for underwater operations. Underwater robots operating in a challenging environment are required to know their pose at all times. All vision-based localization schemes are prone to failure due to poor visibility conditions, color loss, and lack of features. The proposed approach utilizes a model of the robot’s kinematics together with proprioceptive sensors to maintain the pose estimate during visual-inertial odometry (VIO) failures. Furthermore, the trajectories from successful VIO and the ones from the model-driven odometry are integrated in a coherent set that maintains a consistent pose at all times. Health-monitoring tracks the VIO process ensuring timely switches between the two estimators. Finally, loop closure is implemented on the overall trajectory. The resulting framework is a robust estimator switching between model-based and visual-inertial odometry (SM/VIO). Experimental results from numerous deployments of the Aqua2 vehicle demonstrate the robustness of our approach over coral reefs and a shipwreck.

   @inproceedings{joshi_robust_estimator_icra_2023,
     author = {Joshi, Bharat and Damron, Hunter and Rahman, Sharmin and Rekleitis, Ioannis},
     title = {SM/VIO: Robust Underwater State Estimation Switching Between Model-based
                      and Visual Inertial Odometry},
     booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
     year = {2023},
   }

3. ICRA

   Real-Time Dense 3D Mapping of Underwater Environments

   Weihan Wang, Bharat Joshi, Nathaniel Burgdorfer, and 4 more authors

   In IEEE International Conference on Robotics and Automation (ICRA), Sep 2023

   Abs Bib

   This paper addresses real-time dense 3D reconstruction for a resource-constrained Autonomous Underwater Vehicle (AUV). Underwater vision-guided operations are among the most challenging as they combine 3D motion in the presence of external forces, limited visibility, and absence of global positioning. Obstacle avoidance and effective path planning require online dense reconstructions of the environment. Autonomous operation is central to environmental monitoring, marine archaeology, resource utilization, and underwater cave exploration. To address this problem, we propose to use SVIn2, a robust VIO method, together with a real-time 3D reconstruction pipeline. We provide extensive evaluation on four challenging underwater datasets. Our pipeline produces comparable reconstruction with that of COLMAP, the state-of-the-art offline 3D reconstruction method, at high frame rates on a single CPU.

   @inproceedings{wang_underwater_dense_reconstruction_icra_2023,
     author = {Wang, Weihan and Joshi, Bharat and Burgdorfer, Nathaniel and Batsos, Konstantinos and {Quattrini Li}, Alberto and Mordohai, Philippos and Rekleitis, Ioannis},
     title = {Real-Time Dense 3D Mapping of Underwater Environments},
     booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
     year = {2023},
   }

4. ISSR

   Towards Mapping of Underwater Structures by a Team of Autonomous Underwater Vehicles

   Marios Xanthidis, Bharat Joshi, Monika Roznere, and 6 more authors

   In Robotics Research, Sep 2023

   Abs Bib

   In this paper, we discuss how to effectively map an underwater structure with a team of robots considering the specific challenges posed by the underwater environment. The overarching goal of this work is to produce high-definition, accurate, photorealistic representation of underwater structures. Due to the many limitations of vision underwater, operating at a distance from the structure results in degraded images that lack details, while operating close to the structure increases the accumulated uncertainty due to the limited viewing area which causes drifting. We propose a multi-robot mapping framework that utilizes two types of robots: proximal observers which map close to the structure and distal observers which provide localization for proximal observers and bird’s-eye-view situational awareness. The paper presents the fundamental components and related current results from real shipwrecks and simulations necessary to enable the proposed framework, including robust state estimation, real-time 3D mapping, and active perception navigation strategies for the two types of robots. Then, the paper outlines interesting research directions and plans to have a completely integrated framework that allows robots to map in harsh environments.

   @inproceedings{canthidis_multirobot_mapping_issr,
     author = {Xanthidis, Marios and Joshi, Bharat and Roznere, Monika and Wang, Weihan and Burgdorfer, Nathaniel and {Quattrini Li}, Alberto and Mordohai, Philippos and Nelakuditi, Srihari and Rekleitis, Ioannis},
     title = {Towards Mapping of Underwater Structures by a Team of Autonomous Underwater Vehicles},
     booktitle = {Robotics Research},
     year = {2023},
     publisher = {Springer Nature Switzerland},
     pages = {170--185},
     dio = {https://doi.org/10.1007/978-3-031-25555-7_12},
   }

2022

1. AUV

   Underwater Exploration and Mapping

   Bharat Joshi, Marios Xanthidis, Monika Roznere, and 4 more authors

   In IEEE/OES Autonomous Underwater Vehicles Symposium (AUV), Sep 2022

   Abs DOI Bib

   This paper analyzes the open challenges of exploring and mapping in the underwater realm with the goal of identifying research opportunities that will enable an Autonomous Underwater Vehicle (AUV) to robustly explore different environments. A taxonomy of environments based on their 3D structure is presented together with an analysis on how that influences the camera placement. The difference between exploration and coverage is presented and how they dictate different motion strategies. Loop closure, while critical for the accuracy of the resulting map, proves to be particularly challenging due to the limited field of view and the sensitivity to viewing direction. Experimental results of enforcing loop closures in underwater caves demonstrate a novel navigation strategy. Dense 3D mapping, both online and offline, as well as other sensor configurations are discussed following the presented taxonomy. Experimental results from field trials illustrate the above analysis.

   @inproceedings{joshi_auv_2022,
     author = {Joshi, Bharat and Xanthidis, Marios and Roznere, Monika and Burgdorfer, Nathaniel J. and Mordohai, Philippos and Li, Alberto Quattrini and Rekleitis, Ioannis},
     booktitle = {IEEE/OES Autonomous Underwater Vehicles Symposium (AUV)},
     title = {Underwater Exploration and Mapping},
     year = {2022},
     pages = {1-7},
     doi = {10.1109/AUV53081.2022.9965805},
   }

2. IFAC CAMS

   Multi-Robot Exploration of Underwater Structures

   Marios Xanthidis, Bharat Joshi, Jason M. O’Kane, and 1 more author

   IFAC-PapersOnLine, Sep 2022

   14th IFAC Conference on Control Applications in Marine Systems, Robotics, and Vehicles CAMS 2022

   Abs DOI Bib

   This paper discusses a novel approach for the exploration of an underwater structure. A team of robots splits into two roles: certain robots approach the structure collecting detailed information (proximal observers) while the rest (distal observers) keep a distance providing an overview of the mission and assist in the localization of the proximal observers via a Cooperative Localization framework. Proximal observers utilize a novel robust switching model-based/visual-inertial odometry to overcome vision-based localization failures. Exploration strategies for the proximal and the distal observer are discussed.

   @article{xanthidis_ifac_2022,
     title = {Multi-Robot Exploration of Underwater Structures},
     journal = {IFAC-PapersOnLine},
     volume = {55},
     number = {31},
     pages = {395-400},
     year = {2022},
     note = {14th IFAC Conference on Control Applications in Marine Systems, Robotics, and Vehicles CAMS 2022},
     issn = {2405-8963},
     doi = {10.1016/j.ifacol.2022.10.460},
     url = {https://www.sciencedirect.com/science/article/pii/S240589632202506X},
     author = {Xanthidis, Marios and Joshi, Bharat and O'Kane, Jason M. and Rekleitis, Ioannis},
   }

3. ICRA

   High Definition, Inexpensive, Underwater Mapping

   Bharat Joshi, Marios Xanthidis, Sharmin Rahman, and 1 more author

   In IEEE International Conference on Robotics and Automation (ICRA), Sep 2022

   Abs DOI Bib Code

   In this paper we present a complete framework for Underwater SLAM utilizing a single inexpensive sensor. Over the recent years, imaging technology of action cameras is producing stunning results even under the challenging conditions of the underwater domain. The GoPro 9 camera provides high definition video in synchronization with an Inertial Measurement Unit (IMU) data stream encoded in a single mp4 file. The visual inertial SLAM framework is augmented to adjust the map after each loop closure. Data collected at an artificial wreck of the coast of South Carolina and in caverns and caves in Florida demonstrate the robustness of the proposed approach in a variety of conditions.

   @inproceedings{joshi_gopro_icra_2022,
     author = {Joshi, Bharat and Xanthidis, Marios and Rahman, Sharmin and Rekleitis, Ioannis},
     title = {High Definition, Inexpensive, Underwater Mapping},
     booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
     year = {2022},
     pages = {1113-1121},
     doi = {10.1109/ICRA46639.2022.9811695},
   }

2021

1. ICRA_Workshop

   Towards Multi-Robot Shipwreck Mapping

   Marios Xanthidis, Bharat Joshi, Nare Karapetyan, and 8 more authors

   In Advanced Marine Robotics Technical Committee Workshop on Active Perception at IEEE International Conference on Robotics and Automation (ICRA), Sep 2021

   Abs Bib

   This paper introduces on-going work about a novel methodology for cooperative mapping of an underwater structure by a team of robots, focusing on accurate photorealistic mapping of shipwrecks. Submerged vessels present a history capsule and they appear all over the world; as such it is important to capture their state through visual sensors. The work in literature addresses the problem with a single expensive robot or robots with similar capabilities that loosely cooperate with each other. The proposed methodology utilizes vision as the primary sensor. Two types of robots, termed distal and proximal observers, having distinct roles, operate around the structure. The first type keeps a distance from the wreck providing a “bird’s”-eye-view of the wreck in sync with the pose of the vehicles of the other type. The second type operates near the wreck mapping in detail the exterior of the vessel. Preliminary results illustrate the potential of the proposed strategy.

   @inproceedings{icra_workshop_2021,
     author = {Xanthidis, Marios and Joshi, Bharat and Karapetyan, Nare and Roznere, Monika and Wang, Weihan and Johnson, James and {Quattrini Li}, Alberto and Casana, Jesse and Mordohai, Philippos and Nelakuditi, Srihari and Rekleitis, Ioannis},
     title = {Towards Multi-Robot Shipwreck Mapping},
     booktitle = {Advanced Marine Robotics Technical Committee Workshop on Active Perception at IEEE International Conference on Robotics and Automation (ICRA)},
     year = {2021},
   }

2020

1. IROS

   DeepURL: Deep Pose Estimation Framework for Underwater Relative Localization

   Bharat Joshi, Md Modasshir, Travis Manderson, and 5 more authors

   In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sep 2020

   Abs DOI Bib

   In this paper, we propose a real-time deep learning approach for determining the 6D relative pose of Autonomous Underwater Vehicles (AUV) from a single image. A team of autonomous robots localizing themselves in a communication-constrained underwater environment is essential for many applications such as underwater exploration, mapping, multi-robot convoying, and other multi-robot tasks. Due to the profound difficulty of collecting ground truth images with accurate 6D poses underwater, this work utilizes rendered images from the Unreal Game Engine simulation for training. An image-to-image translation network is employed to bridge the gap between the rendered and the real images producing synthetic images for training. The proposed method predicts the 6D pose of an AUV from a single image as 2D image keypoints representing 8 corners of the 3D model of the AUV, and then the 6D pose in the camera coordinates is determined using RANSAC-based PnP. Experimental results in real-world underwater environments (swimming pool and ocean) with different cameras demonstrate the robustness and accuracy of the proposed technique in terms of translation error and orientation error over the state-of-the-art methods. The code is publicly available.

   @inproceedings{joshi_deepurl_iros_2020,
     author = {Joshi, Bharat and Modasshir, Md and Manderson, Travis and Damron, Hunter and Xanthidis, Marios and {Quattrini Li}, Alberto and Rekleitis, Ioannis and Dudek, Gregory},
     title = {DeepURL: Deep Pose Estimation Framework for Underwater Relative Localization},
     booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
     year = {2020},
     pages = {1777-1784},
     doi = {10.1109/IROS45743.2020.9341201},
   }

2019

1. IROS

   Experimental Comparison of Open Source Visual-Inertial-Based State Estimation Algorithms in the Underwater Domain

   Bharat Joshi, Brennan Cain, James Johnson, and 8 more authors

   In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sep 2019

   Abs DOI Bib

   A plethora of state estimation techniques have appeared in the last decade using visual data, and more recently with added inertial data. Datasets typically used for evaluation include indoor and urban environments, where supporting videos have shown impressive performance. However, such techniques have not been fully evaluated in challenging conditions, such as the marine domain. In this paper, we compare ten recent open-source packages to provide insights on their performance and guidelines on addressing current challenges. Specifically, we selected direct and indirect methods that fuse camera and Inertial Measurement Unit (IMU) data together. Experiments are conducted by testing all packages on datasets collected over the years with underwater robots in our laboratory. All the datasets are made available online.

   @inproceedings{joshi_comppaper_iros_2019,
     author = {Joshi, Bharat and Cain, Brennan and Johnson, James and Kalitazkis, Michail and Rahman, Sharmin and Xanthidis, Marios and Hernandez, Alan and Karaperyan, Nare and {Quattrini Li}, Alberto and Vitzilaios, Nikolaos and Rekleitis, Ioannis},
     title = {Experimental Comparison of Open Source Visual-Inertial-Based State Estimation Algorithms in the Underwater Domain},
     booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
     year = {2019},
     doi = {10.1109/IROS40897.2019.8968049},
   }

2014

1. IOE

   Modeling, Simulation and Implementation of Brushed DC Motor Speed Control Using Optical Incremental Encoder Feedback

   Bharat Joshi, Rakesh Shrestha, and Ramesh Chaudhary

   In IOE Graduate Conference,, Sep 2014

   Bib

   @inproceedings{motorcontrol,
     author = {Joshi, Bharat and Shrestha, Rakesh and Chaudhary, Ramesh},
     title = {Modeling, Simulation and Implementation of Brushed DC Motor Speed Control Using Optical Incremental Encoder Feedback},
     booktitle = {IOE Graduate Conference,},
     year = {2014},
   }