RNBF: Real-Time RGB-D Based Neural Barrier Functions for Safe Robotic Navigation

Satyajeet Das1, Yifan Xue2, Haoming Li2, Nadia Figueroa2
1University of Southern California    2University of Pennsylvania
Paper (arXiv) Code Videos BibTeX

RNBF teaser (replace media/teaser.png)

Real-world experiments: Shown are collision-free RNBF-CBF-QP trajectories overlaid on video frames for static (single obstacle - shape 1 & 2, and multi obstacle) and quasi-static obstacle scenarios.

Abstract

Autonomous safe navigation in unstructured and novel environments poses significant challenges, especially when environment information can only be provided through low-cost vision sensors. Although safe reactive approaches have been proposed to ensure robot safety in complex environments, many base their theory off the assumption that the robot has prior knowledge on obstacle locations and geometries. In this paper, we present a real-time, vision-based framework that constructs continuous, first-order differentiable Signed Distance Fields (SDFs) of unknown environments entirely online, without any pre-training, and is fully compatible with established SDF-based reactive controllers. To achieve robust performance under practical sensing conditions, our approach explicitly accounts for noise in affordable RGB-D cameras, refining the neural SDF representation online for smoother geometry and stable gradient estimates. We validate the proposed method in simulation and real-world experiments using a Fetch robot.

Method Overview

RNBF is a real-time, vision-based safety framework that constructs a continuous, first-order differentiable neural Signed Distance Field (SDF) online from posed RGB-D observations. The learned SDF provides both distance values and spatial gradients, enabling direct integration with SDF-based reactive controllers and safety filters. While RNBF is compatible with any controller that consumes an SDF and its gradient, in this work we demonstrate its performance using a Control Barrier Function Quadratic Program (CBF-QP).

RNBF pipeline overview (SDF generation + reactive control)
RNBF-Control pipeline. The SDF generation module reconstructs a neural SDF online from RGB-D input (5–15 Hz). The reactive controller queries the SDF and its gradients at higher frequency (≥ 20 Hz) to enforce safety.

Effect of Depth Noise on SDF

Affordable RGB-D sensors introduce depth noise that can create spurious surface artifacts, leading to unstable SDF gradients and unreliable reactive control. RNBF mitigates this by using robust near-surface supervision (Huber loss), producing smoother reconstructions and more stable gradients under noisy depth measurements.

Effect of depth noise on SDF reconstruction (placeholder)
Comparison of SDF reconstructions under noisy RGB-D depth inputs (replace with your figure and caption). Comparison of SDF reconstructions using L1 loss & huber loss functions on depth data from an Intel RealSense D435i sensor. The leftmost column shows the RGB images of the scene. The second column presents reconstructions using the L1 loss. The remaining columns show reconstructions using the huber loss with varying δ values. Each row corresponds to a scene with a different camera-to-surface distance: in the first row, the wall is approximately 2 meters from the camera; in the second row, the racks and back wall are approximately 7 meters away; and in the third row, the closest obstacle is approximately 3 meters away.

Videos

RNBF Experiment Video

BibTeX

@misc{das2025rnbfrealtimergbdbased,
  title         = {RNBF: Real-Time RGB-D Based Neural Barrier Functions for Safe Robotic Navigation},
  author        = {Satyajeet Das and Yifan Xue and Haoming Li and Nadia Figueroa},
  year          = {2025},
  eprint        = {2505.02294},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2505.02294}
}