Getting Started with LIBVISO2 — Installation, Examples, and Tips

Getting Started with LIBVISO2 — Installation, Examples, and TipsLIBVISO2 is a lightweight C++ library for real-time stereo visual odometry. It provides efficient feature detection, matching, and motion estimation from stereo image pairs and has been widely used in robotics and autonomous driving research for fast ego-motion estimation. This guide walks you through installation, basic usage examples, parameter tuning tips, and common pitfalls to avoid.

---

Table of contents

• Introduction and overview
• Requirements and dependencies
• Installing LIBVISO2 from source
• Building a small example application
• Running LIBVISO2 on sample datasets (KITTI example)
• API overview and common parameters
• Troubleshooting and tips for robust results
• Performance optimization and profiling
• Integrating LIBVISO2 into a larger SLAM/robotics pipeline
• Licensing and alternatives

---

Introduction and overview

LIBVISO2 implements a stereo visual odometry pipeline including:

• feature detection and tracking using a FAST-like corner detector and local patch matching,
• robust motion estimation via RANSAC and least-squares optimization,
• optional outlier rejection and left-right consistency checks,
• fast sparse disparity-based triangulation for depth estimation.

It intentionally focuses on efficiency and simplicity rather than dense mapping, making it suitable as an ego-motion front end that can feed pose estimates into SLAM systems, sensor fusion with IMU, or real-time navigation stacks.

---

Requirements and dependencies

Minimum software and hardware requirements:

• C++ compiler with C++11 support (g++/clang on Linux/macOS; MSVC on Windows)
• CMake (>= 3.1 recommended)
• OpenCV (>= 2.4; 3.x or 4.x preferred for convenience; used for image I/O, optional visualization and debugging)
• Eigen (optional if you prefer to use Eigen for matrix math; LIBVISO2 includes its own lightweight math utilities)
• Standard build tools (make, ninja, or Visual Studio)

Hardware:

• A modern CPU for real-time performance; GPU is not required.
• Stereo camera pair (rectified images) for live use; for testing you can use public datasets (KITTI, Euroc, etc.).

---

Installing LIBVISO2 from source

Steps below assume a Unix-like system (Linux/macOS). Windows steps are similar but use Visual Studio or MSYS/MinGW.

1. Clone the repository:

   git clone https://github.com/raulmur/libviso2.git cd libviso2 

2. Create a build directory and run CMake:

   mkdir build cd build cmake .. -DCMAKE_BUILD_TYPE=Release -DUSE_OPENCV=ON 

   If CMake cannot find OpenCV, pass -DOpenCV_DIR=/path/to/opencv or ensure pkg-config paths are set.

3. Build:

   make -j$(nproc) 

   or use cmake --build . -- -jN on Windows.

4. Optionally run unit/examples provided in the repository.

Notes:

• If you prefer using system packages, install libopencv-dev (or similar) first.
• On macOS, use Homebrew to install OpenCV: brew install opencv.

---

Building a small example application

Below is a minimal outline of a C++ program that uses LIBVISO2’s VisualOdometryStereo class to process stereo frames and print estimated poses.

Example main.cpp:

#include <viso_stereo.h> #include <iostream> #include <vector> int main(int argc, char** argv) {     // stereo calibration parameters (fx, fy, cx, cy, base)     VisualOdometryStereo::parameters param;     param.calib.f = 721.5; // example from KITTI     param.calib.cu = 609.5;     param.calib.cv = 172.4;     param.base = 0.54; // baseline in meters     VisualOdometryStereo viso(param);     // load left/right images as grayscale byte arrays (uint8_t*)     // for example load from OpenCV Mat and pass .data pointer     for (int frame = 0; frame < num_frames; ++frame) {         // fill left_img and right_img with image data and widths/height         if (viso.process(left_img_data, right_img_data, width, height)) {             Matrix pose = viso.getMotion(); // 4x4 pose matrix (double)             std::cout << "Frame " << frame << " motion: " << pose << " ";         } else {             std::cout << "VO failed on frame " << frame << " ";         }     }     return 0; } 

Compile by linking libviso2 and OpenCV:

g++ main.cpp -I/path/to/libviso2 -L/path/to/libviso2/build -lviso2 `pkg-config --cflags --libs opencv4` -o vo_example 

Key points:

• Images must be rectified and same size.
• LIBVISO2 expects raw grayscale data in row-major order (uint8_t*). Converting from cv::Mat is straightforward (cv::Mat::data).

---

Running LIBVISO2 on sample datasets (KITTI example)

KITTI raw/sequences are commonly used. For the KITTI stereo odometry dataset:

1. Download left/right image sequences and calibration files.
2. Use calibration intrinsics to set VisualOdometryStereo::parameters (f, cu, cv, base).
3. Read images, convert to grayscale, and call viso.process for each stereo pair.

Example pseudo-code:

for each stereo frame:     left = imread(left_path, IMREAD_GRAYSCALE)     right = imread(right_path, IMREAD_GRAYSCALE)     viso.process(left.data, right.data, left.cols, left.rows)     pose = pose * viso.getMotion() // accumulate 

Accumulate poses to obtain full trajectory; write poses to text file for visualization (e.g., in MATLAB or Python).

---

API overview and common parameters

Important classes/functions:

• VisualOdometryStereo::parameters — camera intrinsics and algorithm settings.
  • param.calib.f (focal length), param.calib.cu/cv (principal point), param.base (baseline).
  • param.min_depth / param.max_depth — depth range for matches.
  • param.bucket_width / bucket_height — feature bucketing to ensure spatial distribution.
  • param.match_scale_levels — pyramid levels for matching.
• VisualOdometryStereo — constructor takes parameters.
  • process(left, right, width, height) — returns true on successful motion estimation.
  • getMotion() — returns the 4×4 motion matrix between current and previous frame.
  • getMatches() / getCorners() — access to feature matches and keypoints for debugging/visualization.

Typical parameters to tune:

• Feature bucket size and max number of features (controls density).
• RANSAC inlier threshold (pixel reprojection threshold).
• Depth range to discard matches that triangulate to unreasonable distances.
• Pyramid levels and patch size for matching under scale/illumination changes.

---

Troubleshooting and tips for robust results

• Ensure stereo images are rectified. Unrectified images break epipolar geometry and matching.
• Use accurate intrinsics and baseline; small errors cause scale and drift issues.
• Illumination changes: apply histogram equalization or CLAHE to both images to improve matching.
• Motion blur and low texture: reduce frame rate or use IMU fusion; LIBVISO2 is sparse and fails on textureless scenes.
• Outliers: increase RANSAC iterations or tune reprojection threshold.
• Rolling shutter cameras: results degrade; prefer global shutter for fast motion.
• Stereo matching window/patch size: larger patches are more robust but slower and less precise.
• If VO fails intermittently, skip frames and reinitialize rather than feeding bad poses downstream.

---

Performance optimization and profiling

• Run in Release build and enable compiler optimizations (-O3).
• Reduce image resolution for faster processing; tune bucket size to maintain features.
• Preallocate memory and reuse buffers to avoid frequent allocations.
• Use single-threaded lightweight loops; consider parallelizing independent tasks (e.g., image pre-processing) but the core VO pipeline is sequential.
• Profile hotspots (matching, undistort/rectify steps if added) with perf/valgrind/visual studio profiler.

---

Integrating LIBVISO2 into a larger SLAM/robotics pipeline

• Use LIBVISO2 as the odometry front-end and feed its poses into a back-end pose graph or EKF/UKF for loop closures and global consistency.
• Fuse with IMU for scale stability and robustness to fast motion (loose or tight coupling as needed).
• For mapping, combine sparse stereo depth from LIBVISO2 with a dense mapping module (e.g., TSDF or stereo dense matcher).
• Timestamp synchronization: ensure stereo frames and other sensors share a common timebase.

---

Licensing and alternatives

• LIBVISO2 is open-source (check the repository for the exact license). Confirm it fits your project’s license requirements.
• Alternatives:
  • ORB-SLAM3 / ORB-SLAM2 — full SLAM with loop closure and mapping (feature-based).
  • SVO / SVO2 — semi-direct visual odometry optimized for speed.
  • VINS-Mono / VINS-Fusion — visual-inertial odometry.
  • DSO / LSD-SLAM — direct methods (photometric), denser tracking in textured scenes.

Comparison (quick pros/cons):

Library	Pros	Cons
LIBVISO2	Lightweight, fast, stereo-focused	No loop closure, sparse output
ORB-SLAM3	Robust SLAM, loop closures	Heavier, more complex
SVO2	Very fast	Can be brittle in low texture
VINS-Fusion	Visual-inertial, accurate	Requires IMU, more complex

---

Final notes

LIBVISO2 is a practical, efficient choice when you need real-time stereo odometry with minimal complexity. It’s best used as a robust front-end combined with back-end modules for mapping, loop closure, or sensor fusion. Start with KITTI data to validate parameters, tune bucket and RANSAC settings, and progressively integrate with your robot stack.

If you want, I can:

• provide a ready-to-compile example repository layout and full CMakeLists.txt, or
• generate a tuned parameter set for a specific stereo camera model/dataset.