Table of Contents
- Introduction
- Integrating OpenGL in ROS
- Compiling Customized OpenCV for ROS
- Editing CMakeLists.txt
- Testing OpenGL_ROS package
- Code Explained
- OpenCV-OpenGL interface
- Summary
- Reference
Introduction
If you are working on ROS and want to implement simple Augmented reality applications using OpenGL and OpenCV, this post is for you. The AR applications may have the camera image on the background and drawing a 3D object on the top of the image. In this post, you can see how to draw a 3D object, say a 3D cube on the top of an image. You can apply your own algorithms to move the 3D object based on some object detection algorithms in OpenCV.
Integrating OpenGL in ROS
If you have ROS desktop full installation in your PC, you may have OpenCV installed along with the ROS installation. The OpenCV version comes along with ROS is not compiled with WITH_OPENGL=ON configuration. If it is not compiled with OpenGL, we can’t draw any 3D objects on the top of an OpenCV image.
In order to enable OpenGL on OpenCV which is installed along with ROS, you can use the following method.
Compiling Customized OpenCV for ROS
In order to enable OpenGL in OpenCV, you have to build OpenCV from source compilation.
Note: The OpenCV version of the source code should be the same as the installed opencv version. Otherwise, the OpenCV-ROS cv bridge will not work properly. In order to avoid ROS-OpenCV compatibility issues, we have to go for the same version of OpenCV. You can use the following script in order to do this task. You can either build opencv locally or you can install it on the system. The safest method is, installing locally and provide the path of opencv in CMakeLists.txt in order to build it from the local version.
You can clone the following OpenGL_ROS
package to your catkin workspace in order to get started. The package is also having the shell script to install OpenCV.
cd /<path to your catkin_ws>/src/
git clone https://github.com/qboticslabs/opengl_ros
cd opengl_ros
gedit scripts/install_opencv.sh
Here is the install_opencv.sh
script for installing customized OpenCV for ROS.

If you want to install customized OpenCV, first edit the configuration inside this script and save it. The main configuration parameters are;
- OPENCV_INSTALL: If it ‘YES‘, it will build opencv and install into the system path. If ‘NO‘, it will build locally.
Note: I will refer you to stick with the local build of OpenCV first because it is safe to work with. Installing OpenCV may cause an undesired effect on your system. - WITH_CUDA: If you have NVIDIA GPU and if CUDA installed on your system, you can set the config as ‘ON’. This can accelerate the processing using GPU.
- WITH_OPENGL: You have to put this configuration ON for compiling OpenCV with OpenGL.
To install the script you can simply follow the command below. Switch to opengl_ros
folder and execute the following command
./scripts/install_opencv.sh
It will take from 20 min to 2 hr to build OpenCV. The time will depend on your configuration. Make sure the build is successful before going to the next section. This script will create a folder called opencv_dev_install
in your home folder and clone the open cv source code to this folder. If you choose the install option, it may be installed in your system path.
Editing CMakeLists.txt
After building OpenCV, you may have to do some editing in CMakeLists.txt
of the opengl_ros
package, that you have already cloned in the src folder of your catkin workspace.
I have tested this package in ROS Kinetic and Melodic. The current version of CMakeLists.txt
is for ROS Melodic. You can do some simple customization in order to make it compile with Kinetic.
Here are main sections in the CMakeLists.txt

Note: For ROS Kinect users, just search for melodic in the catkin_INCLUDE_DIRS
, and catkin_LIBRARIES
variable and replace it with kinetic.
You can save the file and build the package in the workspace using the catkin_make
command.
catkin_make
If your build is successful, you can able to run the opengl_ros
node.
Testing OpenGL_ROS package
In order to test the opengl_ros
node, you may use an existing package in ROS. This package will basically open your webcam and publish the images in ROS Topic: ‘/usb_cam/image_raw
‘. We can install this package using the following command
Installing usb_cam
package in ROS Melodic
sudo apt install ros-melodic-usb-cam
After installing the usb_cam
package, you can try the following commands to start the opengl_ros
node.
Note: Make sure that you have webcam that is compatible with Ubuntu. Use cheese application in Ubuntu to check the webcam is working.
Launch usb_cam
node to start the camera.
roslaunch usb_cam usb_cam-test.launch
Start the OpenGL-ROS node using the following command
rosrun opengl_ros opengl_ros
Here is the output video
Code Explained
You can find the OpenGL_ROS code cv_gl_ros.cpp
from opengl_ros/src/
.
Here is the flow of the opengl_ros
node.

Acquiring ROS Image messages
If you check the cv_gl_ros.cpp
, you can find a class ImageROS
, which helps to receive and publish ROS images.

Initializing track bar and OpenGL rendering Window
You can see the initialization of the OpenGL render window and two trackbars. One is for changing the x_rotation
and others for changing y_rotation
of the 3D cube.

Render 2D texture and 3D object
The on_opengl(void* param)
is a callback function that is responsible for drawing the 2D texture in the background and drawing a 3D cube on top of it. In this code, the 2D texture is the input image itself. The callback read the trackbar position and render the cube orientation based on that value.

Here is the while()
loop which is doing the image acquisition, 2D and 3D rendering in OpenGL window, and checking for ‘Esc‘ key in order to quit the program.

OpenCV-OpenGL interface
You can find the existing integration of OpenCV-OpenGL APIs from the following link.
Summary
In this post, you have seen how to create simple Augmented reality using OpenGL, OpenCV, and ROS. We have seen how to enable OpenGL on OpenCV which is installed along with ROS. We have seen a script to install OpenCV by source compilation and have discussed how to customize in order to compile with custom OpenCV, ROS Melodic, and Kinetic. After that, we have tested the opengl_ros node and finally, you have seen a brief explanation of the code.
Hello I am so glad I found your blog, I really found you by error,
while I was searching on Aol for something else, Regardless I am here now and would just like to say thanks for a fantastic post and a all round exciting
blog (I also love the theme/design), I don’t have time to read it all at the moment but I have book-marked it and also added in your RSS feeds, so when I have
time I will be back to read more, Please do keep up the superb jo.
Hello, I read your blog regularly. Your story-telling style is awesome, keep up the good work!
Hello there. I discovered your website by way of Google even as searching for a similar matter, your website got here up. It seems to be great. I have bookmarked it in my google bookmarks to come back then. Serene Tybalt Doralyn
Thank you so much. These are beautiful and inspiring! Phylis Zak Cuthbertson