Life is a game, take it seriously

Recording 3D video(oni) files that align rgb image with depth image

In Computer Vision, Point Cloud Library on July 15, 2012 at 10:51 am

by Gooly (Li Yang Ku)

Kinect or xtion like devices provide an easy way to capture 3D depth image or videos. The OpenNI interface that is compatible with these devices comes with a handy tool “NiViewer” that captures 3D video into an oni file. The tool is located under “Samples/Bin/x64-Release/NiViewer” for Linux; and should be in the startup menu if you use Windows. After starting up the tool, you can right click and show the menu. By clicking “Start Capture” and then “Stop” should generate an oni video file in the same folder.

However the rgb video and depth video in the oni file are recorded seperately and not aligned. (Turns out it should be aligned if you press 9, but it didn’t work on my machine, see comment) This is due to the camera position difference between the IR sensor and the rgb sensor. OpenNI do provide functions to adjust the depth image to match the rgb image (Note that it is not doable the other way around). By adding some additional code to the NiViewer, you should be able to record depth video that is aligned with the rgb image.

First open the file “src/NiViewer/Capture.cpp”; change the code that adds the depth node under the “captureFrame()” function to the following.

nRetVal = g_Capture.pRecorder->AddNodeToRecording(*getDepthGenerator(), g_Capture.nodes[CAPTURE_DEPTH_NODE].captureFormat);
START_CAPTURE_CHECK_RC(nRetVal, "add depth node");
g_Capture.nodes[CAPTURE_DEPTH_NODE].bRecording = TRUE;
DepthGenerator* depth = getDepthGenerator();
depth->SetIntProperty ("RegistrationType", 1);
nRetVal = depth->GetAlternativeViewPointCap().SetViewPoint(*getImageGenerator());
if(XN_STATUS_OK != nRetVal)
	depth->SetIntProperty ("RegistrationType", 2);
	nRetVal = depth->GetAlternativeViewPointCap().SetViewPoint(*getImageGenerator());
	if(XN_STATUS_OK != nRetVal)
		displayMessage("Getting and setting AlternativeViewPoint failed");
g_Capture.nodes[CAPTURE_DEPTH_NODE].pGenerator = depth;

Then type make in the terminal under the NiViewer folder. The new NiViewer binary should be generated.
Kinect and xtion sensor uses different methods to generate this alternative view depth image. Kinect does the adjustment in software and xtion does it in hardware. This is why different “RegistrationType” are set in the code above.

  1. I think there’s a way to do it without writing any code… Just enter one of the Depth+RGB overlay modes, such as pressing ‘8’, ‘9’, ‘0’, or ‘-‘, and the registration will be enabled. Then if you record, the registration will stay enabled.

  2. Thanks, it’s funny that I didn’t notice that. However, it didn’t work when I tried it today. I look at the code and it did set the alternative view point under device.cpp. I guess it probably didn’t set the RegistrationType correctly or I got an older version of NiViewer. I’ll change the post to mention that.

  3. Is there a standalone program to record. .ONI depth files?

    • NiViewer is the program for recording depth file. You shouldn’t need to change any code if you use the new version.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: