Over the last decade, there have been immense advancements in video and image processing technologies. This has enabled us to build camera vision systems. A lot of R&D is being carried out in the industries for building driverless vehicles, robots, security systems, medical systems etc. And having camera vision capability is an integral part of these systems.
1. Install OpenCV and V4L Libraries
Install OpenCV libraries, V4L and its dependencies using the commands shown below.
|
1 2 |
sudo apt-get update sudo apt-get install libopencv-dev python-opencv |
|
1 |
sudo apt-get install build-essential cmake |
|
1 |
sudo apt-get install qt5-default libvtk6-dev |
|
1 |
sudo apt-get install zlib1g-dev libwebp-dev libpng-dev libjasper-dev libtiff5-dev libopenexr-dev libgdal-dev libjpeg-dev |
|
1 |
sudo apt-get install libavcodec-dev libavformat-dev libdc1394-22-dev libswscale-dev libtheora-dev libxvidcore-dev libx264-dev libvorbis-dev yasm libopencore-amrnb-dev libopencore-amrwb-dev libv4l-dev libxine2-dev libtbb-dev |
|
1 |
sudo apt-get install v4l-utils |
2. Setup the Hardware
Setup the hardware connections as shown in Fig 1. And make sure that you use a 5V, 2A adapter to power the BBB. So that, the BBB can provide sufficient current to the Webcam. To SSH into your BBB, you can either use an ethernet cable or log in using a USB cable. To log in via USB cable, the IP address of the BBB is generally 192.168.7.2. I’d suggest folks to use an ethernet cable so that they can have access to the internet which will help to proceed with this tutorial.
You can use a WiFi adapter as well to connect to the internet, as described in one of my posts here. But, unfortunately, the BBB has just one USB port which will be used by the Webcam. If you still want to go wireless with your BBB, then you will need a USB hub. Using the USB hub you can connect both the Webcam as well as the WiFi adapter.
After connecting the Webcam to the BBB, we have to check if the BBB is able to detect the Webcam. And if it’s able to identify it as a video capturing device. We can check this by running the commands shown below.
|
1 2 3 4 5 |
root@beaglebone:~# lsusb Bus 001 Device 002: ID 046d:081b Logitech, Inc. Webcam C310 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub root@beaglebone:/dev# ls /dev/video* /dev/video0 |
In the above, It can be seen that the BBB is successfully able to detect the Logitech, Inc. Webcam C310 :). And “/dev/video0” indicates as a video capture device has been detected.
3. V4L Commands
We can use some v4l2 commands to check the type of formats supported by the camera and the current properties of the camera.The below commands can be used for checking the formats and properties of the camera.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
root@beaglebone:~# v4l2-ctl --list-formats ioctl: VIDIOC_ENUM_FMT Index : 0 Type : Video Capture Pixel Format: 'YUYV' Name : YUV 4:2:2 (YUYV) Index : 1 Type : Video Capture Pixel Format: 'MJPG' (compressed) Name : MJPEG root@beaglebone:~/BBB_CV/boneCV# v4l2-ctl --all Driver Info (not using libv4l2): Driver name : uvcvideo Card type : UVC Camera (046d:081b) Bus info : usb-musb-hdrc.1.auto-1 Driver version: 4.1.18 Capabilities : 0x84200001 Video Capture Streaming Extended Pix Format Device Capabilities Device Caps : 0x04200001 Video Capture Streaming Extended Pix Format Priority: 2 Video input : 0 (Camera 1: ok) Format Video Capture: Width/Height : 320/240 Pixel Format : 'YUYV' Field : None Bytes per Line: 640 Size Image : 153600 Colorspace : SRGB Flags : Crop Capability Video Capture: Bounds : Left 0, Top 0, Width 320, Height 240 Default : Left 0, Top 0, Width 320, Height 240 Pixel Aspect: 1/1 Selection: crop_default, Left 0, Top 0, Width 320, Height 240 Selection: crop_bounds, Left 0, Top 0, Width 320, Height 240 Streaming Parameters Video Capture: Capabilities : timeperframe Frames per second: 30.000 (30/1) Read buffers : 0 brightness (int) : min=0 max=255 step=1 default=128 value=128 contrast (int) : min=0 max=255 step=1 default=32 value=32 saturation (int) : min=0 max=255 step=1 default=32 value=32 white_balance_temperature_auto (bool) : default=1 value=1 gain (int) : min=0 max=255 step=1 default=0 value=20 power_line_frequency (menu) : min=0 max=2 default=2 value=2 white_balance_temperature (int) : min=0 max=10000 step=10 default=4000 value=5700 flags=inactive sharpness (int) : min=0 max=255 step=1 default=24 value=24 backlight_compensation (int) : min=0 max=1 step=1 default=1 value=1 exposure_auto (menu) : min=0 max=3 default=3 value=3 exposure_absolute (int) : min=1 max=10000 step=1 default=166 value=421 flags=inactive exposure_auto_priority (bool) : default=0 value=1 brightness (int) : min=0 max=255 step=1 default=128 value=128 contrast (int) : min=0 max=255 step=1 default=32 value=32 saturation (int) : min=0 max=255 step=1 default=32 value=32 white_balance_temperature_auto (bool) : default=1 value=1 gain (int) : min=0 max=255 step=1 default=0 value=20 power_line_frequency (menu) : min=0 max=2 default=2 value=2 white_balance_temperature (int) : min=0 max=10000 step=10 default=4000 value=5700 flags=inactive sharpness (int) : min=0 max=255 step=1 default=24 value=24 backlight_compensation (int) : min=0 max=1 step=1 default=1 value=1 |
You can also use the “v4l2-ctl –list-formats-ext” command to get a much more detailed information about formats supported by the camera.
4. Modifications in Source Code and Issues Encountered
The source code comprises of calls to the V4L API’s to capture videos in real time. I decided to take the source code from D. Molloy’s blog. I had to slightly modify the code to match with my webcam. When I ran the program on the BBB I got stuck with an issue, caused due to the select() call timing out. I started digging into the great internet and came across several folks facing the same issue. However, I did not find any useful information to understand why the select() call was timing out.
After a couple of hours of sniffing around the internet with no luck, I decided to take a look at the source code and especially the select() system call. In the source code, I came across a “timeout” parameter that is passed as an argument to the select() API. This timeout parameter was set to 2 seconds. Then I presumed that the case may be such that, the select() is timing out and returning before the data is even ready or available for the BBB from the webcam. So, whenever I tried to capture the video using a 2 seconds time out value, the select() timeout error used to occur as shown below.
|
1 2 3 |
root@beaglebone:~/BeagleBone_VideoCapture_DE# ./video_capture -f -o -c 600 > out.raw Force Format 1 select timeout |
Hence, In the source code, I simply increased this timeout value to 30 seconds(this was just a random selection) as shown below.
|
1 2 3 4 |
/* Timeout. */ tv.tv_sec = 30; tv.tv_usec = 0; r = select (fd + 1, &fds, NULL, NULL, &tv); |
This resolved the select timeout issue and I was able to capture the video properly. Another issue I have encountered is that for some reason, the BBB is unable to capture YUV videos for resolutions >= 640×480. I haven’t still figured out why this is happening as it may have something to do with usage of the V4L drivers. However, I am able to capture videos of resolution 320×240 and the video quality is pretty decent for a start. I am currently also looking into the issue of not being able to capture HD videos of resolutions > 640×480 and will keep you posted.
You can get the source code and build script by cloning from my Github repository as shown below or download it from here.
|
1 |
git clone https://github.com/deeplyembeddedWP/BeagleBone_VideoCapture_DE.git |
The below describes the instance where the source code has been changed to capture videos of resolution 320×240.
|
1 2 3 4 5 6 |
else if (force_format == 1) { fmt.fmt.pix.width = 320; fmt.fmt.pix.height = 240; ... } |
5. Build Program and Capture Video
Run the build script to generate the binary within the source directory as shown below.
|
1 2 3 4 5 6 7 8 9 10 |
root@beaglebone:~# cd BeagleBone_VideoCapture_DE/ root@beaglebone:~/BeagleBone_VideoCapture_DE# ls -l total 24 -rw-r--r-- 1 root root 867 Aug 25 08:26 README.md -rwxr-xr-x 1 root root 258 Aug 25 07:27 build -rw-r--r-- 1 root root 14565 Aug 25 16:51 video_capture.c root@beaglebone:~/BeagleBone_VideoCapture_DE# ./build Capture Real-time Video using BeagleBone Black - deeplyembedded.org Building the V4L video capture program for resolution of 340x240 Done! |
Passing an argument of -h will display all possible command options you can pass to the program as shown below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
root@beaglebone:~/BeagleBone_VideoCapture_DE# ./video_capture -h Usage: ./video_capture [options] Version 1.3 Options: -d | --device name Video device name [/dev/video0] -h | --help Print this message -m | --mmap Use memory mapped buffers [default] -r | --read Use read() calls -u | --userp Use application allocated buffers -o | --output Outputs stream to stdout -f | --format Force format to 640x480 YUYV -F | --formatH264 Force format to 1920x1080 H264 -c | --count Number of frames to grab [100] - use 0 for infinite Example usage: capture -F -o -c 300 > output.raw Captures 300 frames of H264 at 1920x1080 - use raw2mpg4 script to convert to mpg4 |
We can start the video capture by running the below command. The video output generated by the program will be in raw format.
|
1 2 3 4 5 6 7 8 9 10 |
root@beaglebone:~/BeagleBone_VideoCapture_DE# ./video_capture -f -o -c 600 > out.raw Force Format 1 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ root@beaglebone:~/BeagleBone_VideoCapture_DE# ls -l total 90044 -rw-r--r-- 1 root root 867 Aug 25 08:26 README.md -rwxr-xr-x 1 root root 258 Aug 25 07:27 build -rw-r--r-- 1 root root 92160000 Aug 25 17:02 out.raw -rwxr-xr-x 1 root root 13032 Aug 25 17:01 video_capture -rw-r--r-- 1 root root 14565 Aug 25 16:51 video_capture.c |
The raw output video is converted to mp4 format using FFmpeg as shown below with a frame rate of 30fps.
|
1 |
root@beaglebone:~/BeagleBone_VideoCapture_DE# ffmpeg -s 320x240 -framerate 30 -pix_fmt yuyv422 -f rawvideo -i out.raw output.mp4 |
From your host machine(PC or Laptop) fetch the output.mp4 video from the BBB using sftp as shown below. You can then view the captured video on your laptop.
|
1 2 3 4 5 6 7 8 9 |
vinaydivakar@debian:~$ sftp root@192.168.7.2 Connected to 192.168.7.2. sftp> lcd Desktop/ sftp> cd BeagleBone_VideoCapture_DE/ sftp> get output.mp4 Fetching /root/BeagleBone_VideoCapture_DE/output.mp4 to output.mp4 /root/BeagleBone_VideoCapture_DE/output.mp4 100% 662KB 662.4KB/s 00:00 sftp> exit vinaydivakar@debian:~$ |
The below video will walk you through the entire build and video conversion process described above. The video also shows a live demo of the capture process and the quality of the captured video. You can find all the relevant details of the V4L API’s used in the source code here. I shall continue to research and figure out a way to capture high-resolution YUV videos using the C310. And will keep updating this post accordingly as an when there are incremental improvements.
In the next week, I will probably look on how to capture and process images using some image processing techniques with the BBB. And once I am done, I will share the same with you. So, see you soon in our next session at DeeplyEmbedded.