Detailed javacv development 1: call the local camera video

Detailed javacv development 1: call the local camera video

Quick transfer to: javacv getting started guide: prologue

javaCV series articles:

Javacv development detailed explanation 1: call the local camera video

JavaCV development detailed explanation 2: Implementation of the push streamer, push the local camera video to the streaming media server and the realization of the camera recording video function (based on javaCV-FFMPEG, javaCV-openCV)

JavaCV development detailed explanation 3: The realization of the receiver, recording the rtsp/rtmp video file of the streaming media server (based on javaCV-FFMPEG)

JavaCV development detailed explanation 4: The realization of the streamer (also can be used as a local streamer, pusher, add picture and text watermark, video image frame save), realize rtsp/rtmp/local file forwarding to rtmp streaming server (Based on javaCV-FFMPEG)

JavaCV development detailed explanation 5: Record audio (recording microphone) to a local file/streaming server (based on javax.sound, javaCV-FFMPEG)

JavaCV development detailed explanation 6: local audio (microphone equipment) and video (camera) capture, mix and push (record) to the server (local)

JavaCV development detailed explanation 7: Make audio conversion easier, realize the conversion function of audio parameters such as general audio coding format conversion, resampling (take pcm16le encoded wav to mp3 as an example)

JavaCV Development Detailed Explanation No. 8: Transform the application encapsulated in the rtsp to rtmp stream (no need to transcode, lower resource consumption, better performance, lower latency)


Audio and video encoding and decoding issues: how does javaCV quickly perform audio preprocessing and demultiplexing encoding and decoding (based on javaCV-FFMPEG)

Audio and video encoding and decoding issues: 16/24/32 bit audio byte[] is converted to little-endian short[], int[], take byte[] to short[] as an example

Realize adding picture watermark or text watermark to pictures (also supports adding watermark to video image frames)

Java natively implements functions such as screen device traversal and screen capture (capture)

javacpp-ffmpeg series:

javacpp-FFmpeg series 1: The video is pulled and decoded into YUVJ420P and saved as a jpg image

javacpp-FFmpeg series 2: Universal pull-stream decoder, supports video pull-stream decoding and conversion to image pixel data such as YUV, BGR24 or RGB24

javacpp-FFmpeg series 3: Image data conversion (BGR and BufferdImage conversion, RGB and BufferdImage conversion)

Supplement to javacpp-FFmpeg series: FFmpeg solves the problem of too long retrieval time for avformat_find_stream_info

javacpp-opencv series:

1. javaCV image processing 1: add text watermark to real-time video and capture the video image and save it as a picture to realize the font, position, size, thickness, flip, smoothing and other operations of text watermark

2. javaCV image processing 2: add picture watermark to real-time video, realize the superposition of different sizes of pictures, and control the image transparency

3. java cv image processing 3: use the opencv native method to traverse the camera equipment and call (convenient multi-camera traversal and call, compared to javacv faster camera reading speed and efficiency, convenient image processing after reading)

4. javacv image processing series: domestic vehicle license plate detection and recognition system (accuracy rate of more than 99.7% in 10,000 copies)

Preface: The javacv development kit is a set of development kits used to support java multimedia development. It can be used for local multimedia (audio and video) calls and post-operations of audio, video, pictures and other files (picture modification, audio and video decoding and editing, etc.). Here only use the simplest local camera call to demonstrate the basic functions of javacv


It is recommended to use the latest javaCV1.3 version, which has resolved most of the bugs found in earlier versions

The javacv series of articles use 6 jar packages:

javacv.jar, javacpp.jar, ffmpeg.jar, ffmpeg-system platform.jar, opencv.jar, opencv-system platform.jar.

Among them, the system platform in ffmpeg-system platform.jar and opencv-system platform.jar are changed to the corresponding jar package according to the development environment or test deployment environment. For example, the windows7 64-bit system is replaced with ffmpeg-x86-x64.jar

Why do you want to do this: Because ffmpeg-system platform.jar stores the c/c++ local so/dll library, and ffmpeg.jar is the implementation of the corresponding native library java interface encapsulated by javacpp, and javacpp is a function based on jni Flexible packaging package, easy to implement jni, javacv.jar is a secondary packaging of 9 vision libraries, but the realized functions are limited, so it is recommended that novices are familiar with the APIs of the two C/C++ libraries, openCV and ffmpeg, and then look at javaCV. The idea will be very clear.


The javacv series of articles use ffmpeg for audio and video processing by default, opencv for image processing, and opencv for camera capture

The official github maintenance address of javacv :

1. Use the latest javacv1.4.4

(Note: Please actively and happily replace the dependency packages downloaded from other places with the official jar package and the jar package provided by the blogger; if you use other jar package versions and cause errors, don t rush to ask the blogger why it will report an error, first put the jar Replace the package and try again)

(1) Use maven to add dependencies


(2) Use gradle to add dependencies

dependencies {
    compile group: 'org.bytedeco', name: 'javacv-platform', version: '1.4.4'

(3) Use the local jar package method

The latest version is too big. If you need to download the full package, please go to the official to download

Download javaCV1.3.3 version (full package, file name: , size: 212M):

Download the javaCV1.3 version:

1. javacv1.3 core package: (including javacv.jar, javacpp.jar and javacv-platform.jar)

2. The javacv1.3-opencv package collection:

3. Javacv1.3-ffmpeg package collection:

Collection of all jar packages of javaCV1.3:

Download all the jar packages of javaCV1.2 here:

The following part is javacv1.2 version, it is no longer recommended to download

Basic dependency package: javacv.jar ; javacpp.jar (must be version 1.2 or higher)

Instructions for using jar package:

Opencv dependencies for windows x64 platform: opencv.jar ; oepncv-windows-x86_64.jar (replace other platforms with the corresponding jar package).


1. If it is an Apple Mac system, replace oepncv-windows-x86_64.jar with this package opencv-macosx-x86_64.jar
2. Replace the Linux platform with this package: opencv-linux-x86_64.jar

3. Replace the Android platform with opencv-android-arm.jar or opencv-android-x86.jar

If you want to reduce the size of dependent packages in maven and gradle, you need to manually exclude unnecessary platform dependencies.

2. Why do not need to install opencv?

Starting from javacv0.8, there is no need to install opencv locally, just refer to the reference package of the system platform corresponding to opencv.

(For example, oepncv-windows-x86_64.jar is a typical 64-bit windows environment dependency package)

3. Java obtains the camera video of windows through javacv

The real-time video image interface of the camera finally called:

Simple implementation of previewing the video image of the local camera:

 *  windows 
 *  2016 6 13 
package cc.eguid.javacv;

import javax.swing.JFrame;

import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.javacv.FrameGrabber.Exception;
import org.bytedeco.javacv.OpenCVFrameGrabber;

 * @author eguid  
 * @version 2016 6 13   
 * @see javavcCameraTest  
 * @since  javacv1.2

public class JavavcCameraTest
public static void main(String[] args) throws Exception, InterruptedException
    OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(0);  
    grabber.start();  //
    CanvasFrame canvas = new CanvasFrame(" ");//
        canvas.showImage(grabber.grab());//  Frame frame=grabber.grab(); frame 


Is it very simple, the originally complicated streaming media operation, javaCV can help us quickly realize it.

Support eguid original

The next chapter will explain how to record the video of the local camera into a file and push it to the streaming server

JavaCV development detailed explanation 2: Push streamer implementation and camera recording video function (support windows/linux/mac/Android/ios all platforms)