1, Foreword

The real-time video stream displayed by the Hikvision sdk supports not only the handle method, but also the callback method. In addition to getting the video data, the callback also gets the audio data. Just call the audio device to play it by yourself. As for the callback of the Hikvision sdk, it took a while to complete. Maybe at first, there was no reference to the demo provided and no thorough search, Just looking at the sdk documents and tossing around, I didn't get it done for a week. After finding the right way, I found that it was almost lost. This reminds me of many things, including things in life, isn't it? When you're riveted, experimented with various methods, and are about to give up, in fact, you're one step away from success at this time. It's really one step away. It's also the same with dealing with many things in life. So many times, if you're in the right direction and stick to working hard, if you can't do it, try again and it's estimated to be ok.

Toss for a long time, summarize where the failure is, and call net_ DVR_ RealPlay_ It is also right to set the callback function in V40. The callback function is also included and playm4 is called_ Setdeccallbackmend is also right to set the decoding callback function (this place has been tossing around for a while, but I didn't expect to deal with it in the form of playing MP4). Finally, it was found that the problem lies in the decoded data. The data is also obtained. The default is yv12 data. If you need to convert it into image, you need to do a conversion. A lot of functions were found on the conversion website to test, and they all failed, Later, I found a yv12 to rgb888 format. It's finally ready. I'll go.

Haikang sdk callback process:

1. Call NET_DVR_RealPlay_V40 sets the callback processing function.
2. In the callback processing function RealDataCallBack, open, play and decode are processed successively.
3. Call PlayM4_GetPort gets the channel number not used by the playback library.
4. Call PlayM4_OpenStream opens the video stream.
5. Call PlayM4_SetDecCallBackMend sets the decoding callback function, which only decodes and does not display.
6. Call PlayM4_Play plays a video stream.
7. Call PlayM4_InputData loop decodes data.
8. The audio and video data are processed respectively in the decoding callback function DecCallBack.
9. Call the yv12ToRGB888 function encapsulated by yourself to convert the data into QImage.

Please note the following points about callback functions:

1. CALLBACK function must have keyword CALLBACK.
2. The callback function itself must be a global function or a static function, and cannot be defined as a member function of a specific class.
3. The callback function is not directly called and executed by the developer, but uses the system interface API function as the starting point.
4. Callback functions are usually passed as parameters to the system API, which calls them.
5. The callback function may be called by the system API once or repeatedly.

2, Functional features

1. Supports playing video streams and local MP4 files.
2. Handle and callback modes are supported.
3. Multi thread display image, not card main interface.
4. Automatic reconnection of webcam.
5. You can set the border size, that is, the offset and border color.
6. You can set whether to draw OSD labels, that is, label text or picture and label position.
7. Two OSD positions and styles can be set.
8. You can set whether to save to a file and the file name.
9. You can directly drag the file to the haikangwidget control to play.
10. Support h264/h265 video stream.
11. You can pause and resume playback.
12. Support the storage of single video files and regular storage of video files.
13. Customize the top suspension bar, send click signal notification, and set whether to enable it.
14. You can set the picture stretch filling or equal proportion filling.
15. It can be set that decoding is speed first, quality first and equalization processing.
16. You can take screenshots (original pictures) and screenshots (video form) of the video.
17. Video files are stored as MP4 files.
18. Support focus control and PTZ control.
19. Customizable functions.

3, Renderings

4, Related sites

1. Domestic sites: https://gitee.com/feiyangqingyun/QWidgetDemo
2. International sites: https://github.com/feiyangqingyun/QWidgetDemo
3. Personal homepage: https://blog.csdn.net/feiyangqingyun
4. Zhihu homepage: https://www.zhihu.com/people/feiyangqingyun/
5. Experience address: https://blog.csdn.net/feiyangqingyun/article/details/97565652

5, Core code

//yv12 to RGB888
static bool yv12ToRGB888(const unsigned char *yv12, unsigned char *rgb888, int width, int height)
{
    if ((width < 1) || (height < 1) || (yv12 == NULL) || (rgb888 == NULL)) {
        return false;
    }

    int len = width * height;
    unsigned char const *yData = yv12;
    unsigned char const *vData = &yData[len];
    unsigned char const *uData = &vData[len >> 2];

    int rgb[3];
    int yIdx, uIdx, vIdx, idx;

    for (int i = 0; i < height; ++i) {
        for (int j = 0; j < width; ++j) {
            yIdx = i * width + j;
            vIdx = (i / 2) * (width / 2) + (j / 2);
            uIdx = vIdx;

            rgb[0] = static_cast<int>(yData[yIdx] + 1.370705 * (vData[uIdx] - 128));
            rgb[1] = static_cast<int>(yData[yIdx] - 0.698001 * (uData[uIdx] - 128) - 0.703125 * (vData[vIdx] - 128));
            rgb[2] = static_cast<int>(yData[yIdx] + 1.732446 * (uData[vIdx] - 128));

            for (int k = 0; k < 3; ++k) {
                idx = (i * width + j) * 3 + k;
                if ((rgb[k] >= 0) && (rgb[k] <= 255)) {
                    rgb888[idx] = static_cast<unsigned char>(rgb[k]);
                } else {
                    rgb888[idx] = (rgb[k] < 0) ? (0) : (255);
                }
            }
        }
    }
    return true;
}

//The decoded callback video is YUV420P data (YV12), and the audio is PCM data
static void CALLBACK DecCallBack(qport nPort, char *pBuf, qport nSize, FRAME_INFO *pFrameInfo, quser luser, quser nReserved2)
{
    HaiKangThread *thread = (HaiKangThread *)luser;
    long frameType = pFrameInfo->nType;

    //Video data is T_YV12 audio data is T_AUDIO16
    if (frameType == T_YV12) {
        //qDebug() << TIMEMS << width << height << thread;
        int width = pFrameInfo->nWidth;
        int height = pFrameInfo->nHeight;
        QImage image(width, height, QImage::Format_RGB888);
        if (yv12ToRGB888((unsigned char *)pBuf, image.bits(), width, height)) {
            thread->setImage(image);
        }
    } else if (frameType == T_AUDIO16) {
        //qDebug() << TIMEMS << "T_AUDIO16" << thread;
    }
}

static void CALLBACK RealDataCallBack(LONG lRealHandle, DWORD dwDataType, BYTE *pBuffer, DWORD dwBufSize, void *dwUser)
{
    //Each class corresponds to its own port
    HaiKangThread *thread = (HaiKangThread *)dwUser;
    qport nPort = thread->port;

    DWORD dRet;
    switch (dwDataType) {
        case NET_DVR_SYSHEAD:
            //Get unused channel number
            if (!PlayM4_GetPort(&nPort)) {
                break;
            }

            if (dwBufSize > 0) {
                thread->port = nPort;
                if (!PlayM4_OpenStream(nPort, pBuffer, dwBufSize, 1024 * 1024)) {
                    dRet = PlayM4_GetLastError(nPort);
                    break;
                }

                //Set the decoding callback function to only decode and not display
                if (!PlayM4_SetDecCallBackMend(nPort, DecCallBack, (quser)dwUser)) {
                    dRet = PlayM4_GetLastError(nPort);
                    break;
                }

                //Turn on video decoding
                if (!PlayM4_Play(nPort, NULL)) {
                    dRet = PlayM4_GetLastError(nPort);
                    break;
                }

                //To enable audio decoding, the code stream needs to be a composite stream
                if (!PlayM4_PlaySound(nPort)) {
                    dRet = PlayM4_GetLastError(nPort);
                    break;
                }
            }
            break;

        case NET_DVR_STREAMDATA:
            //Decode data
            if (dwBufSize > 0 && nPort != -1) {
                BOOL inData = PlayM4_InputData(nPort, pBuffer, dwBufSize);
                while (!inData) {
                    sleep(10);
                    inData = PlayM4_InputData(nPort, pBuffer, dwBufSize);
                }
            }
            break;
    }
}