Qt realizes sdp packaging, sdp strategy introduction

sdp introduction

SDP: session des description protocol
Session description protocol (SDP) provides multimedia session description for session notification, session invitation and other forms of multimedia session initialization.
The SDP session description consists of multiple lines =. Where is a character. Is a string whose format depends on.
SDP session description consists of a session_level description and multiple media_level descriptions. The scope of session_level is the whole session. Its position is from the 'v=' line to the first media description. media_level description refers to the description of a single media stream (for example, the vlc sdp file that transmits a single audio or video has only a few words, starting from m=, which is actually a media machine description), and its position is from the 'm=' line to the next media description.

v=0//SDP version number, required, example: "v=0\n"
o=<username> <sessionid> <version> <network type> <address type> <address>//Initiator description of the session, required, example: "o=metaltc 100002 in IP4 0.0.0.0\n"
s=<sessionname>//Session name, required, example: "s=metaltcsession\n"
c=<networktype> <address type> <connection address>//Media connection information, optional, example: "c=IN IP4 0.0.0.0\n"
b=<modifier>:<bandwidth-value>//Bandwidth, optional, example: b=X-YZ:128
t=<start time>  <stop time>//Describes the start time and end time of the session. Required. Example: "t=0 0\n"
m=<media><port> <transport> <fmt list>//A session description includes several media descriptions, which are required. Examples: "m=audio 9 UDP/TLS/RTP/SAVPF 111\n"; "M=video 9 udp/tls/rtp/savpf 125 114\n" \125 and 114 are codecs
a=rtpmap:<payload type><encoding name>/<clock rate>[/<encodingparameters>]//Encoding parameter, optional, example: "a=rtpmap:111 opus/48000/2\n"
a=<TYPE>or a=<TYPE>:<VALUES>//SuggestedAttributes, suggested attributes, optional
a=framerate:<Frame rate>//Unit: frame / second, optional
a=lang:<Language markers>//Default language of session description or language of media description, optional

Audio and video call sdp strategy

Audio and video processes mainly include: sdp interaction, dtls handshake, udp receiving pronunciation video data, session signaling interaction

sdp strategy
1. sdp session description protocol, sdp is not allowed to be lost, so it is recommended to forward it through the signaling server, usually http or websockt.
2. If there is no signaling server forwarding, you can also use frpc(ngrok) for intranet penetration.
3. sdp carries a lot of information. Whether to carry local public ip and port depends on the policy.

udp policy
1. dtls and audio and video data are sent through udp. p2p communication requires udp penetration, which can be achieved by using the open source coturn server stun protocol. After successful penetration, the caller and the callee obtain their own public ip and port respectively, and send them to each other through sdp.
2. If coturn penetration fails, the turn service of coturn can be used for relay forwarding. At this time, the sent sdp does not include the ip and port of the local end.
3. A demo strategy of LAN is relatively easy to implement. The caller knows the called http service ip and port, the caller generates a random udp port, and sends sdp to the called http server as an http client. This sdp can not carry the udp port number; The called also generates a random udp port, writes the sdp and replies to the caller. At this time, the caller resolves to the called udp port, the caller sends audio and video data to the called udp as a udp client, and the called receives the udp package of the caller as a udp server, parses the udp port of the caller, and sends udp data to the caller.

Qt implements sdp packaging

qt shell, the bottom layer is realized by C/C++, and the source code is to be packaged.

#include "sdpmainwindow.h"
#include "ui_sdpmainwindow.h"

sdpMainWindow::sdpMainWindow(QWidget *parent)
    : QMainWindow(parent)
    , ui(new Ui::sdpMainWindow)
{
    ui->setupUi(this);

    session=(chwRtcSession*)calloc(sizeof(chwRtcSession),1);
    session->context.streamConfig=(chwStreamConfig*)calloc(sizeof(chwStreamConfig),1);
    session->context.avinfo=(chwAVInfo*)calloc(sizeof(chwAVInfo),1);
    session->context.cer=(chwCertificate*)calloc(sizeof(chwCertificate),1);

    //chwStreamConfig
    strcpy(session->context.streamConfig->app,"live");
    session->context.streamConfig->streamOptType=chw_Stream_Both;
    strcpy(session->context.streamConfig->remoteIp,"192.168.3.36");//Opposite IP
    session->context.streamConfig->remotePort=9092;//Opposite port
    strcpy(session->context.streamConfig->stream,"livestream");
    session->context.streamConfig->uid=0;
    session->context.streamConfig->isServer=1;
    session->context.streamConfig->localPort=9091;//Local udp port
    session->context.streamConfig->recvCallback.context=this;

    //avinfo
    initVideo(&session->context.avinfo->video);
    initAudio(&session->context.avinfo->audio);
    initSys(&session->context.avinfo->sys);
    initEnc(&session->context.avinfo->enc);
    initRtc(&session->context.avinfo->rtc);

    session->isServer = 1;
    chw_create_rtcContext(&session->context);

    char* localSdp;
    createOffer(session,&localSdp);

    //Wrap print
    std::string prtlocalSdp = localSdp;
    std::string::size_type pos = 0;
    while ((pos = prtlocalSdp.find("\\r\\n", pos)) != std::string::npos)
    {
        prtlocalSdp.replace(pos, 4, "\r\n");
    }
    printf("localsdp=%s\n",prtlocalSdp.c_str());
}

sdpMainWindow::~sdpMainWindow()
{
    delete ui;
}

int32_t sdpMainWindow::createOffer(chwRtcSession *session, char **psdp)
{
    int32_t localport=session->context.streamConfig->localPort;
    chwStreamOptType role=session->context.streamConfig->streamOptType;
    return chw_sdp_genLocalSdp(session,localport, psdp,role);
}

void sdpMainWindow::initVideo(chwVideoInfo* video){
    memset(video,0,sizeof(chwVideoInfo));
    //Local resolution according to the configuration settings, receive the peer video packet analysis header to obtain the peer resolution
    video->width =  640;
    video->height =  480;
    video->outWidth =  640;
    video->outHeight = 480;
    video->rate = 2048;
    video->frame =  25;
    video->rotate = 0;
    video->bitDepth = 8;

    video->videoCacheNum =50;
    video->evideoCacheNum = 50;
    video->videoPlayCacheNum = 5;

    video->videoCaptureFormat = (chwYuvType)("video", "videoCaptureFormat", chwYuy2);
    video->videoEncoderFormat = (chwYuvType)("video", "videoEncoderFormat", chwI420);
    video->videoDecoderFormat = (chwYuvType)("video", "videoDecoderFormat", chwI420);

    video->videoEncoderType = ("video", "videoEncoderType", 0);
    video->videoDecoderType = ("video", "videoDecoderType", 0);
    video->videoEncHwType = ("video", "videoEncHwType", 0);
    video->videoDecHwType = ("video", "videoDecHwType", 0);
    video->vIndex = ("video", "vIndex", 0);
}
void sdpMainWindow::initAudio(chwAudioInfo* audio){
    memset(audio,0,sizeof(chwAudioInfo));
    audio->sample= 44100;
    audio->frameSize= 1024;
    audio->channel= 2;
    audio->bitrate = 128;
    audio->usingMono=0;
    audio->hasAec = 0;
    audio->echoPath = 10;
    audio->aecBufferFrames=0;
    audio->hasAudioHeader = 0;
    audio->audioEncoderType = 0;
    audio->audioDecoderType = 0;
    audio->audioPlayType = 0;
    audio->audioCacheNum = 100;
    audio->audioCacheSize = 100;
    audio->audioPlayCacheNum = 10;

    audio->aIndex = 0;
    audio->aSubIndex = 0;
    //aSubIndex
    if(audio->usingMono){
        if(audio->audioEncoderType<2) audio->audioEncoderType=2;
        if(audio->audioDecoderType<2)audio->audioDecoderType=2;
        audio->channel=1;
        audio->sample=16000;
        audio->frameSize=320;
    }
}
void sdpMainWindow::initSys(chwSysInfo *sys){
    memset(sys,0,sizeof(chwSysInfo));

    sys->transType =  0;
    sys->mediaServer = 9;
    sys->isMultCamera = 0;
    sys->usingDataServer = 0;

    strcpy(sys->localIp , "192.168.3.33");//Local ip
    sys->rtmpPort =  1935;
    sys->srtPort = 8080;
    sys->rtcPort = 1985;
    sys->rtcLocalPort =  16000;
    sys->httpPort =  8080;
    sys->dataPort =  9999;
    sys->hasLogFile= 0;
    sys->logLevel = 1;
    sys->logLevel = 1;
    sys->cameraCount = 3;
    strcpy( sys->cameraIndexs, "1");
    strcpy( sys->rtmpServerIP, "127.0.0.1");
    strcpy(sys->srtServerIP, "127.0.0.1");
    strcpy(sys->rtcServerIP, "127.0.0.1");
    strcpy( sys->httpServerIP, "127.0.0.1");
    strcpy( sys->dataServerIP, "127.0.0.1");
}

void sdpMainWindow::initEnc(chwVideoEncInfo *enc){
    memset(enc,0,sizeof(chwVideoEncInfo));
    enc->preset = 3;
    enc->level_idc =  31;
    enc->profile =  0;
    enc->keyint_max = 30;
    enc->enc_threads =  4;
    enc->createMeta = true;
    enc->gop = 30;

}

void sdpMainWindow::initRtc(chwRtcInfo *rtc){
    memset(rtc,0,sizeof(chwRtcInfo));
    rtc->sessionTimeout =  30*1000000;
    rtc->usingDatachannel =  1;
    rtc->hasIceServer = 0;
    rtc->iceStunPort = 3478;
    rtc->audioQueueCount =  5;
    rtc->videoQueueCount =  5;

    strcpy( rtc->iceServerIP, "127.0.0.1");

}

Print sdp

v=0
o=MetaRtc 140726066547688 2 IN IP4 0.0.0.0
s=MetaRtcSession
t=0 0
a=ice-lite
a=group:BUNDLE 0 1
a=msid-semantic: WMS live/livestream
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 0.0.0.0
a=ice-ufrag:3829
a=ice-pwd:41q6q68f8n58mleb76380ksr11245p4q
a=fingerprint:sha-256 FC:56:1B:EA:E6:F9:23:CD:34:22:B1:B9:8A:B2:4A:8C:C6:34:28:54:46:FE:CF:AD:05:7A:9F:6C:72:2A:EC:B9
a=setup:passive
a=mid:0
a=extmap:0 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:111 opus/44100/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=ssrc:192686677 cname:6a621o75scxudrl4
a=ssrc:192686677 msid:- x827358k-21pr-q9x2-335a-8m50f902017e
a=ssrc:192686677 mslabel:-
a=ssrc:192686677 label:x827358k-21pr-q9x2-335a-8m50f902017e
a=candidate:0 1 udp 2130706431 192.168.3.33 9091 typ host generation 0
m=video 9 UDP/TLS/RTP/SAVPF 125 114
c=IN IP4 0.0.0.0
a=ice-ufrag:3829
a=ice-pwd:41q6q68f8n58mleb76380ksr11245p4q
a=fingerprint:sha-256 FC:56:1B:EA:E6:F9:23:CD:34:22:B1:B9:8A:B2:4A:8C:C6:34:28:54:46:FE:CF:AD:05:7A:9F:6C:72:2A:EC:B9
a=setup:passive
a=mid:1
a=extmap:0 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:125 H264/90000
a=rtcp-fb:125 transport-cc
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:114 red/90000
a=ssrc:192686678 cname:6a621o75scxudrl4
a=ssrc:192686678 msid:- k913003w-1847-r191-141t-7x9w6j460504
a=ssrc:192686678 mslabel:-
a=ssrc:192686678 label:k913003w-1847-r191-141t-7x9w6j460504
a=candidate:0 1 udp 2130706431 192.168.3.33 9091 typ host generation 0

Tags: Qt rtc p2p

Posted by rkumar on Mon, 01 Aug 2022 21:33:17 +0300