Second Development of Huawei IPC Camera Using C#

In the previous article, we implemented a camera login with SDK and a preview ( https://www.cnblogs.com/wdw984/p/13564195.html ), this time we achieve the function of calling the camera's own face capture through SDK.

Because it's short, the code goes straight up here.

First we define a security queue in MainWindow code to store captured face data, a timer to retrieve queue data, and a face capture callback event

        private static ConcurrentQueue<CaptureInfo> _concurrentQueue = new ConcurrentQueue<CaptureInfo>();
        private static HuaWeiSdkHelper.PfRealDataCallBack _fedRealPlayCallbackFaceCapture;
        private Timer _timer;

Initialize a timer in the form loading event to save captured data locally

        private void MainWindow_OnLoaded(object sender, RoutedEventArgs e)
        {
//Code omission in previous article _timer
= new Timer(300) { Enabled = false }; _timer.Elapsed += Timer_Elapsed; }

SDK defines Struct and Enum to be defined for face capture

namespace HuaWeiCamera.Struct
{
    /// <summary>
    /// Metadata Getting Related Parameters
    /// </summary>
    [StructLayout(LayoutKind.Sequential)]
    public struct PU_META_DATA
    {
        /// <summary>
        /// Data capacity
        /// </summary>
        public ushort usCapacity;
        /// <summary>
        /// Effective Number
        /// </summary>
        public ushort usValidNumber;
        /// <summary>
        /// Reference resources PU_UserData Definition
        /// </summary>
        public System.IntPtr pstMetaUserData;
    }
}
namespace HuaWeiCamera.Struct
{
    /// <summary>
    /// Metadata user data
    /// </summary>
    [System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Sequential, Pack = 1)]
    public struct PU_UserData
    {
        /// <summary>
        /// Metadata Type
        /// </summary>
        public LAYER_THREE_TYPE eType;
        /// <summary>
        /// User Metadata Details
        /// </summary>
        public PU_UserData_unMetadata Union1;
    }
}
namespace HuaWeiCamera.Struct
{
    /// <summary>
    /// User Metadata Details
    /// </summary>
    [System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Explicit)]
    public struct PU_UserData_unMetadata
    {
        [System.Runtime.InteropServices.FieldOffset(0)]
        public int bBoolValue;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public byte charValue;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public byte ucharValue;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public short shortValue;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public ushort ushortValue;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public int IntValue;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public uint uIntValue;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public long longlongValue;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public ulong uLonglongValue;
        /// <summary>
        /// Metadata Binary Color
        /// </summary>
        [System.Runtime.InteropServices.FieldOffset(0)]
        public ST_BINARY stBinay;
        /// <summary>
        /// Metadata Rectangle
        /// </summary>
        [System.Runtime.InteropServices.FieldOffset(0)]
        public META_RECT_S stRec;
        /// <summary>
        /// Metadata dash
        /// </summary>
        [System.Runtime.InteropServices.FieldOffset(0)]
        public META_POINT_S stPoint;
        /// <summary>
        /// Metadata dash
        /// </summary>
        [System.Runtime.InteropServices.FieldOffset(0)]
        public META_LINE_S stLine;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public IntPtr stPolyGon;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public IntPtr stColor;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public IntPtr stHumanAttr;
        /// <summary>
        /// Face information
        /// </summary>
        [System.Runtime.InteropServices.FieldOffset(0)]
        public META_FACE_INFO stFaceInfo;
        /// <summary>
        /// Face Attributes
        /// </summary>
        [System.Runtime.InteropServices.FieldOffset(0)]
        public META_FACE_ATTRIBUTES stFaceAttr;

        [System.Runtime.InteropServices.FieldOffset(0)]
        public IntPtr szUserData;
    }
}

LAYER_ THREE_ Enum for TYPE Reference https://support.huawei.com/enterprise/zh/doc/EDOC1100084903

Processing captured face data (stored as this map) in timer events

#region Processing face data
        private void Timer_Elapsed(object sender, ElapsedEventArgs e)
        {
            if (_concurrentQueue.Count == 0)
            {
                Console.WriteLine(@"No face pictures yet");
                return;
            }

            if (!_concurrentQueue.TryDequeue(out CaptureInfo face))
            {
                Console.WriteLine(@"Read Queue Error");
                return;
            }

            if (face._dataFacePic != null && face._dataFacePic.Length > 0)
            {
                Console.WriteLine(@"Face Storage");
                Task.Run(async () =>
                {
                    var saveFaceFile = Path.Combine($"{AppDomain.CurrentDomain.BaseDirectory}", "jpg", $"face_{DateTime.Now:yyyy-MM-dd-HH-mm-ss}.jpg");

                    await YuvHelper.Byte2Jpg(face._dataFacePic, saveFaceFile).ConfigureAwait(false);
                });
            }

            if (face._dataFacePanorama != null && face._dataFacePanorama.Length > 0)
            {
                Console.WriteLine(@"Panoramic Picture Storage");
                Task.Run(async () =>
                {
                    var savePanoramaFile = Path.Combine($"{AppDomain.CurrentDomain.BaseDirectory}", "jpg", $"Panorama_{DateTime.Now:yyyy-MM-dd-HH-mm-ss}.jpg");

                    await YuvHelper.Byte2Jpg(face._dataFacePanorama, savePanoramaFile).ConfigureAwait(false);
                });
            }
        }
        #endregion

Start face capture callback in face capture button event

#region Face Capture
        private void ButtonFace_OnClick(object sender, RoutedEventArgs e)
        {
            if (0 == _ulIdentifyId)
            {
                HuaWeiSdkHelper.InitAndLogin("192.168.2.250", 6060, "ApiAdmin", "HuaWei123", out _ulIdentifyId,
                    out string errMsg);

                if (0 == _ulIdentifyId)
                {
                    MessageBox.Show(errMsg);
                    return;
                }
            }

            var prpInfos = new PU_REAL_PLAY_INFO_S[1];
            var clientInfo = new PU_REAL_PLAY_INFO_S
            {
                ulChannelId = 101,
                hPlayWnd = IntPtr.Zero,
                enProtocolType = PU_PROTOCOL_TYPE.PU_PROTOCOL_TYPE_TCP,
                enStreamType = PU_STREAM_TYPE.PU_VIDEO_MAIN_STREAM,
                enVideoType = PU_VIDEO_TYPE.PU_VIDEO_TYPE_META,//You need to set the video type to metadata here
                enMediaCryptoType = PU_MEDIA_CRYPTO_TYPE.PU_MEDIA_CRYPTO_NONE,
                enMediaCallbackType = PU_MEDIA_CALLBACK_TYPE.PU_MEDIA_CALLBACK_TYPE_META_FRAME,//Callback method is smart metadata
                bKeepLive = true,
                szLocalIp = null,
                szReserved = new byte[32]
            };
            clientInfo.szReserved[22] = 1;//szReserved[22]Represents Intelligent Analysis Data Packaging Format 0:XML,1:metadata
            prpInfos[0] = clientInfo;
            var loginUserId = _ulIdentifyId;
            IntPtr pUsrData = (IntPtr)loginUserId;
            _fedRealPlayCallbackFaceCapture = FaceCaptureReaplayCallbackWithMetaFrame;
            var ulRealHandleCapture = HuaWeiSdkHelper.IVS_PU_RealPlay(_ulIdentifyId, prpInfos, _fedRealPlayCallbackFaceCapture, ref pUsrData);
            if (0 == ulRealHandleCapture)
            {
                MessageBox.Show(HuaWeiSdkHelper.GetLastErrorInfo());
                return;
            }

            _timer.Enabled = true;
        }

        #region Face Capture Data Callback

        private static void FaceCaptureReaplayCallbackWithMetaFrame(IntPtr szBuffer, int lSize, IntPtr pUsrData)
        {
            var ptrstMetaTargetData = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(PU_META_DATA)));
            try
            {
                var bRet = HuaWeiSdkHelper.IVS_User_GetMetaData(szBuffer, lSize, LAYER_TWO_TYPE.TARGET, ref ptrstMetaTargetData);
                if (false == bRet)
                {
                    return;
                }

                if ((IntPtr)0 == ptrstMetaTargetData)
                {
                    return;
                }

                //Marshals data from unmanaged memory blocks to newly allocated managed objects of the specified type
                var pstMetaData = (PU_META_DATA)Marshal.PtrToStructure(ptrstMetaTargetData, typeof(PU_META_DATA));
                //data processing
                if (0 == pstMetaData.usValidNumber)
                {
                    return;
                }

                PU_UserData pstMetaUserData = new PU_UserData();
                int nSizeofPuUserDataInC = Marshal.SizeOf(pstMetaUserData);
                byte[] dataFacePic = null;//Face pictures, if captured on a human face, will be converted into byte[] arrays to fill in
                byte[] dataFacePanorama = null;//Panoramic picture when face is detected
                var faceFeature = new META_FACE_ATTRIBUTES();//Additional data on people's faces
                bool hasFaceFeature = false;
                int target = 0;
                for (int uIndex = 0; uIndex < pstMetaData.usValidNumber; ++uIndex)
                {
                    IntPtr ptr2 = new IntPtr(pstMetaData.pstMetaUserData.ToInt32() + nSizeofPuUserDataInC * uIndex);
                    pstMetaUserData = (PU_UserData)Marshal.PtrToStructure(ptr2, typeof(PU_UserData));//Data to Metadata User Data Structure
                    switch (pstMetaUserData.eType)
                    {
                        case LAYER_THREE_TYPE.TARGET_TYPE:
                            target = pstMetaUserData.Union1.IntValue;
                            break;
                        case LAYER_THREE_TYPE.FACE_PIC://FaceSwapper
                            dataFacePic = new byte[pstMetaUserData.Union1.stBinay.ulBinaryLenth];
                            //Use Address data To get the data in the required memory block
                            Marshal.Copy(pstMetaUserData.Union1.stBinay.pBinaryData, dataFacePic, 0, (int)pstMetaUserData.Union1.stBinay.ulBinaryLenth);
                            break;
                        case LAYER_THREE_TYPE.FACE_PANORAMA://Face Panorama
                            dataFacePanorama = new byte[pstMetaUserData.Union1.stBinay.ulBinaryLenth];
                            //Use Address data To get the data in the required memory block
                            Marshal.Copy(pstMetaUserData.Union1.stBinay.pBinaryData, dataFacePanorama, 0, (int)pstMetaUserData.Union1.stBinay.ulBinaryLenth);
                            break;
                        case LAYER_THREE_TYPE.FACE_FEATURE://Face Attributes
                            hasFaceFeature = true;
                            faceFeature = pstMetaUserData.Union1.stFaceAttr;
                            break;
                        default:
                            break;
                    }
                }
                if ((int)Target.FaceCapture == target)
                {
                    CaptureInfo info =
                        new CaptureInfo
                        {
                            _dataFacePanorama = dataFacePanorama,
                            _dataFacePic = dataFacePic,
                            _faceFeature = faceFeature,
                            _hasFaceFeature = hasFaceFeature
                        };
                    _concurrentQueue.Enqueue(info);//Join the queue to be processed
                }
                HuaWeiSdkHelper.IVS_User_FreeMetaData(out ptrstMetaTargetData);//Release data footprint
            }
            catch (Exception e)
            {
                Console.WriteLine(e);
                throw;
            }
            finally
            {
                Marshal.FreeHGlobal(ptrstMetaTargetData);//Release memory
            }
        }

        #endregion
        #endregion

Release resources when the program exits

        private void MainWindow_OnClosed(object sender, EventArgs e)
        {
            _isExit = true;
            if (_timer.Enabled) _timer.Enabled = false;
            if (_ulRealHandleId > 0)
            {
                HuaWeiSdkHelper.IVS_PU_StopRealPlay(_ulIdentifyId, _ulRealHandleId);
            }
            if (_ulIdentifyId > 0)
            {
                HuaWeiSdkHelper.IVS_PU_Logout(_ulIdentifyId);
            }
            HuaWeiSdkHelper.IVS_PU_Cleanup();
            VideoFileStream.Close();
        }

After SDK has successfully registered the face capture, the face recognition algorithm brought by the camera itself will call back the data to the registration event. In the registration event, the data will be taken out according to the memory address of the face data given in the callback, instantiated into C#data structure, converted the picture into byte[] and written into a picture when processing the queue regularly. That is to complete the camera face recognition capture (some cameras with face matching algorithms can directly conduct face matching).

Tags: WPF

Posted by TMX on Thu, 19 May 2022 23:16:57 +0300