3D commodity display of HMS Core e-commerce solution

The traditional e-commerce commodity display adopts the form of combination of text and text, introduces the relevant parameters of the product, and matches with exquisite pictures to attract the attention of customers. However, due to color difference, size discrepancy and other reasons, graphic commodity display will make consumers have doubts about the inconsistency between pictures and real objects, and consumers need to spend a lot of energy reading and comparing before making a purchase decision.

With the continuous improvement of Internet e-commerce platform, short video has gradually become the mainstream form of commodity display. By taking short videos, the product can be displayed in multiple directions, so that consumers can understand the parameters and advantages of the product in a short time, which greatly improves the shopping speed of consumers. However, the disadvantage of short video display is that the display time is short and it is difficult to interact with consumers.

The development of 3D technology has given the e-commerce industry the latest way to display goods. 3D models can display goods in 360 ° all-round details, so that consumers can more intuitively understand goods and enhance their confidence in products. Consumers can also interact with goods across the screen. They can view goods 360 degrees and zoom in freely to view HD details through single finger dragging, so as to improve the rate of single machine.

However, at present, the 3D modeling technology with good effect has deterred the majority of developers because of its high cost.

1. High technical threshold: professionals plus professional equipment such as depth cameras.

2. High time cost: professionals manually complete the production, rendering and adjustment of the model. To complete the low-precision model of a simple object, the workload starts in hours, while the high-precision model takes longer.

3. High cost: the professional modeling cost of a single commodity is high, and the average price reaches thousands of yuan, while the complex model is more expensive.

3D modeling service of HMS Core For easy modeling. Users only need to use an ordinary RGB camera to automatically generate the 3D geometric model and texture of the object by photographing images from different angles. For example, in the scene of physical display in e-commerce, you can automatically generate the model of the goods you want to display through this ability for 3D display. Users can zoom in or out of the goods 360 ° at will, view the details of the goods, and provide users with a differentiated purchase experience.

The 3D object modeling capability is completed by the end cloud. The end side is responsible for collecting RGB images and capturing multiple images around the object, so as to obtain images from different angles of the object. After shooting, it is uploaded to the cloud to realize 3D object modeling. The process and key technologies of cloud modeling include target detection and segmentation, feature detection and matching, sparse point cloud computing, dense point cloud computing and texture reconstruction.

Effect display

Taking bread as an example, a realistic 3D model of bread can be obtained by taking multiple images around the bread. How is this function realized? The following are the detailed development steps.

Preparation before development

  1. Configure integrated SDK package

In the application of build Add SDK dependency of 3D modeling service in dependencies in gradle file

// 3D Modeling Kit SDK
implementation 'com.huawei.hms:modeling3d-object-reconstruct:1.0.0.300'
  1. Configure androidmanifest xml

Open androidmanifest. In the main folder XML file, which can configure the permissions of reading and writing mobile phone storage and camera according to the scene and use needs, and add it in front of < Application >

<!-- to sdcard Permission to write data in -->
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<!-- Permission to use the camera -->
<uses-permission android:name="android.permission.CAMERA" />

Development steps

  1. The ability to use cloud side services requires the API in "agconnect services. JSON"_ When the API is initialized by the key value_ Key or AccessToken to set application authentication information. AccessToken has higher priority. Both methods can be initialized and set once when the application is started, without multiple settings.

(1) Set AccessToken through setAccessToken method.

ReconstructApplication.getInstance().setAccessToken("your AccessToken");

(2) Set API through setApiKey method_ key. When you register an app on AppGallery Connect, your app will be assigned an api_key.

ReconstructApplication.getInstance().setApiKey("your api_key");
  1. Create a new 3D object modeling engine and initialize it, and create a new 3D object modeling configurator.
// Create a new 3D object modeling engine.
Modeling3dReconstructEngine modeling3dReconstructEngine = Modeling3dReconstructEngine.getInstance(context);
// Create a new 3D object modeling configurator.
Modeling3dReconstructSetting setting = new Modeling3dReconstructSetting.Factory()
    // Set the working mode to picture mode.
    .setReconstructMode(Modeling3dReconstructConstants.ReconstructMode.PICTURE)
    // Set the mapping mode to normal mode or PBR mode.
    .setTextureMode(Modeling3dReconstructConstants.TextureMode.PBR)
    .create();
        
  1. Create a new upload listener callback to process the upload results of the captured object pictures.
private Modeling3dReconstructUploadListener uploadListener = new Modeling3dReconstructUploadListener() {
    @Override
    public void onUploadProgress(String taskId, double progress, Object ext) {
         // Upload progress.
    }
    @Override
    public void onResult(String taskId, Modeling3dReconstructUploadResult result, Object ext) {
         // Upload successfully processed.
    }
    @Override
    public void onError(String taskId, int errorCode, String message) {
        // Upload failure processing.
    }
};
  1. Use the 3D object modeling configurator to initialize the task, and set the upload listener for the new 3D object modeling engine to upload the collected image data.
// Use 3D object modeling configurator initialization task (which needs to be invoked in middle note).
Modeling3dReconstructInitResult modeling3dReconstructInitResult = modeling3dReconstructEngine.initTask(setting);
String taskId = modeling3dReconstructInitResult.getTaskId();
// Set up the upload listener.
modeling3dReconstructEngine.setReconstructUploadListener(uploadListener);
// Call the upload interface of 3D modeling engine to upload the collected image data.
modeling3dReconstructEngine.uploadFile(taskId, filePath);
  1. Query 3D object modeling task status.
// To query the 3D object modeling task status, you need to initialize the task processing class.
Modeling3dReconstructTaskUtils modeling3dReconstructTaskUtils = Modeling3dReconstructTaskUtils.getInstance(context);
// Call the query interface to get the 3D object modeling task state (the interface needs to be invoked in the middle note).
Modeling3dReconstructQueryResult queryResult = modeling3dReconstructTaskUtils.queryTask(taskId);
// Get modeling task status.
int status = queryResult.getStatus();
  1. Create a listener callback and call the preview function to preview the 3D model
Modeling3dReconstructPreviewListener previewListener = new Modeling3dReconstructPreviewListener() {
    @Override
    public void onResult(String taskId, Object ext) {
        // 3D object modeling preview results.
    }
    @Override
    public void onError(String taskId, int errorCode, String message) {
        // Preview error callback function.
    }
};
// Preview the model configuration.
Modeling3dReconstructPreviewConfig config = new Modeling3dReconstructPreviewConfig.Factory().setTextureMode(Modeling3dReconstructConstants.TextureMode.PBR).create();
// Preview the model.
modeling3dReconstructEngine.previewModelWithConfig(taskId, context,config, previewListener);
  1. Create a new download listener callback to process the download results of 3D object modeling model files.
private Modeling3dReconstructDownloadListener modeling3dReconstructDownloadListener = new Modeling3dReconstructDownloadListener() {
    @Override
    public void onDownloadProgress(String taskId, double progress, Object ext) {
        // Download progress.
    }    
    @Override
    public void onResult(String taskId, Modeling3dReconstructDownloadResult result, Object ext) {
        // Download successfully processed.
    }
    @Override
    public void onError(String taskId, int errorCode, String message) {
        // Download failure processing.
    }
};
  1. Create a new download configuration item and transfer the new download listener to the new 3D object modeling engine to download the successfully reconstructed model file.
// Set download configuration items.
Modeling3dReconstructDownloadConfig downloadConfig = new Modeling3dReconstructDownloadConfig.Factory()
    // Configure OBJ or glTF format.
    .setModelFormat(Modeling3dReconstructConstants.ModelFormat.OBJ)
    // Configure normal or PBR mode.
    .setTextureMode(Modeling3dReconstructConstants.TextureMode.PBR)
    .create();
// Set up the download listener.
modeling3dReconstructEngine.setReconstructDownloadListener(modeling3dReconstructDownloadListener);
// Call the download interface of the 3D object modeling engine, pass in the task id, download address and download configuration item, and download the model file.
modeling3dReconstructEngine.downloadModelWithConfig(taskId, savePath, downloadConfig);

After completing the above steps, you can realize 3D modeling ability in e-commerce applications. You can quickly build 3D models for goods with mobile phones, and interested developers can experience them immediately!

In addition to 3D modeling and display of goods, HMS Core e-commerce solution also provides AR interactive shopping capability, which can realize the function of trying on goods and create a shopping experience integrating virtual and reality for consumers. Relevant contents will be explained in detail in the next issue. Please look forward to it.

Learn more > >

visit Official website of Huawei developer Alliance
obtain Development guidance document
Huawei mobile service open source warehouse address: GitHub,Gitee

Follow us and learn the latest technical information of HMS Core at the first time~

Tags: Java Android kotlin

Posted by furotte on Sun, 24 Apr 2022 04:53:56 +0300