With less than 50 lines of code, the Tensorflow model can be deployed in a production environment

Serverless TensorFlow function in public cloud

For software developers and students, AI is paid.

In 2021, the average annual salary of developers of Tensorflow, the most popular AI framework, was US $148508, which was converted into nearly one million yuan.

Now, even for entry-level programming, developers must have AI skills. In fact, it's easy to find a tutorial to train your Tensorflow model for tasks such as image recognition and natural language processing. You only need some basic Python knowledge to train the model, and then run the model for demonstration.

Just knowing how to use simple Python to train the model won't bring much benefit.

Compared with the training model, it is much more difficult to provide the trained model as a reliable Web service to others. For developers, deploying TensorFlow models in a production environment presents significant challenges. The company pays high prices for people who can overcome these challenges.

  • Python language and framework are inefficient in preparing and processing the input and output data of the model. It is estimated that 90% of AI computing workload is used for data preparation. Python language is too cumbersome and too slow.
  • It is difficult to scale services on demand. Due to the computational requirements of AI reasoning, even if there are only a few requests, the server computer may be temporarily suspended. Expanding the number of servers up and down on demand is critical.

However, there are simple ways to solve this problem. With the help of WebAssembly virtual machine SSVM and Tencent cloud serverless , you can deploy the Tensorflow model as a service to the production environment using simple Rust code within 50 lines.

Simple means that the Rust function syntax used is very simple and does not use life ownership.

adopt This template function , you can deploy TensorFlow model for production environment on Tencent cloud for free!

If you follow the steps in this article and deploy the Tensorflow serverless function on Tencent cloud, you will get a series of peripheral products. For details, please click here . Please fill out this form , we will send you a gift as soon as possible.

How does it work

To meet the challenges mentioned above, the technologies selected are as follows:

  • Rust programming language is very fast and memory safe. It is an ideal choice for high-performance data processing and model execution logic.

  • WebAssembly acts as a bridge between modern Rust programs, the latest Tensorflow library and the enhanced public cloud operating system. WebAssembly is a compatibility sandbox that is critical to the developer's experience.

  • Tencent cloud serverless provides an extensible infrastructure to run Rust and WebAssembly functions for TensorFlow reasoning. Combined with the serverless framework, Tencent cloud serverless provides an excellent developer experience for deploying simple software.

Now, let's see how it works. First, from GitHub fork Template project And complete all preparations. You can use GitHub Codespaces IDE or Docker image, or install it on your own computer Rust, ssvmup, serverless framework .

Here is image recognition AI as a service. It recognizes the food in the image through the trained TensorFlow model. With less than 50 lines of simple Rust code, it can be deployed on Tencent cloud serverless. Tencent cloud serverless can be expanded on demand and charged according to the actual use.

View deployed web pages


The Rust code is used to load the input image, load and execute the Tensorflow model to identify the content on the image. The model here is in Tensorflow Lite format, which can recognize the food on the input image.

    // Load the trained TensorFlow lite model.
    let model_data: &[u8] = include_bytes!("lite-model_aiy_vision_classifier_food_V1_1.tflite");
    // The format of the uploaded image is base64 encoding and encapsulated in JSON objects through Tencent cloud API gateway.
    let mut buffer = String::new();
    io::stdin().read_to_string(&mut buffer).expect("Error reading from STDIN");
    let obj: FaasInput = serde_json::from_str(&buffer).unwrap();
    let img_buf = base64::decode_config(&(obj.body), base64::STANDARD).unwrap();
    // Load the uploaded image and adjust it to 192x192, which is the required size of the MobileNet model.
    let flat_img = ssvm_TensorFlow_interface::load_jpg_image_to_rgb8(&img_buf, 192, 192);
    // The image is used as the input tensor to run the model, and the output tensor of the model is obtained.
    let mut session = ssvm_TensorFlow_interface::Session::new(&model_data, ssvm_TensorFlow_interface::ModelType::TensorFlowLite);
    session.add_input("input", &flat_img, &[1, 192, 192, 3])
    let res_vec: Vec<u8> = session.get_output("MobilenetV1/Predictions/Softmax");

res_ The VEC vector contains a list of probabilities for each object in the image (for example, the probability of a cake in the image is 0.8). The following Rust code reads the labels of these objects and prints the object labels with the highest probability from the Tensorflow model output.

let labels = include_str!("aiy_food_V1_labelmap.txt");

let mut i = 0;
let mut max_index: i32 = -1;
let mut max_value: u8 = 0;
while i < res_vec.len() {
    let cur = res_vec[i];
    if cur > max_value {
        max_value = cur;
        max_index = i as i32;
    i += 1;

let mut label_lines = labels.lines();
for _i in 0..max_index {

let class_name = label_lines.next().unwrap().to_string();
if max_value > 50 && max_index != 0 {
    println!("The image {} contains a <a href='https://www.google.com/search?q={}'>{}</a>", confidence.to_string(), class_name, class_name);
} else {
    println!("No food item is detected");

Deploy your own TensorFlow AI inference function

On the back end, the Rust code is compiled into WebAssembly bytecode and executed in the SSVM WebAssembly runtime. SSVM has been pre configured to access the high-performance tensorflow native library in multiple operating system environments, including the serverless container of Tencent cloud. In turn, Tencent cloud serverless provides a simple solution to extend tensorflow inference function.

Open a Terminal window in the Codespaces IDE, and then run the following command from Docker or the command line to build the cloud function.

$ ssvmup build —enable-aot

In the Terminal window, run the following command line to deploy TensorFlow cloud function to Tencent cloud.

$ cp pkg/scf.so scf/

$ sls deploy
... ...
website: [https://sls-website-ap-hongkong-kfdilz-1302315972.cos-website.ap-hongkong.myqcloud.com](https://sls-website-ap-hongkong-kfdilz-1302315972.cos-website.ap-hongkong.myqcloud.com/)

Open the URL you deployed in the browser to see what you ate.


In this article, we discussed how to create simple, safe and high-performance Rust functions to run Tensorflow model, and how to deploy these functions to the public cloud as scalable and on-demand AI services.

Now, you can deploy Tensorflow serverless function on Tencent cloud for free, and Get beautiful surroundings.

Tags: AI TensorFlow cloud computing Rust serverless

Posted by nick1 on Sat, 16 Apr 2022 01:46:55 +0300