Container cornerstone runc source code analysis -- 1 Construction of source code debugging environment

Construction of source code debugging environment

  1. Environment construction
  2. Detailed explanation of createcontainer process

preface

In the era of cloud origin, it goes without saying how important containers are. Runc is a CLI tool used to generate and run containers on Linux according to OCI specification. It is one of the underlying technologies of containers. This series deeply analyzes the runc project from the source code level, taking you to unlock the world behind the mysterious veil of the container.

Before analyzing runc, we need to focus on the oci standard.

Open Container Initiative (OCI) is a project under the Linux foundation responsible for the development of open standards for operating system layer Virtualization (container). Currently, two standards are mainly developed:

  • Runtime specification (runtime spec)
  • Image specification (image spec)

The runtime Standard outlines how to run a file system bundle unpacked from disk (similar to the software bundle distributed on macOS). An OCI image will be downloaded from the upper OCI implementation, and then unpacked into an OCI runtime file system bundle. At this point, the OCI runtime file system package will be run by an OCI runtime.

The OCI standard defines the container runtime and image specifications, so that different container implementations (LXC, Docker, Kata, rkt) run with the same standard, so that developers can build, package and deploy containers and run on solutions from different manufacturers.

Thus, oci standard is the soul of the whole container technology and even the cloud native system built from it. This standard must arouse our enough attention.

Basic environment

  • runc: release-1.1
  • golang: v.1.18
  • os: ubuntu20.04
  • ide: vscode + golang plug-in
  • image: busybox

1, Basic environment construction

  1. golang + vscode
    Environment construction will not be repeated

  2. runc debugging
    vscode debug launch json:

    {
    	// Use IntelliSense to learn about possible attributes.
    	// Hover to view descriptions of existing attributes.
    	// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    	"version": "0.2.0",
    	"configurations": [
    		{
    			"name": "runc",
    			"type": "go",
    			"request": "launch",
    			"mode": "auto",
    			"program": "${workspaceFolder}",
    			"args": [
    				"run",
    				"--bundle",
    				"/root/code/go/kubernetes/runc/bundle",
    				"testcontainer",
    			]
    		}
    	]
    }
    

    Note that different args are required for different commands, which is troublesome, so I directly use the command go run + log to read and debug.
    go run: go run your_runc_path/runc --debug create ...

2, File system package bundle preparation

What is a bundle? Why do we need bundles? Please refer to the documentation: bundle

  1. rootfs preparation
    runC is the runtime of the running container. It is responsible for running the container with standard files and other resources, but it does not include the image management function like docker. Therefore, to run the container with runC, we must first prepare the container's file system and the container's configuration file config JSON (defined by the oci standard).

    $ cd runc
    $ docker pull busybox
    $ docker export $(docker create busybox) > bundle/busybox.tar
    $ tar -C bundle/rootfs -xf bundle/busybox.tar
    

    The above command will create a rootfs. For rootfs, please refer to What is the root file system The explanation of the article in Chinese and English.

  2. config.json preparation

    $ cd runc
    $ go run . --debug spec --bundle bundle # Generate config JSON, - rootless will generate a container configuration in normal user mode
    

    config.json is the configuration file of the container. It is one of the two most important basic files of the container. The other is state JSON. Later articles in the series will introduce this state in detail JSON file. The following describes config json:

    {
    	"ociVersion": "1.0.2-dev",   // oci Standard Version
    	"process": {                 // For the definition of container process, please refer to the document in the official runtime spec project for details
    		"terminal": true,        // When debugging, change it to false. This parameter is very important and will be explained in detail in a series of articles
    		"user": {                // Container user
    			"uid": 0,
    			"gid": 0
    		},
    		"args": [                // For debugging convenience, the sh parameter here is changed to "sleep" and "30"
    			"sh"
    		],
    		"env": [                 // Container environment variable
    			"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    			"TERM=xterm"
    		],
    		"cwd": "/",              // Container initialization directory
    		"capabilities": {        
    			"bounding": [
    				"CAP_AUDIT_WRITE",
    				"CAP_KILL",
    				"CAP_NET_BIND_SERVICE"
    			],
    			"effective": [
    				"CAP_AUDIT_WRITE",
    				"CAP_KILL",
    				"CAP_NET_BIND_SERVICE"
    			],
    			"inheritable": [
    				"CAP_AUDIT_WRITE",
    				"CAP_KILL",
    				"CAP_NET_BIND_SERVICE"
    			],
    			"permitted": [
    				"CAP_AUDIT_WRITE",
    				"CAP_KILL",
    				"CAP_NET_BIND_SERVICE"
    			],
    			"ambient": [
    				"CAP_AUDIT_WRITE",
    				"CAP_KILL",
    				"CAP_NET_BIND_SERVICE"
    			]
    		},
    		"rlimits": [
    			{
    				"type": "RLIMIT_NOFILE",
    				"hard": 1024,
    				"soft": 1024
    			}
    		],
    		"noNewPrivileges": true
    	},
    	"root": {
    		"path": "rootfs",
    		"readonly": true               // For debugging convenience, it can be changed to false here
    	},
    	"hostname": "runc",
    	"mounts": [
    		{
    			"destination": "/proc",
    			"type": "proc",
    			"source": "proc"
    		},
    		{
    			"destination": "/dev",
    			"type": "tmpfs",
    			"source": "tmpfs",
    			"options": [
    				"nosuid",
    				"strictatime",
    				"mode=755",
    				"size=65536k"
    			]
    		},
    		{
    			"destination": "/dev/pts",
    			"type": "devpts",
    			"source": "devpts",
    			"options": [
    				"nosuid",
    				"noexec",
    				"newinstance",
    				"ptmxmode=0666",
    				"mode=0620",
    				"gid=5"
    			]
    		},
    		{
    			"destination": "/dev/shm",
    			"type": "tmpfs",
    			"source": "shm",
    			"options": [
    				"nosuid",
    				"noexec",
    				"nodev",
    				"mode=1777",
    				"size=65536k"
    			]
    		},
    		{
    			"destination": "/dev/mqueue",
    			"type": "mqueue",
    			"source": "mqueue",
    			"options": [
    				"nosuid",
    				"noexec",
    				"nodev"
    			]
    		},
    		{
    			"destination": "/sys",
    			"type": "sysfs",
    			"source": "sysfs",
    			"options": [
    				"nosuid",
    				"noexec",
    				"nodev",
    				"ro"
    			]
    		},
    		{
    			"destination": "/sys/fs/cgroup",
    			"type": "cgroup",
    			"source": "cgroup",
    			"options": [
    				"nosuid",
    				"noexec",
    				"nodev",
    				"relatime",
    				"ro"
    			]
    		}
    	],
    	"linux": {
    		"resources": {               // Generate config JSON command has no -- rootless parameter
    			"devices": [
    				{
    					"allow": false,
    					"access": "rwm"
    				}
    			]
    		},
    		"uidMappings": [            // rootless container parameters
    			{
    				"containerID": 0,
    				"hostID": 0,
    				"size": 1
    			}
    		],
    		"gidMappings": [            // rootless container parameters
    			{
    				"containerID": 0,
    				"hostID": 0,
    				"size": 1
    			}
    		],
    		"namespaces": [
    			{
    				"type": "pid"
    			},
    			{
    				"type": "network"
    			},
    			{
    				"type": "ipc"
    			},
    			{
    				"type": "uts"
    			},
    			{
    				"type": "mount"
    			}
    		],
    		"maskedPaths": [
    			"/proc/acpi",
    			"/proc/asound",
    			"/proc/kcore",
    			"/proc/keys",
    			"/proc/latency_stats",
    			"/proc/timer_list",
    			"/proc/timer_stats",
    			"/proc/sched_debug",
    			"/sys/firmware",
    			"/proc/scsi"
    		],
    		"readonlyPaths": [
    			"/proc/bus",
    			"/proc/fs",
    			"/proc/irq",
    			"/proc/sys",
    			"/proc/sysrq-trigger"
    		]
    	}
    }
    

Now we have rootfs and config The bundle composed of JSON will become the key to analyze the source code later. Next, we can start to analyze the runc source code

summary

  1. Reference articles
    What is RunC?
    Container safety collection - a preliminary study of roof container
    What is the root file system
    On root file system in linux (principle and introduction of rootfs)
  2. The definition of oci standard (runtime spec and image spec) is very important and is the soul of the whole container. So be sure to read it runtime-spec Document of the project

Tags: Kubernetes Container

Posted by transformationstarts on Sun, 22 May 2022 04:23:10 +0300