Basic concepts of embedded Linux partition

background

Partition is a means of dividing the disk into regions. By saving the starting address, size and other information to the partition table, the disk can be divided into several regions for storing different contents.

The use of partitions can not only make the system file entries clear, compatible with multiple file systems and mount them in different partitions, but also improve the stability of the system through the management of Mount permissions, which is a common concept in other systems.

Through personal understanding and use of partitions in Linux, this paper introduces some common concepts, basic information query methods and image making methods for reference only.

Basic concepts

1. Partition table

It contains basic information such as starting address and size, which is generally stored in the memory header. The purpose of adjusting the partition can be achieved by adjusting the partition table. The simple understanding is to divide a whole storage area into several equal parts. The operating system finds the partition location by looking up the start address and size of each partition.

The partition table is generally written into the storage area by brushing the machine. After adjusting the partition table, the corresponding image should be written again; For the commonly used imx6 platform, a shell script is used to record partitions, and the partition table is updated through mfgtool. Here is an excerpt of the partition update part of the mfgtool script:

<!-- create partition -->
<CMD state="Updater" type="push" body="send" file="mksdcard-android.sh.tar">Sending partition shell</CMD>
<CMD state="Updater" type="push" body="$ tar xf $FILE "> Partitioning...</CMD>
<CMD state="Updater" type="push" body="$ sh mksdcard-android.sh /dev/mmcblk%mmc%"> Partitioning...</CMD>

The send instruction copies the compressed package of the partition table to the memory, then decompresses it through the instruction, and finally executes the script content through sh. the script has and only has one parameter, which is the local storage device / dev/mmcblk0, which is a 32G emmc; After the partition is completed, each partition will be further formatted and the corresponding contents of the partition will be brushed in. It is not completely pasted.

After brushing, the system starts up. You can view the total size of the memory and the size of each partition through the command cat /proc/partitions.

The presentation of partition table varies with different platforms. The partition table of Qualcomm platform is managed through xml file. A small part of the excerpt here is for reference only:

<partition label="aboot" size_in_kb="1024" type="400FFDCD-22E0-47E7-9A23-F16ED9382388" bootable="false" readonly="true" filename="emmc_appsboot.mbn"/>
<partition label="abootbak" size_in_kb="1024" type="EBD0A0A2-B9E5-4433-87C0-68B6B72699C7" bootable="false" readonly="true" filename="emmc_appsboot.mbn"/>
<partition label="boot" size_in_kb="65536" type="20117F86-E985-4357-B9EE-374BC1D8487D" bootable="false" readonly="true" filename="boot.img"/>
<partition label="recovery" size_in_kb="65536" type="9D72D4E4-9958-42DA-AC26-BEA7A90B0434" bootable="false" readonly="true" filename="recovery.img"/>
<partition label="devinfo" size_in_kb="1024" type="1B81E7E6-F50D-419B-A739-2AEEF8DA3335" bootable="false" readonly="true" filename=""  sparse="true"/>
<partition label="system" size_in_kb="2097152" type="97D7B011-54DA-4835-B3C4-917AD6E73D74" bootable="false" readonly="true" filename="system.img" sparse="true"/>
<partition label="vendor" size_in_kb="1048576" type="97D7B011-54DA-4835-B3C4-917AD6E73D74" bootable="false" readonly="true" filename="vendor.img" sparse="true"/>

This is a partition table unique to Qualcomm platform. The meaning of the parameters is clear at a glance. Their order also determines the index value of the partition. This is a process similar to that from machine language to C language. The purpose is to make it more convenient for users to configure the partition; So you can guess that the editing of xml is only the first step. Qualcomm has a supporting python script to parse the xml. Through the parsing and transformation of the script, the partition table is finally presented in the form of binary. Update the partition table through QFIL. After the system is started, check whether the partition adjustment is successful in the same way.

Compared with the imx platform, Qualcomm's partition is more advanced, which can not only support more parameters. This table also indicates the image file corresponding to each partition, read-only permission, whether it is in spark format, etc. Therefore, this table not only contains partitions, but also serves as a brush guide.

2.Linux partition interface

After the partition table is written into the machine, the system will enumerate the block device nodes according to the partition information after startup. Because these driver s are written by the system and have no debugging experience in relevant functions, I am not clear about the specific enumeration process. The enumerated device nodes are under / dev/block /, and the partitions of Qualcomm platform have their own labels. You can view the labels corresponding to the partitions through the by name folder, They are connected together through soft connection. Some excerpts here are for reference only.

lrwxrwxrwx 1 root root   20 1970-01-01 08:00 sbl1 -> /dev/block/mmcblk0p4
lrwxrwxrwx 1 root root   20 1970-01-01 08:00 sbl1bak -> /dev/block/mmcblk0p5
lrwxrwxrwx 1 root root   21 1970-01-01 08:00 sec -> /dev/block/mmcblk0p18
lrwxrwxrwx 1 root root   21 1970-01-01 08:00 splash -> /dev/block/mmcblk0p26
lrwxrwxrwx 1 root root   20 1970-01-01 08:00 ssd -> /dev/block/mmcblk0p3
lrwxrwxrwx 1 root root   21 1970-01-01 08:00 syscfg -> /dev/block/mmcblk0p35
lrwxrwxrwx 1 root root   21 1970-01-01 08:00 system -> /dev/block/mmcblk0p24
lrwxrwxrwx 1 root root   21 1970-01-01 08:00 tbox -> /dev/block/mmcblk0p50
lrwxrwxrwx 1 root root   20 1970-01-01 08:00 tz -> /dev/block/mmcblk0p8
lrwxrwxrwx 1 root root   20 1970-01-01 08:00 tzbak -> /dev/block/mmcblk0p9
lrwxrwxrwx 1 root root   21 1970-01-01 08:00 userdata -> /dev/block/mmcblk0p53
lrwxrwxrwx 1 root root   21 1970-01-01 08:00 vendor -> /dev/block/mmcblk0p25
root: /# /dev/block/platform/soc/7824900.sdhci/by-name #

With the device node, the application can access the content in the corresponding partition through the interface.

We can also view the corresponding information of different partitions under / sys/class/block /.

3.image production

After the partition work is completed, it is only after the partition of the memory area is completed. It is like drawing a parking space and parking a car. It is valuable, so here is the content written in the partition.

Partition content can be basically divided into two types: one is an image without a file system, and the other is an image with a file system; Whether there is a file system determines whether the partition needs to be mounted. Please introduce these two methods respectively;

a) An image without a file system can be understood as a huge binary, which stores data and code. Without a file system, it does not need to be mounted.

So, for example, the bootloader/boot image/recovery image we use, and a considerable part of the firmware do not have a file system. However, boot and recovery are relatively special. Both are in the form of kernel+dtb+ramdisk. Ramdisk itself has a file system. You can see from the mount instruction that it is the rootfs root file system, which is often called by us. It is mounted by copying it to memory, so it is called ramdisk, If you are interested, you can check the relevant information for further understanding. There is no more to expand here. The kernel dtb is pure machine code. The way to use it is to copy it into memory and wait for bootloader to boot. When the first line of kernel instructions are executed, the system will be handed over to the kernel;

In addition, some firmware or raw data store not code but data. The way to use it is to directly read the contents of the partition. Of course, what you read is cold hexadecimal;

b) An image with a file system is more warm than an image without a file system. I believe we must be very familiar with application development. Our commonly used partitions such as system/vendor/data are ext4 file systems, and a considerable number of applications are placed under system and vendor;

We can check which file systems the system supports through the command cat /proc/filesystems; Common file systems include ext/vfat/fuseblk and other virtual file systems such as sys/proc/debugfs/configfs. Their internal files are not saved in memory, but are dynamically generated after the system is started. They are used to represent system parameters and states, and all files are connected.

Before Android, image s with file systems were created through some basic Linux instructions or written into shell scripts for one click execution; Android integrates this process into the build / compile script, which is not expanded here;

root: /# dd if=/dev/zero of=spongebob.img bs=1M count=1024
root: /# mkfs.ext4 spongebob.img
root: /# mkdir spongebob
root: /# mount -o rw -t ext4 spongebob.img spongebob
root: /# cp source_file spongebob/; sync
root: /# umount spongebob/

View the file system through the file command:

root: /# file boot.img emmc_appsboot.mbn spongebob_raw.img system.img nvram.img 
boot.img:          Android bootimg, kernel, ramdisk, page size: 2048, cmdline (console=ttyHSL0,115200,n8 androidboot.console=ttyHSL0          )
emmc_appsboot.mbn: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, stripped
spongebob.img:     Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (extents) (large files)
system.img:        Android sparse image, version: 1.0, Total of 524288 4096-byte output blocks in 2601 input chunks.
nvram.img:         data

4. Mounting and use

After the image is created, it can be written to the corresponding partition through dd instruction;

root: /# dd if=/mnt/media/xxx/spongebob.img of=/dev/block/mmcblk0p54

If there is no file system, the application layer can directly read and write the partition content through the device node;

For a partition with a file system, mount the partition to the specified directory through the mount command, so that you can directly access the contents of the partition under the mount point path; In Android, the mounting process is generally saved in init In RC, the init process parses it and performs the mounting action of each partition. When mounting, you can specify permissions to restrict the access of applications and improve the stability of the system.

Of course, our test partition can also be mounted manually through the command. Check the partition mounting through the mount or df command;

root: /# mkdir -p /data/tmpdir/
root: /# mount -o ro -t ext4 /dev/block/mmcblk0p54 /data/tmpdir/

5. Make full use of zoning space

Here's a problem I've encountered. Partition size only represents the size of reserved space, but how much space is actually used is determined by image size. Image size determines how much partition is occupied, and the rest is wasted.

Therefore, there is a problem of full utilization. Generally, we can make the image size as large as the partition, and there will be no problem if we brush it in. In this way, we can make full use of the partition space; However, this will also bring a problem. The commonly used user data partitions are as few as a few G and as large as more than a dozen G and tens of G. the userdata image produced by this method is super large and infinitely increases the size of the upgrade package. Therefore, Android optimizes the image size through spark, which greatly reduces the image size and solves the problem of too large upgrade package.

6. Customize ext4 spark image for Android O system

See / device / qcom / common / generate for Qualcomm platform details_ extra_ images. MK, refer to the following excerpt:

#####################################################################################################
# support for spongebob image
ifneq ($(strip $(BOARD_SPONGEBOBIMAGE_PARTITION_SIZE)),)

TARGET_OUT_SPONGEBOB := $(PRODUCT_OUT)/spongebob

INSTALLED_SPONGEBOBIMAGE_TARGET := $(PRODUCT_OUT)/spongebob.img

define build-spongebobimage
  @echo "target spongebob image"
  $(hide) mkdir -p $(1)
  $(hide) $(MKEXTUSERIMG) -s $(TARGET_OUT_DATA) $(2) ext4 spongebob $(BOARD_SPONGEBOBIMAGE_PARTITION_SIZE)
  $(hide) chmod a+r $@
  $(hide) $(call assert-max-image-size,$@,$(BOARD_SPONGEBOBIMAGE_PARTITION_SIZE),yaffs)
endef

$(INSTALLED_SPONGEBOBIMAGE_TARGET): $(MKEXTUSERIMG) $(MAKE_EXT4FS)
	$(hide) $(call build-spongebobimage,$(TARGET_OUT_SPONGEBOB),$(INSTALLED_SPONGEBOBIMAGE_TARGET))

ALL_DEFAULT_INSTALLED_MODULES += $(INSTALLED_SPONGEBOBIMAGE_TARGET)
ALL_MODULES.$(LOCAL_MODULE).INSTALLED += $(INSTALLED_SPONGEBOBIMAGE_TARGET)

endif

###################################################################################################

.PHONY: spongebobimage
spongebobimage: $(INSTALLED_SPONGEBOBIMAGE_TARGET)

Variable BOARD_SPONGEBOBIMAGE_PARTITION_SIZE is image size, which can be configured in other mk; After the environment variable is imported, the image can be created through the command make spongebobimage.

 

summary

The contents mentioned in this article are basically some conceptual contents, including some simple instructions to view the system partition information and how to make ext4 image;

However, what is worth studying is that the kernel uses the partition table to create block devices. We know that emmc has an agreed protocol, and the part of the peripheral protocol is solidified in emmc, so there is no need for the chip manufacturer to provide additional driver to load into the Linux system, and the part of the driver will be realized by the Linux source code; How to generate the image of ext4, including Android compilation script, is also a complex and interesting process. There is a chance to further study and explore it.

Tags: Linux Android

Posted by mlla2 on Wed, 20 Apr 2022 23:55:15 +0300