dsg_08 serialization system

Serialization

The purpose of serialization is to reduce the load of the network
Serialization technology:

google protobuf
avro
java built-in serialization

Relatively speaking, java serialization is worse than google protobuf and

google protobuf

Download protocol-2.5.0-win32 zip
Design object
Description object
Compile description
Import source code into the project
Use object

Description language of google protobuf

Field type

protobuf type	java type
double	double
float	float
int32	int
int64	long
bool	boolean
string	String

demo

Define a Person serialization class description

//Define package name
option java_package = "cn.com.trueway.pojo";
option java_outer_classname = "AddressBookProtos";
message Person{
	//required indicates that the field must be assigned a value
	required string username = 1;
	//optional indicates that the field can be empty
	optional string xb = 2;
	//repeated specifies that the field is a collection
	repeated string hobbyList;
}

Open the command window and compile the description file to generate java serialized objects

	#protoc. exe --java_ The generation path of the out class is a description file that needs to be edited
	protoc.exe --java_out . person.proto

Copy the generated java serialized object to the project

/**
 * Serialization
 * @throws Exception
 */
public static void writeInJava() throws Exception {
    List<String> hobbyList = new ArrayList<>();
    hobbyList.add("Play basketball");
    hobbyList.add("play football");
    hobbyList.add("Swimming");
    AddressBookProtos.Person person = AddressBookProtos.Person.newBuilder().setUsername("Zhang San").setXb("male").addAllHobbyList(hobbyList).build();

    person.writeTo(new FileOutputStream("D:/aa/person.data"));
}

/**
 * Anti concatenation line
 * @throws Exception
 */
public static void read() throws Exception{
    AddressBookProtos.Person person = AddressBookProtos.Person.parseFrom(new FileInputStream("D:/aa/person.data"));
    System.out.println(person.getUsername());
}

avro

characteristic:

Self description language, data structure and data are stored in files, cross language, using json format to store data
Compressible and cuttable

schema syntax

schema data type

data type	describe
null	No value type
int	32 byte signed integer
long	64 byte signed integer
float	32-bit floating point number
double	32-bit floating point number
bytes	8-bit byte type
string	Unicode character sequence

Inherent attribute

Attribute name	describe
name	The name at the same level as fields is the object name, and in fields is the attribute name
namespace	Namespace
type	The type at the same level as fields is the description document, and in fields is the data type
fields	Attribute array

Use of avro

Compile mode

Define schema file

{
 "type":"record",
 "namespace":"cn.com.trueway.pojo",
 "name":"PersonAvro",
 "fields":[
 	{
 		"name":"username", 
 		"type":"string" 
 	},
 	{ 
 		"name":"xb",
 		"type":"string" 
 	},
 	{ 
 		"name":"age",
 		"type":"int" 
 	}
 ]
}

In avro-tools-1.8.0 Open the command window under the jar package directory of jar and enter the following command to compile the schema and generate java classes
```
#Format: Java - jar avro-tools-1.8.0 Jar compile schema file name generation path
java -jar avro-tools-1.8.0.jar compile schema person.avsc .
```

Import the generated java class into the project and import the dependency

<dependency>
    <groupId>org.apache.avro</groupId>
    <artifactId>avro</artifactId>
    <version>1.8.1</version>
</dependency>

import org.apache.avro.file.DataFileReader;
import org.apache.avro.file.DataFileWriter;
import org.apache.avro.specific.SpecificDatumReader;
import org.apache.avro.specific.SpecificDatumWriter;
	/**
     * Serialization
     * @throws Exception
     */
    public static void write() throws Exception {
        //Create writer object
        SpecificDatumWriter<PersonAvro> sdw = new SpecificDatumWriter<>(PersonAvro.class);
        //write file
        DataFileWriter<PersonAvro> dw = new DataFileWriter<>(sdw);
        PersonAvro pa = new PersonAvro();
        pa.setUsername("Zhang San");
        pa.setXb("man");
        pa.setAge(23);
        //Write class description to file
        dw.create(pa.getSchema(),new File("D:\\aa\\person.avro"));
        //Add data to
        dw.append(pa);
        dw.close();
    }

    /**
     * Anti serialization
     * @throws Exception
     */
    public static void read() throws Exception {
        SpecificDatumReader<PersonAvro> sdr = new SpecificDatumReader<>(PersonAvro.class);

        DataFileReader<PersonAvro> dr = new DataFileReader<>(new File("D:\\aa\\person.avro"), sdr);

        Iterator<PersonAvro> iterator = dr.iterator();

        while (iterator.hasNext()){
            System.out.println(iterator.next().getUsername());
        }
    }

Non compiled mode