[Cloud Native Training Camp] Module 2 Go Language Advanced

  • function call
  • Common grammar
  • Multithreading
  • Deep understanding of channel
  • Write a producer-consumer program based on channel

function

Main function

  • Every Go program should have a main package
  • The main function in the Main package is the entry of the Go language program
package main
func main () {
	args := os.Args
	if len(args) != 0 {
		println("Do not accept any argument")
		os.Exit(1)
	}
	println("Hello world")
}

init function

  • Init function: will be run when the package is initialized
  • Use init functions sparingly
    • When multiple dependent projects reference the unified project, and the initialization of the referenced project is completed in init and cannot be run repeatedly, it will lead to
      cause startup error
package main
var myVariable = 0
func init() {
	myVariable = 1
}

Pass variable length parameters

Variable-length parameters in Go allow the caller to pass any number of parameters of the same type

  • function definition
func append(slice []Type, elems ...Type) []Type
  • call method
myArray := []string{}
myArray = append(myArray, "a","b","c")

built-in function

Callback

Functions are passed into other functions as parameters, and are called and executed inside other functions

  • strings.IndexFunc(line, unicode.IsSpace)
  • leaderelection of Kubernetes controller s

Example:

func main() {
	DoOperation(1, increase)
	DoOperation(1, decrease)
}
func increase(a, b int) {
	println("increase result is:", a+b)
}
func DoOperation(y int, f func(int, int)) {
	f(y, 1)
}
func decrease(a, b int) {
	println("decrease result is:", a-b)
}

Closure

anonymous function

  • cannot exist independently
  • can be assigned to other variables
    x:= func(){}
  • can be called directly
    func(x,y int){println(x+y)}(1,2)
  • Can be used as a function return value
    func Add() (func(b int) int

interface

  • An interface defines a set of methods

    type IF interface {
    	Method1(param_list) return_type
    }
    
  • Applicable scenarios: There are a lot of interface abstractions and multiple implementations in Kubernetes

  • Struct does not need to explicitly declare the implementation of the interface, just implement the method directly

  • In addition to implementing the interface defined by interface, Struct can also have additional methods

  • A type can implement multiple interfaces (multiple inheritance in Go)

  • Interfaces in Go do not accept property definitions

  • Interfaces can nest other interfaces

  • Precautions

    • Interface may be nil, so the use of interface must be in advance
      Empty first, otherwise it will cause the program to crash(nil panic)
    • Struct initialization means space allocation, references to struct will not have null pointers

reflection mechanism

  • reflect.TypeOf() returns the type of the object being checked
  • reflect.ValueOf() returns the value of the inspected object
myMap := make(map[string]string, 10)
myMap["a"] = "b"
t := reflect.TypeOf(myMap)
fmt.Println("type:", t)
v := reflect.ValueOf(myMap)
fmt.Println("value:", v)

Common grammar

error handling

  • The Go language has no built-in exception mechanism, only the error interface is provided for defining errors

    type error interface {
    Error() string
    }
    
  • New errors can be created via errors.New or fmt.Errorf
    var errNotFound error = errors.New("NotFound")

  • Usually, most of the application's processing of error is to judge whether the error is nil,
    To classify errors, it is usually left to the application to customize, for example, kubernetes customizes different types of errors that interact with the apiserver.

type StatusError struct {
	ErrStatus metav1.Status
}
var _ error = &StatusError{}

// Error implements the Error interface.
func (e *StatusError) Error() string {
	return e.ErrStatus.Message
}

defer

Execute a statement or function before the function returns

  • Equivalent to finally in Java and C#

Common defer usage scenarios: remember to close the resources you open

  • defer file.Close()
  • defer mu.Unlock()
  • defer println("")

Panic and recover

  • Panic: Can actively call panic when an unrecoverable error occurs in the system, panic will make the current thread crash directly
  • defer: guarantee execution and return control to the caller of the function that received the panic
  • recover: function to recover from panic or error scenarios
defer func() {
	fmt.Println("defer func is called")
	if err := recover(); err != nil {
		fmt.Println(err)
	}
	}()
panic("a panic is triggered")

thread lock

Understanding thread safety

Lock:

  • The Go language guarantees thread safety, which can be guaranteed by using channel s and shared memory.
  • The Go language not only provides a communication model based on CSP, but also supports multi-threaded data access based on shared memory, and provides the basic primitives of locks in the Sync package.
  • sync.Mutex Mutex lock, Lock locks, unlock unlocks. Both read and write are mutually exclusive.
  • sync.RWMutex is a read-write separation lock, which does not restrict concurrent reads, but only restricts concurrent writes and concurrent reads and writes.
  • The semantics of sync.WaitGroup is to define a group. If there are 100 threads in this group, each thread should call Done() at the end, and only execute wait when Done() is reduced to 0 at the end. ()
  • sync.Once ensures that a certain piece of code is executed only once
  • sync.Cond allows a group of Goroutine s to be woken up when certain conditions are met (producer, consumer)
sync.NewCond(&sync.Mutex{})
package main

import (
	"fmt"
	"sync" 
	"time"
)

func main() {
	defer fmt.Println("1")
	defer fmt.Println("2")
	defer fmt.Println("3")
	loopFunc()
	time.Sleep(time.Second)
}

func loopFunc() {
	lock := sync.Mutex{}
	for i := 0; i < 3; i++ {
		// go func(i int) {
		lock.Lock()
		defer lock.Unlock()
		fmt.Println("loopFunc:", i)
		// }(i)
	}
}

mutex example:

package main

import (
	"fmt"
	"sync"
	"time"
)

func main() {
	go rLock()
	go wLock()
	go lock()
	time.Sleep(5 * time.Second)
}

func lock() {
	lock := sync.Mutex{}
	for i := 0; i < 3; i++ {
		lock.Lock()
		defer lock.Unlock()
		fmt.Println("lock:", i)
	}
}

func rLock() {
	lock := sync.RWMutex{}
	for i := 0; i < 3; i++ {
		lock.RLock()
		defer lock.RUnlock()
		fmt.Println("rLock:", i)
	}
}

func wLock() {
	lock := sync.RWMutex{}
	for i := 0; i < 3; i++ {
		lock.Lock()
		defer lock.Unlock()
		fmt.Println("wLock:", i)
	}
}

cond example: producer vs consumer

package main

import (
	"fmt"
	"sync"
	"time"
)

type Queue struct {
	queue []string
	cond  *sync.Cond
}

func main() {
	q := Queue{
		queue: []string{},
		cond:  sync.NewCond(&sync.Mutex{}),
	}
	go func() {
		for {
			q.Enqueue("a")
			time.Sleep(time.Second * 2)
		}
	}()
	for {
		q.Dequeue()
		time.Sleep(time.Second)
	}
}

func (q *Queue) Enqueue(item string) {
	q.cond.L.Lock()
	defer q.cond.L.Unlock()
	q.queue = append(q.queue, item)
	fmt.Printf("putting %s to queue, notify all\n", item)
	q.cond.Broadcast()
}

func (q *Queue) Dequeue() string {
	q.cond.L.Lock()
	defer q.cond.L.Unlock()
	for len(q.queue) == 0 {
		fmt.Println("no data available, wait")
		q.cond.Wait()
	}
	result := q.queue[0]
	q.queue = q.queue[1:]
	return result
}

thread scheduling

  • Process: The basic unit of resource allocation

process switching overhead

  • direct cost
    - Toggle Page Table Global Directory (PGD)
    - switch kernel mode stack
    - Switch the hardware context (the data that must be loaded into the register before the process resumes is collectively referred to as the hardware context)
    - Refresh TLB
    - Code execution by the system scheduler

  • overhead
    - Processes that require direct access to memory have more IO operations due to CPU cache invalidation

  • Thread: The basic unit of scheduling

thread switching overhead

  • Threads are essentially just a group of processes that share resources, and thread switching essentially still requires the kernel to perform process switching.
  • Because a group of threads share memory resources, all threads of a process share the virtual address space. Compared with process switching, thread switching mainly saves the switching of virtual address space.
  • Whether it is a thread or a process, it is described by task_strut in linux. From the perspective of the kernel, there is no essential difference with the process.
  • The pthread library in Glibc provides NPTL (Native POSIX Threading Library) support

User thread: Without the help of the kernel, the executable unit created by the application in the user space is created and destroyed completely in the user mode, reducing the consumption of the kernel mode (dependency of system calls).

Goroutine

Go language implements user mode thread based on GMP model

  • G: Indicates goroutine, each goroutine has its own stack space, timer,
    The initial stack space is about 2k, and the space will grow with demand.
  • M (equivalent to the number of CPU s): abstractly represents the kernel thread, records the kernel thread stack information, when the goroutine is scheduled
    When reaching a thread, use the goroutine's own stack information.
  • P: stands for scheduler, responsible for scheduling goroutines and maintaining a local goroutine team
    Column, M obtains and executes the goroutine from P, and is also responsible for some memory management.


where G is located

  • Processes have a global G queue
  • Each P has its own local execution queue
  • There are G s that are not in the run queue
    • G in the channel blocking state is placed in sudog
    • Detach G from P bound to M, like a system call
    • For multiplexing, execution ends by entering G in P's gFree list

Goroutine creation process

  • Get or create a new Goroutine structure
    • Find free goroutines from the processor's gFree list
    • If there is no idle Goroutine, a new structure with sufficient stack size will be created via runtime.malg
  • Move the parameters passed by the function to the stack of the Goroutine
  • Update the properties related to Goroutine scheduling, and the update status is _Grunnable
  • The returned Goroutine will be stored in the global variable allgs

Put the Goroutine on the run queue

  • Goroutine set to the processor's runnext as the task to be executed by the next processor
  • When the processor's local run queue has no remaining space (256), a part of the Goroutine in the local queue and the Goroutine to be added will be added to the global run queue held by the scheduler through runtime.runqputslow

scheduler behavior

  • In order to ensure fairness, when there are goroutines to be executed in the global run queue, schedtick is used to ensure that there are certain
    Chances (1/61) will look up the corresponding Goroutine from the global run queue
  • Find the Goroutine to execute from the run queue local to the processor
  • If the Goroutine is not found by the first two methods, a blocking lookup will be performed through runtime.findrunnable
    Goroutine
    • Find from local run queue, global run queue
    • Find out if there are goroutines waiting to run from the network poller
    • Attempt to steal pending goroutines from other random processors via runtime.runqsteal

Homework

Modify the producer-consumer model in exercise 1.2 into a multiple-producer and multiple-consumer model

Tags: Go Operation & Maintenance Cloud Native

Posted by celsoendo on Wed, 04 May 2022 19:43:02 +0300