Understand the automated container deployment (git+jenkins+harbor+kubernetes)

I What is automated container deployment

   in short, we just need to click (or trigger through gitlab hook), and the code will automatically complete construction - > image generation - > container deployment. Compared with traditional deployment, it can save many steps, especially suitable for agile development projects with frequent changes (in fact, container deployment projects are best to use automation, which is quite easy).
  this article is mainly about the understanding and use of various components, and the specific installation steps will not be carried out. It's a simple personal understanding. I hope you can make more corrections.
   the following figure is the flow chart and tools for automated container deployment.

II What is jenkins

   Jenkins is an open source continuous integration (CI) tool with friendly operation interface, which is mainly used for continuous and automatic construction / testing of software projects and monitoring the operation of external tasks. In short, it is responsible for the whole process of our automatic deployment. Whether it is construction, production image or deployment, it is controlled by it. Many things can be completed automatically through different configurations.
   Jenkins commonly used version control tools are SVN and GIT, and commonly used build tools are Maven, Ant and Gradle.

III What are images and dockerfile s

   traditional projects, such as jar package, need to be placed on the host, Java jar runs, such as war package, need to be placed on the host, install a tomcat, and then start SH starts the Tomcat run. One disadvantage of these traditional deployments is that the environment and deployment package are separated. Versions similar to JDK and Tomcat are fixed on the host. If we want to migrate to a new host, we need to install the new environment and ensure that the version of the environment is consistent. This is both troublesome and error prone, resulting in deployment failure.
   therefore, Docker image came into being. In fact, it is equivalent to a runnable deployment package, as well as the environment in which the deployment package runs, operating system, configuration parameters and all other things needed during operation. The image can be started and run on any host, and the environment is consistent, which solves some pain points of traditional deployment. The container is actually equivalent to a running image.
   dockerfile is a file that describes how the image is generated. It generally includes the basic image (such as operating system or tomcat), startup parameters and startup commands.

IV What is harbor

  Harbor is the image warehouse, which can access the image. kubernetes can pull the image through the command of docker pull image warehouse / project name / image name: label, and jenkins can push the built image to the image warehouse. Other commonly used image warehouses include Docker hub, etc.

V What are kubernetes and rancher

   Kubernetes is a portable and extensible open source platform for managing containerized workloads and services, which can promote declarative configuration and automation, referred to as k8s for short. In short, it is a tool that can arrange containers and realize automatic operation, maintenance and management of containers. As mentioned above, containers are running images. Images are deployable units that contain the environment. It is not enough for containers to run alone. We also need functions such as load balancing, service discovery, health check, failure restart, etc. k8s is such a platform.
  rancher is an open source enterprise container management platform. It includes kubernetes, which belongs to the upper layer of container choreography tool. K8s has a very powerful management function for containers, but because it is managed through the command line, the visual display ability is relatively poor. Rancher has a UI interface, through which various functions of k8s can be easily performed, and various functions such as log collection, multi k8s cluster management and monitoring can be added. It can be considered as k8s enhanced version.

Vi Start configuration

1. First create a jenkins project



   the first is the project description. Discard old builds is to discard old builds. Generally, it is recommended to configure (because each project is built for hundreds of mb at a time, and the disk pressure is high). Days to keep builds is to save builds within a few days, and Max # of builds to keep is the maximum number of builds.

2. Configure git


   configure the url of gitlab's pull code. Note that if it is red, the network is generally unavailable or the certificate is wrong. Authentication failed is the certificate is wrong. Click add to enter the account password of gitlab to configure a certificate.

3. Configure hooks

  mention that the hook is webhook, which provides gitlab with an interface to push events and tell jenkins to start automatic deployment.

  check build when a change is pushed to gitlab Gitlab webhook URL, click Advanced options, and then generate the key. With the URL and key, you can configure the hook on gitlab.

   enter the gitlab project, click setting on the left to enter the hook configuration page, fill in the URL and key obtained by jenkins, check Trigger and click Save to configure the hook. It should be noted that push events triggers the hook every time the code is pushed, and tag push event triggers the hook only when the tag is pushed. Generally speaking, tag push is selected to prevent too frequent construction, and the tag can be used as an image tag at the same time to distinguish the image of each construction.

4. Configure build and image generation

#!/bin/bash
#harbor environment variable configuration
export REGISTRY=harbor domain name
export PROJECT=harbor Project name
export BUILD_IMAGE_NAME=Image name
#git_tag_name is the label when gitlab hooks are pushed. It is generally used as a mirror label
export BUILD_IMAGE_TAG=${GIT_TAG_NAME}
export HORBOR_USERNAME=harbor account number
export HORBOR_PASSWORD=harbor password
#Image name
export IMAGE_ADDRESS=${REGISTRY}/${PROJECT}/${BUILD_IMAGE_NAME}:${BUILD_IMAGE_TAG}

#Create an image according to Dockerfile and push the image to harbor
#Open docker service
service docker start
#Log in to the image warehouse
docker login ${REGISTRY} -u ${HORBOR_USERNAME} -p ${HORBOR_PASSWORD}
#Specify the dockerfile according to - f, package the image according to the dockerfile, and specify the image name according to - t
docker build -f ${WORKSPACE}/scripts/apollo-on-kubernetes/apollo-portal-server/Dockerfile -t ${IMAGE_ADDRESS} .
#Image push to warehouse
docker push ${IMAGE_ADDRESS}
#Delete mirror (to relieve jenkins disk pressure)
docker rmi -f  ${IMAGE_ADDRESS}

Select Add   shell to package the script and execute it.

5. Configure k8s push

  Jenkins needs plug-ins to support k8s. Generally, there are the following types:
  Kubernetes Cli Plugin: the plug-in can be operated directly in Jenkins using the kubernetes command line.
If you need to install this plug-in, you need to
  Kubernetes Continuous Deploy Plugin: kubernetes deployment plug-in can be used as needed
   click Manage jenkins to enter jenkins settings, and then click manage plugins to manage plug-ins. Most plug-ins can be directly searched and installed. If the network does not support it, you can also use the plug-in website, such as https://mirrors.tuna.tsinghua.edu.cn/jenkins/ (mirror image of Tsinghua University) or official website https://plugins.jenkins.io/ Download the plug-in before installing it.
  click add post step and select deploy to kubernetes.

   if there is no certificate, you need to obtain the kubernetes config file from the kubernetes cluster, click the cluster, click the kubeconfig file, and then copy the content.


   click add, kind, select kubeconfig in jenkins voucher, check enter directly, paste the previously copied kubeconfig content into content, enter id and description for identification, and then save.

   so far, the configuration of jenkins is completed. You can automatically deploy containers with one click through the hook or click build now.
   note: first create the corresponding namespace and project in harbor and k8s clusters. Refer to the following example for yaml file, or manually deploy the service in Rancher, export yaml, and delete some time and unique id parameters.

VII dockerfile example

General springBoot project example

# Basic image, which is generally the basic operating system image (centos with JDK, etc.)
FROM xxxxx/xxxx/docker-base-os-centos-jdk8:1.0.0
# time zone
ENV TZ Asia/Shanghai
# Copy the jar of the original project and change its name
ADD ${WORKSPACE}/src/webapp/target/webapp-1.0.0.jar server.jar
# Start command
ENTRYPOINT ["java","-jar","-Xms4000m","-Xmx4000m","-Xss256k","-Xmn200m","/server.jar"]

tomcat project example

# Basic image. Here is the tomcat basic image. The address is obtained from the image warehouse
FROM xxxxx/xxxx//tomcat
# time zone
ENV TZ=Asia/Shanghai
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
#Delete all files under webapp
RUN rm -rf /usr/local/tomcat/webapps/*
#Set XX under target Copy war to / usr/local/tomcat/webapps /
ADD ./src/target/javafw-1.0.war /usr/local/tomcat/webapps/ROOT.war
#port
EXPOSE 8080
#Set start command
ENTRYPOINT ["/usr/local/tomcat/bin/catalina.sh","run"]

VIII yaml file example

Workload configuration example, deployment yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    workload.user.cattle.io/workloadselector: deployment-web-admin-web
  name: admin-web
  namespace: web
spec:
  replicas: 1
  selector:
    matchLabels:
      workload.user.cattle.io/workloadselector: deployment-uac-web-admin-web
  strategy:
    rollingUpdate:
      maxSurge: 0
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        workload.user.cattle.io/workloadselector: deployment-web-admin-web
    spec:
      containers:
      - args:
        - --spring.profiles.active=test
# Image name, note, git_tag_name is the label of the push obtained by jenkins from gitlab
        image: XXXXXXXXX/XXXXX/image-name:${GIT_TAG_NAME}
        imagePullPolicy: Always
        name: admin-web
        ports:
#Container port
        - containerPort: 8081
          name: web-8081
          protocol: TCP
      restartPolicy: Always

Cluster service (for internal access), clusterlp service yaml:

# Internally exposed ports
apiVersion: v1
kind: Service
metadata:
  annotations:
    field.cattle.io/targetWorkloadIds: '["deployment:web:admin-web"]'  # Configuring namespaces and workloads
  name: admin-web
  namespace: web
spec:
  ports:
  - name: web-8081-admin-web
    port: 8081
    protocol: TCP
    targetPort: 8081
  selector:
    workload.user.cattle.io/workloadselector: uac-web-admin-web
  type: ClusterIP

Public service, port exposure (for external access of the cluster), nodeport service yaml:

# Exposed port
apiVersion: v1
kind: Service
metadata:
  annotations:
    field.cattle.io/targetWorkloadIds: '["deployment:web:admin-web"]'    # Configuring namespaces and workloads
  name: admin-web-nodeport
  namespace: web
spec:
  ports:
  - name: uac-web-8081
#Exposed port
    nodePort: 8555
#Container internal port
    port: 8081
    protocol: TCP
    targetPort: 8081
  selector:
    workload.user.cattle.io/workloadselector: deployment-web-admin-web
  type: NodePort

IX FAQ record

When a general build fails, you can click a single build and click console output to view the build log and the reason for the failure.

  1. jenkins error in spring boot Maven plugin build: repackage failed: Unable to find main class
    https://blog.csdn.net/qq_35530005/article/details/109333356

Tags: Docker jenkins DevOps

Posted by Natty_Dreadlock on Wed, 27 Apr 2022 07:51:45 +0300