Deploy Ingress service

14. Deploy Ingress (master01)

Service reverse proxy
Deploy traifik version 2.0

14.1 create traefik CRD Yaml file

Global validation, not limited to ns, defines kind type, which is used for ingress route Use kind in yaml.

mkdir /root/traefik && cd /root/traefik
cat >traefik-crd.yaml<<EOF
## IngressRoute
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
---
## IngressRouteTCP
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
---
## Middleware
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
EOF
14.1.1 create treafik crd resources
[root@master01 traefik]# kubectl create -f traefik-crd.yaml 
customresourcedefinition.apiextensions.k8s.io/ingressroutes.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/ingressroutetcps.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/middlewares.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/tlsoptions.traefik.containo.us created
[root@master01 traefik]# kubectl get CustomResourceDefinition
NAME                                   CREATED AT
ingressroutes.traefik.containo.us      2020-08-11T07:41:42Z
ingressroutetcps.traefik.containo.us   2020-08-11T07:41:42Z
middlewares.traefik.containo.us        2020-08-11T07:41:42Z
tlsoptions.traefik.containo.us         2020-08-11T07:41:43Z

14.2 create traefik RABC file

traefik-rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: kube-system
  name: traefik-ingress-controller
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups: [""]
    resources: ["services","endpoints","secrets"]
    verbs: ["get","list","watch"]
  - apiGroups: ["extensions"]
    resources: ["ingresses"]
    verbs: ["get","list","watch"]
  - apiGroups: ["extensions"]
    resources: ["ingresses/status"]
    verbs: ["update"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["middlewares"]
    verbs: ["get","list","watch"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["ingressroutes"]
    verbs: ["get","list","watch"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["ingressroutetcps"]
    verbs: ["get","list","watch"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["tlsoptions"]
    verbs: ["get","list","watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: kube-system
14.2 file creation permission
[root@master01 traefik]# kubectl create -f traefik-rbac.yaml 
serviceaccount/traefik-ingress-controller created
clusterrole.rbac.authorization.k8s.io/traefik-ingress-controller created
clusterrolebinding.rbac.authorization.k8s.io/traefik-ingress-controller created

Create efikmap.conf file

traefik-config.yaml

kind: ConfigMap
apiVersion: v1
metadata:
  name: traefik-config
data:
  traefik.yaml: |-
    serversTransport:
      insecureSkipVerify: true
    api:
      insecure: true
      dashboard: true
      debug: true
    metrics:
      prometheus: ""
    entryPoints:
      web:
        address: ":80"
      websecure:
        address: ":443"
      redistcp:
        address: ":6379"
    providers:
      kubernetesCRD: ""
    log:
      filePath: ""
      level: error
      format: json
    accessLog:
      filePath: ""
      format: json
      bufferingSize: 0
      filters:
        retryAttempts: true
        minDuration: 20
      fields:
        defaultMode: keep
        names:
          ClientUsername: drop
        headers:
          defaultMode: keep
          names:
            User-Agent: redact
            Authorization: drop
            Content-Type: keep
14.3.1 create traifik configmap resource configuration
[root@master01 traefik]# kubectl create -f traefik-config.yaml  -n kube-system
configmap/traefik-config created

14.4 setting node labels

The node label is set to correspond to traefik deploy Nodeselector: incressproxy: "true" node in yaml file will be used as the node of traefik service, and the node without label will not provide traefik service.

[root@master01 traefik]# kubectl label nodes 192.168.68.149 IngressProxy=true
node/192.168.68.149 labeled
[root@master01 traefik]# kubectl label nodes 192.168.68.151 IngressProxy=true
node/192.168.68.151 labeled
14.4.1 viewing node labels

Check for success

[root@master01 traefik]# kubectl get nodes --show-labels
NAME             STATUS   ROLES    AGE   VERSION   LABELS
192.168.68.149   Ready    <none>   21h   v1.15.1   IngressProxy=true,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=192.168.68.149,kubernetes.io/os=linux
192.168.68.151   Ready    <none>   24h   v1.15.1   IngressProxy=true,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=192.168.68.151,kubernetes.io/os=linux

14.5 create traefik deployment file

Note that ports 80 and 443 of each node node cannot be occupied. Check each node
The ports field can be customized and the ports you need can be added. It is commonly used as 80 and 443.
The production environment can appropriately adjust the size of cpu and memory resources.
traefik-deploy.yaml

apiVersion: v1
kind: Service
metadata:
  name: traefik
  labels:                       
    app: traefik-metrics
spec:
  ports:
    - name: web
      port: 80
    - name: websecure
      port: 443
    - name: admin
      port: 8080
  selector:
    app: traefik
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: traefik-ingress-controller
  labels:
    app: traefik
spec:
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      name: traefik
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 1
      containers:
        #- image: traefik:latest
        - image: traefik:2.0.5
          name: traefik-ingress-lb
          ports:
            - name: web
              containerPort: 80
              hostPort: 80
            - name: websecure
              containerPort: 443
              hostPort: 443
            - name: admin
              containerPort: 8080
            - name: redistcp
              containerPort: 6379
              hostPort: 6379
          resources:
            limits:
              cpu: 200m
              memory: 300Mi
            requests:
              cpu: 100m
              memory: 256Mi
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
          args:
            - --configfile=/config/traefik.yaml
          volumeMounts:
            - mountPath: "/config"
              name: "config"
      volumes:
        - name: config
          configMap:
            name: traefik-config 
      tolerations:              #Set to tolerate all stains and prevent nodes from being set with stains
        - operator: "Exists"
      nodeSelector:             #Set the node filter and start it on the node of a specific label
        IngressProxy: "true"
14.5.1 deploy traefik resources
[root@master01 traefik]# kubectl apply -f traefik-deploy.yaml -n kube-system
service/traefik created
daemonset.apps/traefik-ingress-controller created

[root@master01 traefik]# kubectl get DaemonSet -A
NAMESPACE     NAME                         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR       AGE
kube-system   traefik-ingress-controller   2         2         2       2            2           IngressProxy=true   11m

14.6 traefik routing configuration

14.6.1 configuring traefik dashboard

traefik-dashboard-route.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard-route
  namespace: kube-system
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`ingress.abcd.com`)
      kind: Rule
      services:
        - name: traefik
          port: 8080
14.6.2 deploying dashboard
[root@master01 traefik]# kubectl apply -f traefik-dashboard-route.yaml 
ingressroute.traefik.containo.us/traefik-dashboard-route created
14.6.3 accessing dashboard

Modify local hosts file
192.168.68.149 ingress.abcd.com
Access dashboard

14.7 deploy access services

Create nginx service

kubectl run nginx-ingress-demo --image=nginx --replicas=1 -n kube-system
kubectl expose deployment nginx-ingress-demo --port=1099 --target-port=80 -n kube-system

[root@master01 traefik]# kubectl run nginx-ingress-demo --image=nginx --replicas=1 -n kube-system
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/nginx-ingress-demo created
[root@master01 traefik]# kubectl expose deployment nginx-ingress-demo --port=1099 --target-port=80 -n kube-system
service/nginx-ingress-demo exposed

Create route

cat >nginx-ingress-demo-route.yaml<<EOF
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-nginx-demo-route
  namespace: kube-system
spec:
  entryPoints:
    - web
  routes:
    - match: Host(\`nginx.abcd.com\`)
      kind: Rule
      services:
        - name: nginx-ingress-demo
          port: 1099
EOF

Create nginx proxy routing service

[root@master01 traefik]# kubectl create -f nginx-ingress-demo-route.yaml
ingressroute.traefik.containo.us/traefik-nginx-demo-route created
[root@master01 traefik]# kubectl get IngressRoute -A
NAMESPACE     NAME                       AGE
kube-system   traefik-dashboard-route    16h
kube-system   traefik-nginx-demo-route   57s

After binding the domain name, you can use nginx abcd. Com to visit the nginx service created above.

14.8 create https service

Proxy dashboard https service
Create a self signed certificate

cd /root/traefik
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=cloud.abcd.com"

Store the certificate in kubernetes secret

kubectl create secret tls dashboard-tls --key=tls.key --cert=tls.crt -n kube-system

View system secret

[root@master01 traefik]# kubectl get secret -n kube-system
NAME                                     TYPE                                  DATA   AGE
coredns-token-mv2gz                      kubernetes.io/service-account-token   3      41h
dashboard-tls                            kubernetes.io/tls                     2      13s
default-token-q5bxc                      kubernetes.io/service-account-token   3      2d
traefik-ingress-controller-token-ldcr8   kubernetes.io/service-account-token   3      18h

Create proxy route

cat >kubernetes-dashboard-route.yaml<<EOF
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: kubernetes-dashboard-route
  namespace: kubernetes-dashboard
spec:
  entryPoints:
    - websecure
  tls: 
    secretName: dashboard-tls
  routes:
    - match: Host(\`cloud.abcd.com\`)
      kind: Rule
      services:
        - name: kubernetes-dashboard
          port: 443
EOF

Create kubernetes dashboard proxy routing service

[root@master01 traefik]# kubectl create -f kubernetes-dashboard-route.yaml 
ingressroute.traefik.containo.us/kubernetes-dashboard-route created
[root@master01 traefik]# kubectl get IngressRoute -A
NAMESPACE              NAME                         AGE
kube-system            traefik-dashboard-route      18h
kube-system            traefik-nginx-demo-route     48m
kubernetes-dashboard   kubernetes-dashboard-route   19s

Bind cloud abcd. COM domain name and access through https.

What is the meaning of Ingress?
When we do svc Association, we use nodeport mode to access from the outside. The process is that k8s according to the port configuration in yaml file, we generate the mapping relationship between the port of service (svc) and the port in the container, and then automatically generate a random port of node (port number higher than 32768) to map to the port of service (svc). If many services use nodeport mode at the same time, there will be several problems. 1 We need to find a port corresponding to a service. It will be more complex. 2 The ports on node can not be provided to other services after they are provided to a service. This is a one-to-one correspondence between the service and the port, and the number is limited. How to solve these problems is to introduce the ingress mode. Traefik is an open source tool to implement ingress. The implementation method is that the front-end uses the domain name to associate the service and port. The correspondence between the back-end service and pod does not need to be concerned by the user, and is automatically completed by traefik route. When accessing a service, we only need to access the domain name, which can be resolved to the address of the pod. There is no need to find the IP address and corresponding port of the node where the service pod is located. It simplifies the process of service access.

User access - > domain name - > ingess Service - > SVC Service - > pod

Description of ingress route yaml file:

apiVersion: traefik.containo.us/v1alpha1  -----ingress-route api type
kind: IngressRoute  ---------------------------Resource type,no kubernetes Resource category, but in traefik-crd.yaml Defined in.
metadata:
  name: kubernetes-dashboard-route  -----------ingress-route name,Can pass kubectl get IngressRouter see
  namespace: kubernetes-dashboard   -----------Belonging namespace
spec:
  entryPoints:
    - websecure   -----------------------------Customer access portal traefik-config.yaml Interior configuration
  tls: 
    secretName: dashboard-tls   ---------------tls secret file name 
  routes:
    - match: Host(\`cloud.abcd.com\`)   -------External customer access domain name
      kind: Rule
      services:
        - name: kubernetes-dashboard    -------back-end svc Name, can be from kubectl get svc see
          port: 443    ------------------------svc Port number

14.9 ingress TCP service access

Modify the configuration file traefik config Yaml and traefik deploy port field block in yaml file:
traefik-config.yaml

    entryPoints:
      redistcp:
        address: ":6379"

traefik-deploy.yaml

          ports:
          - name: redistcp
              containerPort: 6379
              hostPort: 6379
14.9.1 TCP ingress cases

Enable redis service

kubectl apply -f redis-tcp-deploy.yaml
[root@master01 traefik]# kubectl apply -f redis-tcp-deploy.yaml 
deployment.extensions/redis-tcp created
service/redis-tcp-svc created

Enable traefik route proxy tcp port

[root@master01 traefik]# more traefik-redis-tcp-route.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: redis-tcp-ingress
spec:
  entryPoints:
    - redistcp
  routes:
  - match: HostSNI(`*`)
    services:
    - name: redis-tcp-svc
      port: 6379
      weight: 10
      terminationDelay: 400

kubectl apply -f traefik-redis-tcp-route.yaml

[root@master01 traefik]# kubectl apply -f traefik-redis-tcp-route.yaml 
ingressroutetcp.traefik.containo.us/redis-tcp-ingress created

[root@master01 traefik]# kubectl get IngressRouteTCP -A
NAMESPACE   NAME                AGE
default     redis-tcp-ingress   42s

14.9.2 testing redis tcp service

We install redis service on master02

yum -y epel-release
yum -y install redis

Modify the hosts file. The host we use in the redis route file is *, so we can bind a domain name at will, assuming that the access domain name is redis cc. COM, or we can directly access the IP and port of node.

cat >> /etc/hosts<<EOF
192.168.68.149 redis.cc.com
EOF

Access the redis service we installed from master02

redis-cli -h redis.cc.com -p 6379
redis.cc.com:6379> set a 1123
OK
redis.cc.com:6379> get a
"1123"
redis.cc.com:6379>info #Get the redis server information. There are many contents here, so they are not all displayed
192.168.68.149:6379> info
# Server
redis_version:6.0.6
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:19d4277f1e8a2fed
redis_mode:standalone
os:Linux 3.10.0-1127.el7.x86_64 x86_64
...

14.10 monitoring traefik services

The following contents will be carried out after the monitoring system of "lesson 6: deploying cluster monitoring system" is deployed
Enable traefik monitoring service
kubectl apply -f traefik-serviceMonitor.yaml

[root@master01 traefik]# kubectl apply -f traefik-serviceMonitor.yaml 
servicemonitor.monitoring.coreos.com/traefik created

Add page to grafana
Import template file traefik 2-1587191399741 json

Installing the grafana plug-in

prometheus-operator-6685db5c6-hszsn   1/1     Running   1          28h
[root@master01 prometheus]# kubectl exec -it -n monitoring grafana-5dc77ff8cb-9lcgc /bin/bash
bash-5.0$ grafana-cli plugins install grafana-piechart-panel
installing grafana-piechart-panel @ 1.6.0
from: https://grafana.com/api/plugins/grafana-piechart-panel/versions/1.6.0/download
into: /var/lib/grafana/plugins

✔ Installed grafana-piechart-panel successfully 

Restart grafana after installing plugins . <service grafana-server restart>

Delete pod
kubectl delete pods grafana-5dc77ff8cb-9lcgc -n monitoring
k8s will automatically rebuild a grafana pod and refresh the dashboard to load the plug-in we just installed.

Log in to the grafana monitoring page again to see the corresponding monitoring display

Simulated access
The number of 404 accesses on the monitor can be viewed by simulating the access to the error page

for i in `seq 1 100001`;do sleep 0.5;curl http://nginx.cc.com/213131;done

Tags: Kubernetes

Posted by jpt62089 on Fri, 20 May 2022 15:54:16 +0300