Configuring k8s cluster on mac to allow service access to intranet via clusterIp

Catalog

Environmental preparation

mac native settings access their k8s service via ip

brew install docker-connector

Configure brew's docker-connector

Map native network to docker container

Other machines in the intranet access this k8s service via ip

Configure dns resolution in intranets

Install dnsmasq

Open k8s CoreDns

Configure dnsmasq

Reference material

Expected effect: We create a deployment of nginx in the default namespace, corresponding to service name nginx-service 80 port open cluster Ip access, expecting other machines (not necessarily in clusters) to enter in the Intranet http://nginx-service.default.svc.cluster.local/ Access to nginx services

Environmental preparation

nginx-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s.kuboard.cn/layer: web
    k8s.kuboard.cn/name: nginx
  name: nginx
spec:
  selector:
    matchLabels:
      k8s.kuboard.cn/layer: web
      k8s.kuboard.cn/name: nginx
  template:
    metadata:
      labels:
        k8s.kuboard.cn/layer: web
        k8s.kuboard.cn/name: nginx
    spec:
      containers:
        - image: 'nginx:alpine'
          name: nginx
          ports:
            - containerPort: 80
              name: http
              protocol: TCP

nginx-service.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 80
  selector:
    k8s.kuboard.cn/name: nginx
  type: ClusterIP

The results after successful execution are as follows:


To achieve input from other machines (not necessarily in clusters) in the Intranet http://nginx-service.default.svc.cluster.local/ Access to nginx services requires three conditions

  1. The dns of all the machines in the intranet resolve nginx-service. Default. Svc. Cluster. You get 10.103.15.56 at local time
  2. Machine access in k8s cluster 10.103.15.56 This ip can see nginx pages
  3. Non-clustered machines in the intranet access 10.103.15.56 This ip can see nginx pages

Tested, if the k8s cluster is linux, the second condition is "Machine access in the k8s cluster 10.103.15.56 This IP can see the page of nginx" is feasible. If you are in this situation, you can skip the next step directly. However, my k8s is built by mac's docker, and it is not accessible for my own computer to access this ip. (

mac native settings access their k8s service via ip

brew install docker-connector

The installation commands are as follows:

brew install wenjunxiao/brew/docker-connector

Configure brew's docker-connector

Want to expose 10.103.0.0/16 of k8s for local access, so `vim/usr/local/etc/docker-connector. Conf`Add a line configuration

route 10.103.0.0/16 expose
expose 0.0.0.0:2512 #Open ports for access by other machines
token user1 192.168.31.103 #Configure machine token

Map native network to docker container

A command maps docker-connector to docker container

docker run -it -d --restart always \
--net host --cap-add NET_ADMIN \
--name connector wenjunxiao/mac-docker-connector

At this point, this Cluster IP service of k8s can be accessed by the local computer via ip, with the effect diagram attached

Other machines need to be installed if they want to access your local container docker-accessor

brew install wenjunxiao/brew/docker-accessor

After installation, you need to get the address and token from the container provider to run with sudo

sudo docker-accessor -remote 192.168.31.17:2512 -token user1

Other machines in the intranet access this k8s service via ip

In order for other machines in the intranet to enter 10.103.15.56 for this IP to access this nginx, it is necessary to have this ip's traffic automatically forwarded to that k8s machine, that is, to make this k8s machine a transit machine for 10.103.0.0/16 this segment of network traffic. There are two ways to achieve this:

1. Set up ip segment forwarding via router

2. Configure ip segment traffic forwarding on each computer using the route command

The second method is used here, and the command on the linux machine is

sudo route add -net 10.103.0.0 netmask 255.255.0.0 gw 192.168.31.17

Configure dns resolution in intranets

Install dnsmasq

Intranet dns software preferred dnsmasq, mac installation command is

brew install dnsmasq

Open k8s CoreDns

coredns-service.yaml,coredns udp 53 port mapped to 32192 port on node

---
apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/port: '9153'
    prometheus.io/scrape: 'true'
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: 'true'
    kubernetes.io/name: CoreDNS
  name: kube-dns
  namespace: kube-system
spec:
  ports:
    - name: dns
      nodePort: 32192
      port: 53
      protocol: UDP
      targetPort: 53
    - name: dns-tcp
      nodePort: 32192
      port: 53
      protocol: TCP
      targetPort: 53
    - name: metrics
      nodePort: 31144
      port: 9153
      protocol: TCP
      targetPort: 9153
  selector:
    k8s-app: kube-dns
  type: NodePort

Configure dnsmasq

The path on the mac is /usr/local/etc/dnsmasq.conf, the path on linux is/etc/dnsmasq.conf, add the following configuration, specify the domain name dns of k8s service to go to 32192 port to query ip:

server=/svc.cluster.local/127.0.0.1#32192

Restart dnsmasq when configuration is complete. The mac command is

sudo brew services start dnsmasq

The linux command is

sudo systemctl restart dnsmasq

The last address to set up the computer's dns server is the address of the intranet dns server, which is omitted. In enterprises, dns addresses are usually set on routers. Take a look at the results at the end

Reference material

The mac/windows docker container ip is open to the host https://blog.csdn.net/adparking/article/details/114026613

dnsmasq installation using tutorial https://xdhuxc.github.io/posts/common/dns/dns-dnsmasq/ 

Tags: Operation & Maintenance Docker Kubernetes

Posted by zkoneffko on Wed, 04 May 2022 04:39:22 +0300