Docker's distributed warehouse Harbor and Harbor high availability

Docker's distributed warehouse Harbor

Harbor is an enterprise level Registry server for storing and distributing Docker images. It is open source by VMware. It extends the open source Docker Distribution by adding some necessary functional features for enterprises, such as security, identification and management. As an enterprise private Registry server, harbor provides better performance and security. Promote users to use Registry to build and run environment transfer mirrors
Image efficiency. Harbor supports the replication of image resources installed in multiple Registry nodes. All images are saved in private Registry to ensure that data and intellectual property rights are controlled in the company's internal network. In addition, harbor also provides advanced security features, such as user management, access control and activity audit.

Harbor composition

#harbor is composed of many containers to achieve complete functions
[root@harbor1 ~]# docker ps -a
CONTAINER ID        IMAGE                                    COMMAND                  CREATED             STATUS                    PORTS                                                              NAMES
b18677878280        goharbor/nginx-photon:v1.7.6             "nginx -g 'daemon of..."   10 minutes ago      Up 10 minutes (healthy)   0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp   nginx
94641cfdb140        goharbor/harbor-portal:v1.7.6            "nginx -g 'daemon of..."   10 minutes ago      Up 10 minutes (healthy)   80/tcp                                                             harbor-portal
d26a3568510c        goharbor/harbor-jobservice:v1.7.6        "/harbor/start.sh"       10 minutes ago      Up 10 minutes                                                                                harbor-jobservice
379e870217ec        goharbor/harbor-core:v1.7.6              "/harbor/start.sh"       10 minutes ago      Up 10 minutes (healthy)                                                                      harbor-core
4e25b286cf9e        goharbor/harbor-adminserver:v1.7.6       "/harbor/start.sh"       10 minutes ago      Up 10 minutes (healthy)                                                                      harbor-adminserver
ba8c9ff9f266        goharbor/redis-photon:v1.7.6             "docker-entrypoint.s..."   10 minutes ago      Up 10 minutes             6379/tcp                                                           redis
3a98dd13af8c        goharbor/harbor-db:v1.7.6                "/entrypoint.sh post..."   10 minutes ago      Up 10 minutes (healthy)   5432/tcp                                                           harbor-db
44b6646b5f21        goharbor/registry-photon:v2.6.2-v1.7.6   "/entrypoint.sh /etc..."   10 minutes ago      Up 10 minutes (healthy)   5000/tcp                                                           registry
f225f5977ed3        goharbor/harbor-registryctl:v1.7.6       "/harbor/start.sh"       10 minutes ago      Up 10 minutes (healthy)                                                                      registryctl
cc2307d0c46c        goharbor/harbor-log:v1.7.6               "/bin/sh -c /usr/loc..."   10 minutes ago      Up 10 minutes (healthy)   127.0.0.1:1514->10514/tcp                                          harbor-log

(1) Proxy: corresponds to the startup component nginx. It is an nginx reverse proxy, which delegates the Notary client (image authentication), Dockerclient (image upload and download, etc.) and browser access request (Core Service) to the back-end services.
(2) UI (Core Service): corresponding to the startup component harbor UI. The underlying data storage uses mysql database, which mainly provides four sub functions:
① ui: a web management page ui.
② API: exposed API service of harbor
③ auth: user authentication service. The user information in the token after decode is authenticated here; auth backend can be connected to db, ldap and uaa
④ . token service (not shown in the figure above): it is responsible for issuing a token for each docker push / pull command according to the user's role in each project. If the request sent from the docker client to the registry does not contain a token, the registry will redirect the request to the token service to create a token.
(3) Registry: the corresponding startup component registry. Responsible for storing image files and the pull/push command for image processing. Harbor enforces access control on the image. Registry will forward each pull and push request of the client to the token service to obtain a valid token.
(4) Admin Service: corresponding to the startup component harbor adminserver. It is the configuration management center of the system, which checks the storage consumption. The configuration of adminserver needs to be loaded when the ui and jobserver are started.
(5) Job service: corresponding to the startup component harbor jobservice. In charge of image replication, he communicates with the registry, pulls the image from one registry, then push es it to another registry, and records the job_log.
(6) Log Collector: corresponding to the startup component harbor log. The log summary component summarizes logs through docker's log driver.
(7) DB: corresponding to the startup component harbor dB, which is responsible for storing project, user, role, replication and image_ metadata data of scan, access, etc.

1. Experimental environment preparation

harbor The server	10.0.0.47	centos7
harbor The server	10.0.0.57	centos7
harbor client	10.0.0.100	Ubuntu
harbor client	10.0.0.88	centos8

docker is installed on all four hosts, and the steps are omitted

2. Install docker compose first

Install docker compose
Ubuntu installation method

#Method 1: install through pip with a newer version of docker_compose-1.25.3, recommended
[root@ubuntu1804 ~]#apt -y install python-pip
[root@ubuntu1804 ~]#pip install docker-compose
[root@ubuntu1804 ~]#docker-compose --version
docker-compose version 1.25.3, build unknown
#Method 2: Download and install the corresponding version directly from github
#See instructions: https://github.com/docker/compose/releases
curl -L https://github.com/docker/compose/releases/download/1.25.3/docker-
compose-`uname -s`-`uname -m` -o /usr/bin/docker-compose
chmod +x /usr/bin/docker-compose
#Method 3: install directly. The version is older. docker-compose-1.17.1-2 is not recommended
[root@ubuntu1804 ~]#apt -y install docker-compose
[root@ubuntu1804 ~]#docker-compose --version
docker-compose version 1.17.1, build unknown

harbor1 executes the command and harbor2 executes it repeatedly

[root@harbor1 ~]# yum -y install python-pip
[root@harbor1 ~]# pip install docker-compose
[root@harbor1 ~]# pip install --upgrade pip     #centos7 may report an error when installing compose. You need to execute this command
[root@harbor1 ~]# docker-compose --version
/usr/lib/python2.7/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
  from cryptography.hazmat.backends import default_backend
docker-compose version 1.26.2, build unknown

3. Download the Harbor installation package and extract it

For the following, use the harbor stable version 1.7.6 installation package

[root@harbor1 ~]# wget https://storage.googleapis.com/harbor-releases/release-1.7.0/harbor-offline-installer-v1.7.6.tgzls

Unzip offline package

[root@harbor1 ~]#mkdir /apps
[root@harbor1 ~]#tar xvf harbor-offline-installer-v1.7.6.tgz -C /apps/

4. Edit profile harbor cfg

Both servers execute

[root@harbor1 ~]#vim /apps/harbor/harbor.cfg
#Just modify the following two lines
hostname = 10.0.0.101  #Modify this line to point to the current host IP or FQDN
harbor_admin_password = 123456 #Modify the password of the specified harbor login user admin in this line. The default user / password is admin / harbor 12345
#Optional
ui_url_protocol = http #The default is OK. If it is changed to https, you need to specify the following certificate path
ssl_cert = /data/cert/server.crt #The default is OK. For https, you need to specify the following certificate file path
ss_cert_key = /data/cert/server.key  #The default is OK. For https, you need to specify the following private key file path

5. Run the harbor installation script

Both servers execute

[root@harbor1 ~]# yum -y install python
[root@harbor1 ~]# /apps/harbor/install.sh 

6. Realize the automatic startup of harbor

Method 1: through the service file

[root@harbor ~]#vim /lib/systemd/system/harbor.service
[Unit]
Description=Harbor
After=docker.service systemd-networkd.service systemd-resolved.service
Requires=docker.service
Documentation=http://github.com/vmware/harbor
[Service]
Type=simple
Restart=on-failure
RestartSec=5
ExecStart=/usr/bin/docker-compose -f /apps/harbor/docker-compose.yml up
ExecStop=/usr/bin/docker-compose -f /apps/harbor/docker-compose.yml down
[Install]
WantedBy=multi-user.target
[root@harbor ~]#systemctl daemon-reload
[root@harbor ~]#systemctl enable harbor

Method 2: through RC Local implementation

[root@harbor ~]#cat /etc/rc.local
#!/bin/bash
cd /apps/harbor
/usr/bin/docker-compose up
[root@harbor ~]#chmod +x /etc/rc.local

7. Log in to the harbor host website

Access with browser: http://10.0.0.47/
User name: admin
Password: Harbor The password specified in CFG

8. Using single host harbor

8.1. Establish project

You must create a project on harbor before you can upload the image


8.2. Command line login to harbor

[root@ubuntu1804 ~]#vim /lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd. Sock -- secure registry 10.0.0.47 -- secure registry 10.0.0.57 # add the warehouse IP address that allows login
[root@ubuntu1804 ~]#systemctl daemon-reload
[root@ubuntu1804 ~]#systemctl restart docker
[17:39:48 root@Ubuntu ~]#docker login 10.0.0.47
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
#Check whether the process adds the above settings
[17:40:31 root@Ubuntu ~]#ps aux | grep dockerd
root       2085  0.4  8.3 828156 82424 ?        Ssl  17:39   0:00 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry 10.0.0.47 --insecure-registry 10.0.0.57
root       2255  0.0  0.1  13136  1036 pts/0    S+   17:41   0:00 grep --color=auto dockerd

8.3 label the local image and upload it to harbor

The image cannot be uploaded to the harbor warehouse without modifying the name of images to the specified format

The format is:

Harbor host IP/Project name/image name:edition

example:

#You must log in to harbor before uploading the image
[18:52:10 root@Ubuntu ~]#docker tag d6e46aa2470d 10.0.0.47/example/alpine:v1.0
[18:58:25 root@Ubuntu ~]#docker images
10.0.0.47/example/alpine                              v1.0                d6e46aa2470d        4 weeks ago         5.57MB
[18:58:39 root@Ubuntu ~]#docker push 10.0.0.47/example/alpine:v1.0
The push refers to repository [10.0.0.47/example/alpine]
ace0eda3e3be: Pushed 
v1.0: digest: sha256:d7342993700f8cd7aba8496c2d0e57be0666e80b4c441925fc6f9361fa81d10e size: 528

8.4. Download the image of harbor

Before downloading, you must modify the service file of docker and add the address of harbor server before downloading

example:

[root@Centos8 ~]#vim /lib/systemd/system/docker.service 
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry 10.0.0.47 --insecure-registry 10.0.0.57
[root@Centos8 ~]#systemctl daemon-reload 
[root@Centos8 ~]#systemctl restart docker
[root@Centos8 ~]#docker pull 10.0.0.47/example/alpine:v1.0
v1.0: Pulling from example/alpine
188c0c94c7c5: Pull complete 
Digest: sha256:d7342993700f8cd7aba8496c2d0e57be0666e80b4c441925fc6f9361fa81d10e
Status: Downloaded newer image for 10.0.0.47/example/alpine:v1.0
10.0.0.47/example/alpine:v1.0
[root@Centos8 ~]#docker images
REPOSITORY                 TAG                 IMAGE ID            CREATED             SIZE
10.0.0.47/example/alpine   v1.0                d6e46aa2470d        4 weeks ago         5.57MB

Automatically label and upload image script

[19:34:38 root@Ubuntu ~]#cat build.sh 
#!/bin/bash
docker build -t $1 .
docker tag $1 10.0.0.47/example/$1
docker push 10.0.0.47/example/$1

9. Achieve harbor high availability

Harbor supports the policy based docker image replication function, which is similar to the master-slave synchronization of MySQL. It can realize the synchronous image between different data centers and different operating environments, and provides a friendly management interface, which greatly simplifies the image management in the actual operation and maintenance. It has been used in many cases of Internet companies using harbor to build intranet docker warehouse, and has realized the two-way replication function.

If you want to copy the image of the first host to the second host, you need to configure it in the first host interface and copy the image in the 10.0.0.47 warehouse to the 10.0.0.57 warehouse.

Host operation at 10.0.0.47

Create replication rule



The rule is saved successfully, and the copy starts automatically

To realize bidirectional replication, you need to do the same on the 10.0.0.57 host

#Upload the image on the 10.0.0.57 host to test the two-way replication
[20:57:12 root@Ubuntu ~]#docker tag 0d120b6ccaa8 10.0.0.57/example/centos:v1.0
[20:57:28 root@Ubuntu ~]#docker push 10.0.0.57/example/centos:v1.0
The push refers to repository [10.0.0.57/example/centos]
291f6e44771a: Pushed 
v1.0: digest: sha256:fc4a234b91cc4b542bac8a6ad23b2ddcee60ae68fc4dbd4a52efb5f1b0baad71 size: 529

Tags: Linux Operation & Maintenance CentOS Docker Ubuntu

Posted by skippy111 on Fri, 06 May 2022 15:38:20 +0300