Harbor: A trusted cloud native registry for Kubernetes and Docker

Key features

Harbor offers several significant advantages over other container registries, including security, signing, vulnerability scans, and auditing. Let's discuss the key features of Harbor:

• Ownership and deployment: When you host your container images on other registries, such as Amazon ECR or Azure Container Registry, your content is not completely under your control. Harbor gives you peace of mind, as it can be deployed in a private data center where you are the sole owner of your content.
• Role-based access control: Corporate organizations often require stringent access control. In Harbor, the repositories and users are organized into projects, and you can control access to each project individually.
• Image replication: Harbor allows you to replicate repositories between multiple container registries, such as Docker Hub, Google GCR, Azure ACR, Amazon ECR, Artifactory, and Quay, with the help of endpoints. This is particularly useful for high availability and load balancing.
• External identity providers: In addition to integrated database authentication, Harbor supports external identity providers, such as Active Directory (AD) and Lightweight Directory Access Protocol (LDAP), for authentication and authorization. Furthermore, it supports OpenID Connect (OIDC) to verify the users authenticated by external identity providers.
• Signing and verification: Harbor supports Notary and Cosign to sign artifacts and verify signatures. These services are crucial for enabling content trust and making sure your container images are legitimate.
• Vulnerability analysis: Harbor can utilize Trivy, Clair, and other pluggable scanners to perform static vulnerability scanning of images and artifacts to ensure that your content is free from vulnerabilities. Starting with Harbor 2.2, Trivy is now the default vulnerability scanner, but you can also install Clair, or even both, if required.
• CVE exception policies: When a vulnerability is identified in an image, the security setting can prevent the use of such images. You might want to add certain common vulnerabilities and exposures (CVEs) to your allowlist to permit the use of images until the vulnerability is fixed. Harbor allows you to add a system-wide or project-specific allowlist by specifying the CVE IDs. You can also set an expiration time for these allowlists so that a certain vulnerability can be ignored only for a specific period.
• Webhook events and triggers: Harbor can notify the webhook endpoint when a certain event occurs in a project. It also allows the integration of Harbor with various CI/CD tools. So, you can configure Harbor to keep admins informed about important events as they occur. For example, you can get notified when a container image is deleted or a vulnerability is discovered. You can even trigger deployment to the Kubernetes cluster when an image is successfully pushed.
• Robot accounts: Starting with Harbor v2.2, admins can now create robot accounts that can be used for performing automated and repetitive tasks in Harbor. Robot accounts cannot log in to the Harbor web interface, but they can be assigned permissions to do things like pull artifacts, create a tag, create a scan, etc. You can also configure an expiration period for robot accounts.
• Garbage collection: Garbage collection (GC) is a compute-intensive operation, but it is necessary to clean up and claim free space back to the file system. With earlier versions, the Harbor instance needed to be switched to read-only mode when garbage collection was running, which required longer downtimes. However, Harbor v2.1 introduces online nonblocking garbage collection, so your team no longer needs to wait while GC is running. You can configure GC to run according to your preferred schedule.
• Project quotas: Project quotas allow admins to keep a check on storage usage. For example, when you set a 10 GB storage quota for a project, it cannot grow beyond that size, so you can avoid running out of storage accidentally. In addition to project quotas, you can use tag retention policies to remove outdated images and optimize storage automatically.
• Internal TLS communication: Harbor supports TLS encryption to secure internal communication between various Harbor components to prevent eavesdropping, tampering, and other attacks. Service-to-service TLS encryption is highly recommended for production environments.
• Auditing: Auditing is a key feature of any enterprise-class tool to help with compliance and troubleshooting. Harbor supports auditing critical events, such as user authentication, repository creation, image push/pull, image deletion, and much more. These events are recorded in an audit log, which may be kept locally or sent to a remote syslog server.
• Graphical user interface: A GUI is a really important aspect of any modern application because not everyone likes the command line interface. Harbor offers a neat web interface where admins and users can easily manage repositories, projects, container images, settings, etc.
• Backup and restore: The backup and restore feature is always loved by admins because you never know when something might go wrong. You can use Velero (an open-source tool) to back up and restore your Harbor registry server when needed.
• REST API: The REST API allows you to interact with Harbor programmatically to perform various operations, such as managing container images, repositories, users, and projects. It also allows developers to build custom applications to interact with the Harbor registry server.

Note that listing all Harbor features would blow up this article.

Prerequisites

The Harbor registry can be deployed on a Kubernetes cluster. However, to keep things simple, I will show you how to install it on Ubuntu 22.04. The recommended hardware configuration for Harbor is as follows:

• Processor: 4 cores
• Memory: 4 GB
• Storage: 160 GB

Additionally, the following software components must be installed on the host:

• Docker engine
• Docker Compose
• OpenSSL

Run the following commands in the same order to install these components:

sudo apt update && sudo apt install openssl ca-certificates curl gnupg -y
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo \
"deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
"$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

These commands will install all the prerequisites, such as the Docker engine, Docker Compose, and OpenSSL.

Harbor installation

The Harbor installer is available on GitHub in two variants. If the host system (where you want to install Harbor) is connected to the internet, you can download an online installer. Since it pulls everything from the internet during installation, it is fairly small in size. To install Harbor on an offline host, you need to download a much larger offline (full) installer.

To download and install Harbor, connect to the Ubuntu system using SSH and follow these steps:

1. First, create a self-signed TLS certificate using openssl, and move the certificate and private key to their respective directories with the following commands:
   

   mkdir certs
   openssl req -x509 -nodes -days 730 -newkey rsa:4096 -sha256 -keyout certs/harbor.testlab.local.key -addext "subjectAltName = DNS:harbor.testlab.local" -out certs/harbor.testlab.local.crt
   sudo mv certs/harbor.testlab.local.crt /etc/ssl/certs/
   sudo mv certs/harbor.testlab.local.key /etc/ssl/private/
   

   

   Generating a self signed certificate for Harbor with OpenSSL

   Don't forget to modify these commands according to your environment. Alternatively, you can generate a free letsencrypt certificate.

2. Now download the offline or online harbor installer, depending on your situation. My Ubuntu system is connected to the internet, so I will download the online installer with this command:
   

   curl -s https://api.github.com/repos/goharbor/harbor/releases/latest | grep browser_download_url | cut -d '"' -f 4 | grep '\.tgz$' | grep online | wget -i -
   

   

   Downloading Harbors online installer

   To download the offline (full) installer, use the following command instead:
   curl -s https://api.github.com/repos/goharbor/harbor/releases/latest | grep browserdownloadurl | cut -d '"' -f 4 | grep '\.tgz$' | grep offline | wget -i -

3. Now, unpack the archive with the following command:
   tar -xvzf harbor-online-installer-v.tgz
   

   Unpacking the online installer archive

   After unpacking, you will see a new directory named harbor in your current working directory.

4. In the harbor directory, there is a configuration template (harbor.yml.tmpl), an installation script (install.sh), and a preparation script (prepare).
   

   Viewing the contents of the harbor directory

5. Copy the harbor.yml.tmpl template to the harbor.yml file.
   

   Creating a harbor configuration file from a template

6. Now, open the harbor.yml file with nano (or another text editor) and adjust the various options, as shown in the screenshot below:
   

   Making changes to the harbor.yml file

7. We have already discussed how Harbor uses various components. It can be installed in various configurations. To start the Harbor installation, use one of the following commands:
   

   #install harbor only
   sudo ./install.sh
   #install harbor with notary service
   sudo ./install.sh --with-notary
   #install harbor with trivy service
   sudo ./install.sh --with-trivy
   #install harbor with chartmuseum service
   sudo ./install.sh --with-chartmuseum
   #install harbor with all services
   sudo ./install.sh --with-notary --with-trivy --with-chartmuseum
   

   
   I installed Harbor with the notary and trivy services, as shown in the screenshot below:
   

   Installing Harbor with notary and trivy services

   The notary service is used for signing and signature verification, and the trivy service is used for vulnerability scanning. In upcoming Harbor versions, you need to use cosign instead of a notary service. Furthermore, you can also append the --with-clair parameter to use the Clair service—a much more extensive vulnerability analysis tool compared to trivy.

Harbor lifecycle

Harbor essentially works in Docker containers, so we can manage the entire lifecycle with Docker compose commands. To manage the Harbor registry, navigate into the harbor directory and then use the commands shown below.

To stop the Harbor registry, run the sudo docker compose stop command.

To start the Harbor registry, run the sudo docker compose start command.

Whenever you want to make any changes to the Harbor configuration (harbor.yml), first run the sudo docker compose down -v command to stop the containers and remove the associated volumes that are defined in the Docker Compose file.

After making changes to the harbor.yml file, run these commands in the same order:

./prepare
sudo docker compose up -d

The first command runs the prepare script, and the second command launches the Harbor containers with the updated configuration.

Harbor GUI

Once the installation is finished successfully, you can access the Harbor web interface using the hostname defined in the harbor.yml file. In my case, it is https://harbor.testlab.local.

If the password you specified in the harbor.yml doesn't work at first login, try running these commands:

sudo docker compose down
sudo docker compose up -d

Note that covering every GUI option is beyond the scope of this post, so I will cover only a few settings that are needed to start hosting your images.

1. First, change your password from the GUI, as shown in the screenshot.
   

   Changing the Harbor admin password from the GUI

2. Under the Users menu, click the New User button to create a new user.
   

   Creating a new user in the Harbor GUI

   Harbor creates a default library project, which is public.

3. Click the Add Project button to create a new project to store your private container images.
   

   Creating a new project in the Harbor GUI

4. To set the project quota, click the Project Quotas menu.
   

   Setting the project quota in the Harbor GUI

5. As discussed in the Key features section, Harbor allows you to replicate container images between other Harbor and non-Harbor registries. So, if you have another Harbor registry instance or an Amazon ECR instance, you can create an endpoint for that in the Harbor GUI. To do so, select the Registries menu and then click the New Endpoint button.
   

   Creating an endpoint for the external registry in the Harbor GUI

6. After you define an endpoint, click the Replications menu, and then click the New Replication Rule button.
   

   Configuring a new replication rule in the Harbor GUI

Here, you can configure the replication mode, source registry, resource filter, trigger mode, etc. At this point, our Harbor registry server is ready to host some private container images.

Push images to Harbor

I will connect to a different VM where I have a customized container image stored. Consider this your development machine, where you have packaged your business application in a container image. Remember, since our Harbor registry uses a self-signed certificate, we need to add the following line in the /etc/docker/daemon.json file.

{
"insecure-registries" : ["https://harbor.testlab.local"]
}

You can skip this step if you are already using a valid TLS certificate. Now, use the following commands to log in to your private Harbor registry and push the image:

#login to private harbor registry
docker login harbor.testlab.local
#list available docker images
docker image ls
#tag a docker image
docker tag webapp harbor.testlab.local/private/webapp
#push a docker image to harbor registry
docker push harbor.testlab.local/private/webapp

You can see in the screenshot that the image was pushed successfully. You can also see this in the Harbor web interface.

Use a Harbor image in Kubernetes

We can now use Minikube to run a Kubernetes cluster on another VM. But since we are using a self-signed certificate with our Harbor registry, we also need to copy its certificate to the ~/.minikube/certs directory on the host where we will run Minikube. This can be done with a simple scp command:

sudo scp surender@kube-srv1:/etc/ssl/certs/harbor.testlab.local.crt ~/.minikube/certs/

Once the certificate is copied, follow these steps to launch a Kubernetes cluster and run your business app:

1. Run the minikube start command, as shown in the screenshot:
2. Remember, we have stored our container image in a private registry that cannot be pulled without a secret. To add a secret, run the following command:
   

   kubectl create secret docker-registry harbor --docker-server=harbor.testlab.local --docker-username=surender --docker-password=Pass@890 --docker-email=root@testlab.local
   

   

   Creating a secret for Harbor in the Kubernetes cluster

   This command adds a new secret to the Kubernetes cluster with the name harbor.

3. Now, create a new webapp-deployment.yaml file, and paste the following code:
   

   apiVersion: apps/v1
   kind: Deployment
   metadata:
   name: webapp-deployment
   labels:
   app: webapp
   spec:
   selector:
   matchLabels:
   app: webapp
   replicas: 2
   template:
   metadata:
   labels:
   app: webapp
   spec:
   containers:
   - name: webapp
   image: harbor.testlab.local/private/webapp:latest
   ports:
   - containerPort: 80
   imagePullSecrets:
   - name: harbor
   

   
   Notice that at the end, we use the image option to specify our container image stored in Harbor and imagePullSecrets to specify the name of the secret (harbor, in our case).
4. Once you have the deployment file ready, run the following command to create a deployment:
   

   kubectl apply -f webapp-deployment.yaml
   

   

   Creating a deployment in the Kubernetes cluster

   You can see that the deployment is created, and the pods are also created successfully. If the status of your pod shows ErrImagePull or ImagePullBackOff, something might be wrong with your configuration.

   

   Kubectl pod status showing ErrImagePull or ImagePullBackOff

   In that case, you need to run the kubectl get events command to see what's wrong. If you see the Failed to pull image "harbor.testlab.local/private/webapp:latest": rpc error: code = Unknown desc = Error response from daemon: Get "https://harbor.testlab.local/v2/": x509: certificate signed by unknown authority error in the logs, you need to run the minikube delete command, copy the Harbor certificate to the ~/.minikube/certs directory and run the minikube start command again.

5. Now that our pods are running, run the following command to expose the deployment:
   

   kubectl expose deployment webapp-deployment --port=80 --type=LoadBalancer
   

   Subscribe to 4sysops newsletter!

   

   Exposing a deployment in the Kubernetes cluster

   Since we are not working on a supported cloud platform, the deployment will be exposed with a NodePort service instead of LoadBalancer.

6. Finally, run the minikube service webapp-deployment command, and your web app will be launched automatically in a web browser on the host where you are running Minikube.
   

   Opening the deployment service in a web browser

   

   Viewing the web app deployed in the Kubernetes cluster

The screenshot above shows that I was able to access the web app on the Kubernetes host. That is all for this post. You have now successfully deployed a Harbor registry server, pushed a container image using the Docker client, and finally, deployed the image in a Kubernetes cluster with Minikube.