- Kubernetes CoreDNS - Mon, Dec 4 2023
- Update container images with Copa - Mon, Nov 27 2023
- Deploying stateful applications with Kubernetes StatefulSets - Wed, Nov 1 2023
Docker's networking subsystem works with the help of drivers, where the default driver uses a network bridge. When you create a Docker container without explicitly specifying a network, it is automatically connected to the default bridge network. However, some applications don't work properly in a bridged network using NAT. This is where the macvlan network driver comes in. In this guide, you will learn how to configure macvlan.
Prerequisites
To follow along with this guide, you need a physical server or a VM running on Ubuntu Linux. If you haven't already installed Docker, run the following commands to install it:
sudo apt update sudo apt install docker.io -y
Up- and downsides of the default bridge network
Before directly jumping into macvlan, let me first cover the advantages and disadvantages of the default Bridge and Host networks.
Pros of the default bridge network
- It creates an isolated private network where containers can communicate directly with each other and with the Docker daemon. For internet access, containers need to use NAT.
- It uses an isolated subnet (172.17.0.0/16), which allows containers to communicate using IP address only. It doesn't support container name resolution.
Cons of the default bridge network
- It doesn't support name resolution, so containers cannot communicate using names.
- Every new container is by default connected to this network when the network isn't explicitly specified. So you do not get the best security and isolation.
- It isn't ideal for every application, since containers use NAT for internet access. NAT is known for its slow network performance, and your application might not support NAT (or double NAT).
The following screenshot shows how you can view default Docker networks:
To get complete control of Docker networking, you can use a custom (or user-defined) network. This also allows you to use container names instead of IP addresses because you can then work with name resolution. However, this doesn't solve the NAT performance issue.
To avoid NAT problems, you can use the Host network. The Host network allows Docker containers to share the same IP address and port number on the Docker host. However, this solution comes with its own problems. The Host network doesn't offer any network isolation, and it is prone to port-conflict issues. In a nutshell, the application appears to be running directly in the host operating system.
Why use the macvlan network?
The macvlan network assigns a unique MAC address to each container, making it appear to be a physical device on your network, just like a traditional virtual machine. The Docker daemon then routes the traffic to containers on the basis of their MAC address. It also allows you to assign an IP address from the same subnet in which the Docker host resides. This avoids the use of the host network, there is no NAT overhead, and you won't run into network performance issues.
Let's consider that you have a legacy application like a network traffic monitoring system that you want to containerize. Your application currently runs on a VM, since it requires being directly connected as a separate device on a physical network. In such a situation, macvlan is an ideal choice. Below are some benefits of macvlan:
- Allocates a unique MAC address to each container, allowing it to have a full TCP/IP stack.
- Allows you to allocate IP addresses to containers from the same subnet, which is managed by your enterprise IT.
- Allows a Docker container to become part of a physical network as a separate network device. It allows containers to utilize advanced network services, such as VLANs and IPAM.
- Removes the additional overhead caused by NAT.
How does macvlan work?
To understand how the macvlan network works, take a look at the diagram below.
You can see that the physical network interface (enp0s3) in my Docker host has IP address 192.168.0.40/24. It is connected to the gateway router (192.168.0.1), which is in the same subnet.
The macvlan network (macvlan_net) on the Docker host is configured with the parent interface set as a physical interface (enp0s3). At the bottom of the diagram, you can see the Docker command used to create the macvlan network. I will discuss this in more detail in the next section.
Configure the macvlan network.
Now, let's create our macvlan network based on the diagram.
- To create a macvlan network, use the docker network create command with additional options:
sudo docker network create \ --driver macvlan \ --subnet 192.168.0.0/24 \ --gateway 192.168.0.1 \ --ip-range 192.168.0.80/28 \ --aux-address 'host=192.168.0.80' \ --opt parent=enp0s3 \ macvlan_net
As you can see in the screenshot, I've split the docker network create command into multiple lines using a backslash (\) for better readability and understanding.
- The --driver option specifies the macvlan driver instead of the default bridge driver.
- The --subnet option specifies the subnet for the new macvlan network, which must be the same as that of the Docker host.
- The --gateway option specifies the gateway for the macvlan network.
- The --ip-range is optional, and you can skip it if you're planning to manually allocate an IP address to each Docker container. Just make sure that the IP range specified here is excluded from your primary DHCP server to avoid potential IP conflicts. I used the 192.168.0.80/28 subnet, which gives me 16 IP addresses in the macvlan.
- The --aux-address 'host=192.168.0.80' option allows you to exclude a specific IP address from the above mentioned IP range. This will cause the Docker DHCP to start allocating with 192.168.0.81.
- The --opt parameter specifies additional options, such as the parent interface. You can use it multiple times to specify multiple options.
- The last option sets a name for the new macvlan network.
- You can now view the new Docker network using the commands shown below:
sudo docker network ls sudo docker inspect macvlan_net
- Let's now create two containers, as per our diagram.
sudo docker run --name webapp --rm --detach --network macvlan_net nginx sudo docker run --name database --rm --detach --network macvlan_net --env MARIADB_ROOT_PASSWORD=Pass@321 mariadb
The first command creates a webapp container using an nginx image, and the second command creates a database container using the mariadb image in detach (background) mode. The --network option is key here, since it specifies the macvlan network to connect the containers instead of the default network.
- If you run the docker inspect macvlannet command again, you will be able to view the containers and their network configuration, as shown in the screenshot.
- You can see that both containers received a unique MAC address. The IP address is allocated from the IP range we used while defining our macvlan network. If you do not want to use an auto-assigned IP address from the Docker DHCP range, you can specify the static IP address of your choice with the --ip option when creating the container, as shown in the following command:It is your responsibility to make sure the IP address you allocate this way is not in use by another device in your network.
sudo docker run --name alpine --rm -itd --network macvlan_net --ip 192.168.0.100 alpine
Now, inside your container, you will be able to ping other containers connected to the macvlan network using their IP addresses and container names.sudo docker exec -it webapp /bin/bash
Verifying connectivity with other containers in the same macvlan network
As you can see in the screenshot above, I can ping other containers using their names, which means the name resolution is working. However, I cannot ping the default gateway because the promiscuity of the parent interface is 0, by default. Because of this, the other devices on your network will not be able to ping the Docker containers. - To allow communication between the Docker containers and the default gateway, you need to enable promiscuity mode on the host interface (enp0s3), as shown in the following commands:
sudo ip -detail -color link show enp0s3 sudo ip link set dev enp0s3 promisc on
If your Docker host is a virtual machine, you might also need to enable promiscuous mode on the virtual network interface in the VM settings.
Now, your container will be able to ping the gateway, and other devices will be able to ping docker containers.
Similarly, other devices on the network can now ping both Docker containers connected to the macvlan network.
- There is one last problem. By default, the traffic originating from the Docker containers to the Docker host (and vice versa) is filtered by the kernel to enforce strong network isolation and security. Therefore, our Docker host (192.168.0.40) at this point is not able to ping Docker containers, and containers will not be able to ping the Docker host.To solve this problem, we could create a macvlan interface on the Docker host and tell the Docker host to use this macvlan interface to pass traffic to the containers. Do you remember that we excluded an IP address, 192.168.0.80, from the macvlan IP range? We can use this IP address for our new macvlan interface on the Docker host. To do so, run the following commands:
sudo ip link add macvlan_int link enp0s3 type macvlan mode bridge sudo ip address add 192.168.0.80/32 dev macvlan_int sudo ip link set macvlan_int up sudo ip route add 192.168.0.80/28 dev macvlan_int
- Take a look at the newly created macvlan interface and route on the Docker host with the following commands:
sudo ip -br -col add show sudo ip route
Your Docker host can now communicate with containers, and containers can communicate with the Docker host without any problem. See the following screenshots for reference:
Subscribe to 4sysops newsletter!
Conclusion
You just learned how to deploy two Docker containers in a macvlan network inside Docker. Your containers act as individual network devices and can successfully communicate with other devices on your network.
Read the latest IT news and community updates!
Join our IT community and read articles without ads!
Do you want to write for 4sysops? We are looking for new authors.
–aux option is missing in the screen capture so inspect display is incomplete.
Just corrected the screenshot. Thanks for the hint.
End of point 5. I CAN ping the gateway from my container, even with a host NIC promiscuity equal to 0.
The only thing I can’t do at this stage is to ping from the dockerd host to the container and reciprocally.
Great write up!
I’m a little confused, as before step 7 in the end of the article you provided a screenshot where we can see there’s ping from the host (srv2) to the docker containers.
Then, in step 7 you specify there’s a problem to ping them, and offer the solution. Finally, right before the conclusion you use the same screenshot again, showing now there’s ping.
Will you explain?
It was a wrong screenshot that I just corrected. Thank you for the hint.
In step 7 you explained that “the traffic originating from the Docker containers to the Docker host (and vice versa) is filtered by the kernel”.
However, in steps 5 and 6 we can see successful ping in both directions.
Will you please explain?
In step 5, docker containers could ping each other successfully but not the default gateway of docker host. In step 6 after changing promiscuity, ping from containers to gateway (and from external network devices to containers) start working as expected.
The kernel-level filtering only stops the communication between docker containers and docker host; It won’t affect communication between containers and other devices on your network.
My previous comment and your reply got deleted, so I’ll try again.
In step 6 you’re showing we can ping dockers from the outside, and annotating a picture “Verifying connectivity from other network device to the Docker containers”, but then in step 7 you’re explaining how to enable bidirectional communication. Please explain.
I spent a long time studying Docker’s macvlan and went through most of the articles on Google, wasting a lot of time. Finally, I found this article that explained everything in the simplest and clearest way, and I’m extremely grateful!
I’m glad you liked it. Thank you for the comment.
The last command fails for me with Error: Invalid prefix for given prefix length.
This is the network interface details:
ip addr show eno1
altname enp2s0
inet 10.10.0.7/24 brd 10.10.0.255 scope global noprefixroute eno1
valid_lft forever preferred_lft forever
docker network create \
–driver macvlan \
–subnet 10.10.0.0/24 \
–gateway 10.10.0.1 \
–ip-range 10.10.0.30/28 \
–aux-address ‘host=10.10.0.30’ \
–opt parent=eno1 \
macvlan_net
sudo ip link set dev eno1 promisc on
sudo ip link add macvlan-shim link eno1 type macvlan mode bridge
sudo ip address add 10.0.0.30/32 dev macvlan-shim
sudo ip link set macvlan-shim up
sudo ip route add 10.0.0.30/28 dev macvlan-shim
Hi,
In the last command, you’re adding a route to 10.0.0.30/28 but your docker macvlan network ip range is 10.10.0.30/28.
As a side note, 10.10.0.30/28 is a valid host address but not a valid network address. You might want to use 10.10.0.16/28 as the IP range for your Docker network.