- Scalability and availability for Azure Web Apps - Wed, Jun 28 2017
- Managing Azure Virtual Machine scale sets - Wed, May 31 2017
- Azure virtual machine scale sets - Mon, May 8 2017
Containers are certainly one of the hottest topics today. This year, Microsoft partnered with Docker to bring the Docker platform to Windows Server 2016 by introducing Windows/Hyper-V containers as well as native Docker engine support. That's a huge step for overall container technology to push them through to production in enterprise organizations.
There are many things to consider when implementing containers. Conversation usually starts with applications.
- Which application is a good fit for containers – monolithic, micro-services, stateless or stateful?
- Service discovery?
- What about security patching?
- Container monitoring?
- Orchestration tools?
Given the simplicity of container technology, it's now much easier to deal with underlying infrastructure and fabric to run containers. Actually, containers don't care too much about infrastructure as they keep all processes inside the container, isolated from external threats.
But for enterprise level deployments and production environments, network settings are something you may want to take into account. How do containers talk to each other? What type of network setup should we have for containers? How can you provide public access to containers or configure a network to allow containers to talk to each other within a private network?
Microsoft leverages similar Hyper-V networking components (vSwitch, vNIC) in order to provide the network layer to containers. Once you enable the container feature on a Windows box, a new Hyper-V Virtual Ethernet Adapter will be created and all containers will have a vNIC connected to vSwitch.
vNIC created for containers
Viewing NAT configuration
After you install the containers feature and the Docker package, a default network called "NAT" will be created. You can retrieve container networks using the Docker CLI or the PowerShell Get-ContainerNetwork cmdlet.
docker network ls
Listing available networks with Docker CLI
Get-ContainerNetwork
Listing available networks with the PowerShell cmdlet Get ContainerNetwork
NAT is the same network as "bridge0" in Linux container environments. By default, it's created for each container host with an IP Prefix of 192.16.0.0/12. If you don't specify any additional network parameter/flag while running containers, each container will be attached to this network, and the Docker engine provides a free IP address available on the NAT network to containers. The default NAT network also supports port forwarding from container host to internal containers. For example, you can simply run SQL Server Express in a container by providing the "p" flag so that specified port numbers will be mapped from host to container. In this way, I can easily access my database from an external network.
Let's check the networking configuration for a container. In order to run commands inside the container, you can use docker exec. The following command will run PowerShell inside my container:
docker exec -it ac0722b09f2e powershell
Checking network configuration inside the container
As can be seen, the container has a virtual network adapter (vNIC) connected to my virtual switch and it's configured with an IP address from the default NAT network prefix.
All containers have a configuration file that allows you to check each and every detail including networking. This file can be found under c:\ProgramData\Docker\Containers\<containerID>
Container config file location
The config.v2.json file has all the configuration details for this container:
Config JSON file content for containers
If you want to get more information for a particular container network, you can use the "docker network inspect <networkName>" command.
This shows available container networks, all connected containers, and network resources.
Docker network inspect command
On the above output, you can easily see that no containers are attached to this network yet. If you run some containers and don't specify any custom network flags, these containers will be attached to the NAT network automatically.
After starting some containers, you can try to inspect the same network again to see the attached containers.
Docker network inspect command to see attached containers
Connect and disconnect NAT
If you recall from the first command (docker network list), there was also one additional network called "none." You can connect your existing containers to another available network using docker disconnect/connect commands. The following commands switch my existing container network from "nat" to "none."
docker network disconnect nat <containerID> docker network connect none <containerID>
Connectdisconnect container networks
If you check the config.json file again, you will see that the networkID and endpointID values are now empty.
Config file with no network attached containers
Configuring NAT
You may want to change the IP prefix of the default NAT network so that the engine will assign private IP addresses to containers from the range you specified.
Under the C:\ProgramData\Docker\config\ directory you can create the daemon.json file which is actually the Docker config file for the Docker service on your Windows host. It doesn't show up automatically when you install the container feature, so you need to create that file manually in most cases.
In the current version of Windows containers, only following options are supported in Docker configuration files:
{
"authorization-plugins": [],
"dns": [],
"dns-opts": [],
"dns-search": [],
"exec-opts": [],
"storage-driver": "",
"storage-opts": [],
"labels": [],
"log-driver": "",
"mtu": 0,
"pidfile": "",
"graph": "",
"cluster-store": "",
"cluster-advertise": "",
"debug": true,
"hosts": [],
"log-level": "",
"tlsverify": true,
"tlscacert": "",
"tlscert": "",
"tlskey": "",
"group": "",
"default-ulimits": {},
"bridge": "",
"fixed-cidr": "",
"raw-logs": false,
"registry-mirrors": [],
"insecure-registries": [],
"disable-legacy-registry": false
}
By changing the values for those options, you can configure your Docker environment as per your requirements. You only need to add the desired options to the configuration file and remove unused options. For our example, we want to change the IP prefix of the default NAT network.
<"fixed-cidr": ""> is the option which will create the default NAT network with the IP prefix you specified.
In order to play with the Docker configuration file, you need to stop the Docker service first.
Stop-Service docker
Then you need to remove the existing NAT network using native PowerShell commands:
Get-ContainerNetwork | Remove-ContainerNetwork
Preparing Docker service for Docker config file changes
Now you can create the daemon.json file and make the desired changes. Below, I just added a section to the configuration file:
{
"fixed-cidr" : "192.168.10.0/24"
}
And finally, you can start the Docker service again.
Start-Service docker
If you use the Get-ContainerNetwork or docker network inspect command, you can see that a new NAT network is created by the Docker service with the IP prefix we specified in configuration file:
Inspecting NAT network with custom IP prefix
Hereafter, all created containers will pick an IP address from this custom NAT network unless you specify custom network flags in the run command.
Subscribe to 4sysops newsletter!
In the second part, we are going to look at creating custom NAT drivers and additional network drivers you can implement in Windows containers.
Is it possible to have multiple nat networks on one docker host?
My requirements are:
1. I need to be able to create multiple groups of containers connected with a chosen network. So for instance container A, B and C are in network1, while X and Z containers are in network2.
2. I need to be able to map containers ports to host ports
I followed your instruction but i got the following error when i start docker.
“HNS failed with error : The object already exists. ”
Any idea what could be the cause?
Hi Michael,
could be an issue with a previous NAT you have created before. I would run Get-NetNat |Remove-NetNat command first and try again.
also make sure to use most updated version of all components as issues are being fixed quite often.
This is the type and level of discussion which is useful to me. Thank you.
I have use-cases similar to what Piotr mentioned above:
[Piotr]”1. I need to be able to create multiple groups of containers connected with a chosen network. So for instance container A, B and C are in network1, while X and Z containers are in network2.
2. I need to be able to map containers ports to host ports”
I have a windows server 2016 datacenter VM where I turned on the container feature and hyper-v role as well as installed Docker. I see the ‘nat’ network and can access the containers successfully using the exec -it command.
I ran my container with a -p port map (host-port:container-port) and it starts successfully. I got the IP of the container and use that IP to test the service running in my container, however, I cannot ping the container from this 2016 VM or hit my service in a browser or invoke-webrequest or curl call.
The sample container prints to the powershell console – docker run microsoft/dotnet-samples:dotnetapp-nanoserver. However, no containers that require a call from the server 2016 host into the container, nor any call out of the container is successful.
How do I get my service to communicate with the windows host and external apps/services that will call it?
push … Windows Server 2016 Datacenter with Containers …
Whatever I try I can not connect to my Azure Network machines.
The Switch seems to be internal because I can ping the Host but no other computer and also not the gateway.
As I’m searching a solution for this since one week and there are so many people who have the exact same problem and there is no working solutions for this i will have to contact ms.