Networking

Internal Networking

Lets start with a single node kubernetes cluster with a single pod and the node has an IP address say it is 192.168.1.2 in this case. This is the IP address we use to access the kubernetes node, ssh etc,.
On a other side, if we are using minikube setup (IP address of the minikube VM inside hypervisor will be considered, laptop may be having different IP).

In docker world, IP address is assigned to a conatiner whereas in kubernetes world IP address is always assigned to a POD.
Each POD in the kubernetes, gets its own internal IP address.(10.244.0.2)

When the kubernetes is initially confgured, we create an internal private network with the address 10.244.0.0 and all the pods are attached to it. When we deploy multiple PODS, they all get a separate IP addresses assigned from this network like 10.244.0.2, 10.244.0.3, 10.244.0.4.
The PODs can communicate with each other through this IP(10.244.0.0), but accessing the other PODs using this internal IP address may not be a good idea as it subject to change when the PODs are recreated.

Cluster Networking

For example, we have two nodes running kubernetes with IP addresses 192.168.1.2 and 192.168.1.3 assigned to them (note these are not part of a cluster yet). Each node has a single POD deployed. These PODs are attached to their internal network and have their own IP addresses assigned.
However, if we observe the internal networks addresses, we can see that they are the same. The two networks have same address(10.244.0.0) and the PODs deployed have the same address too(10.244.0.2).
This is not going to work well when the nodes are part of the same cluster. The PODs have the same IP addresses assigned to them and that leads to IP conflicts in the network. When a kubernetes cluster is setup, kubernetes does not automatically setup any kind of networking to handle this issues. As a matter of fact, kubernetes expects us to setup networking to meet certain fundamental requirements like

• All containers/PODs can communicate to one another without NAT (network address translation-method of remapping an IP address space to another).
• All nodes can communicate with all containers and vice versa without NAT.

We have multiple pre-build solutions available for this such as cisco SCI networks, vmware NSX, fannel, cilium etc,. Dependening on the platform we are deploying kuberenets cluster on, we can use one of these.
kubernetes cluster on our system -- cilium, fannel are good options
kubernetes cluster on vmware -- NSX

With customer networking either with fannel or cilium setup, it now manages the networks and IPs in the nodes and assigns a different network address for each network in the node. This creates a virtual network of all PODs and nodes where they all assigned a unique IP address and by using simple routing techniques, the cluster networking enables communication between the differnt PODs or nodes to meet the networking requirements of kubernetes.
Thus now all the PODs can communicate to each other using the assigned IP address.