Understanding Network Protocols: TCP, UDP, ICMP, IPsec and GRE

I’ve been wanting to properly understand what’s actually happening when two machines “connect” over the internet. We throw around words like TCP, UDP and VPN all the time, but I realised I didn’t understand how they fit together and that started to bother me!

This is me writing it down to make it click properly.

Firstly, what is a “connection”?

When you open a website, nothing physical connects your laptop to the server, we all know that. Instead what happens is your device breaks data into small chunks called packets and those packets get sent across loads of networks until they reach their destination.

A “connection” is just an exchange of those packets following agreed rules which are known as protocols.

A device on a network with an IP address is called a host, so that could be your laptop, a cloud virtual machine, your router, or a server etc.

Networking layers

Before diving more into protocols, I want to spend time on the layers of a network. When I first started reading about networking I kept seeing references to “layers” and it wasn’t immediately obvious what that meant. Networking is split into layers so that different problems are handled separately. Each layer has a specific responsibility and doesn’t need to know how the other layers work.

There is a model called the OSI model (Open Systems Interconnection) which covers the seven layers at play but for now all we need to know is the:

• Physical layer – the actual signal (cable, fibre, WiFi).
• Network layer (IP) – figuring out how to get a packet from one machine to another across networks.
• Transport layer (TCP / UDP) – deciding how two machines exchange data (reliable? fast? ordered?).
• Application layer – the actual thing you care about (HTTP, DNS, SSH, etc).

Each layer builds on the one below it. For example when you open a website:

• HTTP (application layer) sends a request
• TCP (transport layer) ensures it arrives reliably
• IP (network layer) routes it across the internet
• The physical layer moves the bits across cables or radio waves

Transport protocols

This is about how machines talk to each other and it's where most developers will start to recognise things.

TCP (Transmission Control Protocol)

TCP stands out as being reliable and connection-based. Before sending any data, both machines agree to start talking in a small setup process known as a handshake (using SYN and ACK messages). Once that’s done, TCP:

• Numbers every packet
• Makes sure packets arrive in order
• Resends anything that gets lost
• Slows down if the network looks congested

This makes TCP slower than other options, but very safe and reliable. Given its strengths, it's used for:

• Websites (HTTP / HTTPS)
• SSH
• Databases
• Email

If the data must arrive correctly then TCP is usually a default.

UDP (User Datagram Protocol)

UDP is much simpler, whilst I read about TCP being like sending a book in the post, you want every page delivered in the right order, no duplicates etc, then UDP is more like throwing those pages out of a moving car because UDP sends packets without:

• Checking they arrived
• Guaranteeing order
• Resending missing data

Because it does less work, it’s faster and lower latency and so is used for:

• Video calls
• Online gaming
• Streaming
• DNS lookups

If one video frame drops, you won’t notice. If one bank transaction drops, you definitely will and that’s the trade-off.

Network control – how the internet actually manages itself

These following protocols don’t carry website data, they are instead used to help the network function.

ICMP (Internet Control Message Protocol)

ICMP is basically the network’s feedback system. When you run ping, you’re using ICMP.

Routers use ICMP to say things like:

• “Destination unreachable”
• “Time exceeded” (used in traceroute)

It doesn’t deliver web pages it reports problems, so you would see ICMP in use with TCP, they are just at different layers performing different tasks.

There is also an ICMPv6 version for use on IPv6 networks which actually does more as it helps devices:

• Find each other
• Auto-configure IP addresses

So in modern networks ICMPv6 is quite important in infrastructure.

Security at the IP layer (IPsec)

If traffic needs protecting before it leaves the machine this is where IPsec comes in.

AH (Authentication Header)

AH provides integrity and authentication, basically proving a packet wasn’t modified. It does not encrypt the contents though. From what I’ve read, it’s not used much these days because other more prevalent technologies can carry out similar functions.

ESP (Encapsulating Security Payload)

In traditional site-to-site VPNs, ESP is often what provides the encryption underneath. It provides:

• Encryption
• Integrity checking
• Authentication

If a company has a site-to-site VPN between offices then it’s probably ESP underneath.

Tunnelling

GRE (Generic Routing Encapsulation)

GRE wraps one packet inside another packet. It doesn’t encrypt anything instead it encapsulates traffic so it can move across networks that wouldn’t normally support it.

GRE gets combined with IPsec so you get the routing flexibility from GRE with the encryption from an IPsec protocol.

Where do HTTP, DNS and SSH fit in?

Up to now I’ve mainly talked about transport and network-level protocols. But what about things developers use every day like HTTP, DNS and SSH? Those live at the application layer. They sit on top of transport protocols.

• HTTP usually runs over TCP.
• DNS usually runs over UDP, but can use TCP when needed.
• SSH runs over TCP and provides encrypted remote access.

I’m planning to dig into these properly in a follow-up post because that’s where things like HTTPS, TLS and SSL certificates come in which I really want to know about. I also now want to know more about ports and TLS!

Final thoughts

There is a lot to learn when looking at networking properly but writing some of this out has helped me see the layering more clearly.