20-CS-6053 Network Security Spring 2019
Network Protocols and Firewalls

Secret Key, Public Key, Hash Algorithms, IPSec, Kerberos, Authentication, more


Some of the Figures below have been lifted from the following sites: www.elabs.fr, and www.vicomsoft.com. It is hoped administrators at those sites will not be upset by this.

Network Layers

OSI Model - The Seven Layers

  1. Physical: unstructured stream of bits across a link
  2. Data Link: organizes physical layer's bits into packets
  3. Network: computes internet paths and forwards packets from src to dst
  4. Transport: establishes reliable communication stream
  5. Session: extra features to maintain reliability
  6. Presentation: encodes application data into system-independent format
  7. Application: user applications

TCP/IP Model - Five Layers

  1. Physical: unstructured stream of bits across a link
  2. Data Link: organizes physical layer's bits into packets
  3. IP (internet protocol): computes internet paths and forwards packets from src to dst
  4. TCP and UDP: TCP establishes reliable communication stream
  5. Application: user applications

Why Layers?

New protocols for safely, reliably, confidentially communcating information can be facilitated without fundamentally changing the transport or physical nature of the interenet. The idea is to send data in packets with a header containing subsections each with some level information.


What type of protection?

  1. By segmenting the network, limits damage to a few machines
  2. Confidentiality
  3. Denial of Service
  4. Corruption of Data

How does it fit into a network?

What it does

  1. Examines all traffic (every packet) to see if it meets certain criteria. If so, the packet passes. If not, it is filtered (and maybe some alarms are set off).
  2. Logs all attempts to enter a network. Although logs can be spoofed or eliminated.
  3. Possible criteria for filtering:
    1. Source and/or destination address and port numbers
    2. Type of traffic (say by protocol - maybe disallow ntp)
    3. Packet attribute or state

What it does not do

  1. Stop "good" users with modems
  2. Stop "malicious" users with or without modems
  3. Stop damage due to password misuse (Phonemasters cracker ring)

How it works

  1. Filtering criteria depends on the layer at which firewall operates
  2. The lowest layer it can operate on is layer 3 (that is, routing packets).
  3. At layer 3: can determine whether packet is from trusted source, but cannot be concerned with what it contains or other associated packets.
  4. At layers 4 and 5: can determine much more about packet, especially in relation to associated packets
  5. However, the lower the layer a packet is intercepted, the more secure the firewall: if an intruder cannot get past level 3, s/he cannot grab ahold of the operating system

Firewalls may have their own IP layer

They catch each packet before the OS sees it.

Types of Firewalls

Packet Filtering Firewall


  1. Cheap
  2. Transparent to end users: applications do not have to be reworked
  3. Can filter services built around protocols supported by the firewall


  1. Administrator must be familiar with protocols on the network
  2. No authentication
  3. Does not conceal protected network's architecture
  4. A compromised computer presumed "secure" can penetrate the firewall
  5. Does not protect against weaknesses in unfiltered services
  6. Logs would be meaningless

Circuit Level Gateway

TCP traffic is monitored to determine whether a requested session is legitimate. Information going out to the internet appears to come from the gateway, thereby hiding information about protected networks. These do not filter individual packets.

Application Level Gateway (proxies)


  1. Incoming or outgoing packets are filtered unless directed toward a specific proxy to handle a specific service. For example, http server will not field requests for ftp service
  2. Can even filter requests such as http::get or http::post
  3. Can log activity effectively
  4. Security rules are easy to define
  5. User authentication is implemented
  6. Internal architecture of the network may be concealed.
  7. Address translation in effect


  1. Expensive
  2. Too powerful for small networks
  3. Require manual configuration of each client
  4. Cause major reduction of throughput
  5. Not transparent to end users: need special client software to take advantage
  6. System administration is much harder
  7. No easy support for new services

Multilayer Inspection Firewall

Are expensive and are potentially less secure than simpler firewalls if not administered by highly competent personnel.

Firewall Problems

  1. Convenience is sacrificed
  2. Traffic bottleneck
  3. Security is concentrated in one spot

Firewall Attacks

SYN Flooding

Each TCP/IP layer imposes limits regarding the number of TCP connections waiting for a port (usually small) and therefore service. Normal handshake for entry is as follows:

There is a time out of about 75 seconds: if no ACK is received the request is cancelled. But if the SYN/ACK packet goes to the wrong machine a RST is sent instead of an ACK and connection is terminated.

Hacker spoofs return address to machine that does not exist and never responds to SYN/ACK packets. The result is denial of service.

IP Spoofing

Hacker uses the IP address of a trusted computer to send packets to a firewalled network. But ACK packets are sent to the trusted computer which would respond with RST. So, simultaneously, the hacker SYN floods the trusted computer so it cannot respond.

Use virtual private network protocol such as IPSec. Then data and header are encrypted and signed and source address can be authenticated.

Source Routing

Normally, the route a packet takes from its source to its destination is determined by the routers between the source and destination. The packet itself only says where it wants to go (the destination address), and nothing about how it expects to get there.

There is an optional way for the sender of a packet (the source) to include information in the packet that tells the route the packet should take to get to its destination; thus the name ``source routing''.

Source routing can be strict or loose. Strict source routing lets a manager specify the path through all the routers to the destination. Return responses use the same path in reverse. Loose source routing lets managers specify an address that the packet must pass through on its way to the destination. It is loose source routing that aids an attacker.

A remote attacker might seek to access a Unix system protected with TCP wrappers, or a Windows NT Internet Information Server (IIS) protected by an access list based on source addresses. If the attacker simply spoofs one of the permitted source addresses, the attacker may never get a response. However, if the attacker both spoofs an address and sets the loose-source-routing option to force the response to return to the attacker's network, the attack can succeed.

The simplest defense against loose source routing is to not permit these packets to enter (or leave) the network. Just about any firewall will block any packet that has source routing enabled by default. You can also configure routers to block packets with source routing. TCP wrappers and many Unix OSs can also block source-routed packets.

Ping Flooding

Send burst of ping packets to disrupt bandwidth.

ICMP Redirect

An ICMP Redirect tells the recipient system to over-ride something in its routing table. It is legitimately used by routers to tell hosts that the host is using a non-optimal or defunct route to a particular destination, i.e. the host is sending it to the wrong router. The wrong router sends the host back an ICMP Redirect packet that tells the host what the correct route should be. If you can forge ICMP Redirect packets, and if your target host pays attention to them, you can alter the routing tables on the host and possibly subvert the security of the host by causing traffic to flow via a path the network manager didn't intend. ICMP Redirects also may be employed for denial of service attacks, where a host is sent a route that loses it connectivity, or is sent an ICMP Network Unreachable packet telling it that it can no longer access a particular network.

Many firewall builders screen ICMP traffic from their network, since it limits the ability of outsiders to ping hosts, or modify their routing tables.

IP Fragmentation

IP packets can be 65536 bytes but ethernet frames are 1500 bytes, max. Fragment the ethernet packets and include info for reassembly. But TCP header only shows up in first ethernet frame. Hence, it is possible to send fragmented IP packets which reassemble to more than 65536 (Microsoft problem) and crash the targeted computer.

Server Hijacking

This is where a spammer will take many thousands of copies of a message and send it to a huge list of email addresses. Because these lists are often so bad, and in order to increase the speed of operation for the spammer, many have resorted to simply sending all of their mail to an SMTP server that will take care of actually delivering the mail.

Of course, all of the bounces, spam complaints, hate mail, and bad PR come for the site that was used as a relay. There is a very real cost associated with this, mostly in paying people to clean up the mess afterward.

Exploiting Bugs in Applications

Various versions of web servers, mail servers, and other Internet service software contain bugs that allow remote (Internet) users to do things ranging from gain control of the machine to making that application crash and just about everything in between.

The exposure to this risk can be reduced by running only necessary services, keeping up to date on patches, and using products that have been around a while.

Bugs in Operating Systems

Again, these are typically initiated by users remotely. Operating systems that are relatively new to IP networking tend to be more problematic, as more mature operating systems have had time to find and eliminate their bugs. An attacker can often make the target equipment continuously reboot, crash, lose the ability to talk to the network, or replace files on the machine.

Here, running as few operating system services as possible can help. Also, having a packet filter in front of the operating system can reduce the exposure to a large number of these types of attacks.

And, of course, chosing a stable operating system will help here as well. When selecting an OS, don't be fooled into believing that ``the pricier, the better''. Free operating systems are often much more robust than their commercial counterparts

What Can Be Done To Help?

In case of TCP: check the ACK bit of every packet. That is, allow only those packets which are responding to requests from behind the firewall. Hence, a remote machine cannot initiate a TCP connection. This eliminates the possibility of SYN flooding, among other things.

But, cannot serve unless ports are opened up.

Does not protect UDP packets (no ACK there). But DNS for example uses UDP.

Some applications must initiate TCP connections (example: FTP). However, implementation of a "passive" mode might help.