Evading Intrusion Detection

by Seclabs

Evading Intrusion Detection

Executive Summary

Network Associates Security Labs

[email protected]

I. Introduction

Network intrusion detection is a security tool that attempts to detect actual attacks (as opposed to potential vulnerabilities) against networked hosts by analyzing network traffic. There are many different ways in which network traffic can be obtained by a network intrusion detection system (IDS). This document demonstrates that one class of network IDS, which relies on passive network packet capture (commonly referred to as ?sniffing?), is fundamentally flawed. We show that sniffer-based IDS is, as a technology, immature, and thus cannot be relied upon for network security.

The basis for these claims is the results of several months of research which scrutinized the most popular commercially-available network ID systems, including ISS RealSecure, WheelGroup NetRanger, AbirNet SessionWall-3. We also examined Network Flight Recorder?s NFR traffic analysis engine product, testing it in it?s capacity to perform as a network IDS.

In each system tested, we found disastrous problems that would allow an attacker not only to fool the IDS into believing an attack had occurred when it hadn?t (allowing malicious crackers to ?frame? other people for nonexistent incursions), but also to slip arbitrary real attacks past the systems undetectably. ID systems that provide ?countermeasures?, such as the ability to reconfigure router filters, could also be subverted to attack the networks employing them, by denying connectivity to legitimate traffic.

Worse still, several of the problems we uncovered have no currently known solutions. These problems can be exploited to prevent an IDS from discerning specifically what type of attack is being employed by an attacker. Although an IDS relying on sniffers can detect that an attack is occurring, it cannot reliably provide the detailed information that it is marketed to be able to provide.

This document outlines the general problems faced by ID systems that rely on passive network packet capture to detect intrusions. It goes on to illustrate how these problems can be exploited by an attacker to reliably evade (or subvert) the IDS. Finally, we discuss the ramifications of the problems, and some measures that can be taken to mitigate the risk they pose.

II. Problems

Our research identifies two fundamental problems faced by sniffer-based IDS. The first is that there is not enough information ?on the wire? for a sniffer to reconstruct reliably what any two machines are saying to each other; the second is that sniffer-based IDS is inherently vulnerable to denial of service attacks.

A. Lack of Information

At first glance, it may be very surprising that a network sniffer can be incapable of figuring out what?s happening on a network. Assuming that a sniffer has access to all the network packets exchanged by two machines, it would seem obvious that an IDS has all the information it needs to reconstruct the information being passed in those packets. However, this reasoning is based on a major (and incorrect) assumption: that all the packets a sniffer reads off the network are valid.

In reality, attackers trying to elude an IDS can forge packets. If an attacker spoofs them correctly, these forged packets can be very difficult to discern from ?real? packets. If an IDS cannot differentiate the fake packets from the real ones, it can become hopelessly confused, recording information that is completely different from what was actually sent by the attacker.

Furthermore, attempts by an IDS to separate the fake packets from the real ones can be exploitable in and of themselves. If an IDS mistakenly discards ?real? packets, it can miss information crucial to the detection of an attack. An attacker can take advantage of this by sending entire attacks inside of packets that an IDS will never even look at.

Our work isolates two different attacks that exploit this problem --- ?insertion?, where an attacker forces an IDS to read information that isn?t valid, confusing it about what?s actually happening --- and ?evasion?, where an attacker slips entire packets past the IDS by making them look invalid when they aren?t.

B. Denial of Service

?Denial of service? (DOS) attacks attempt to compromise the ?availability? of a computing resource. By this, we mean that they attempt to force a computer system to cease functioning the way it is intended to. DOS attacks can be as simple as ?ping floods?, which prevent people from using networks by clogging them with useless packets, to carefully coordinated protocol attacks (such as the recent ?teardrop? IP fragmentation bug).

It is particularly important that security systems be resistant to denial of service attacks. If an attacker can prevent a security system from working, she can negate the protection it provides, attacking other systems as if it didn?t exist. For instance, an attacker that can disable a firewall can potentially attack the systems behind it.

Because of this, most good security systems are designed to ?fail closed?. A system that ?fails closed? continues to provide protection even when it is disabled. For example, a fail-closed firewall will cut network connectivity if it crashes; no traffic can pass through such a firewall unless the firewall is functioning properly.

By their very nature, sniffer-based ID systems are ?fail-open?. Sniffers are, as their vendors claim, unobtrusive; other systems on the network don?t even need to be aware that they exist. If a sniffer fails, network traffic still flows across the network. This means that an attacker can target a sniffer-based IDS directly, disabling it and preventing it from recording her attacks against other systems.

Unfortunately, it?s widely believed that denial-of-service attacks are among the most difficult security problems to solve. An attacker can consume all the memory available on an IDS by flooding the network with traffic; the IDS, which needs to process traffic for all hosts on the network (unlike normal machines), is apt to starve long before other hosts do. Additionally, a bug in a network IDS can be exploited to crash the system outright. Unlike firewalls, when the IDS crashes, the network it protects is exposed to attacks.

C. Summary

These core problems lead to the existence of three attacks that can be used against a network IDS --- ?insertion? and ?evasion?, which allow an attacker to fool the IDS into incorrectly reconstructing information from network packets, and denial-of-service attacks, which can be exploited to stop the IDS from working outright. In the next section of this paper, we take a look at each of these attacks, and present examples that real ID systems are vulnerable to today.

III. Attacks

An attacker targeting a network IDS has many potential objectives. The most likely of these is simply to attack a machine that the IDS is watching without the IDS noticing it; any of the attacks we present can allow an attacker to do this. Another possible goal is to ?spoof? attacks from fake addresses, framing other people for attacks. An attacker can also create ?fake? attacks in order to trigger a reaction from the IDS, causing it to cut network connectivity for a legitimate service.

We outline here three different attacks that can be employed by an attacker to compromise a network IDS. The first two aim to reduce the accuracy of the system, causing it to see something other than what is actually happening on the network. The third tries to completely disable the IDS.

A. Insertion

The first of our attacks, which we call ?insertion?, allows an attacker to confuse an IDS by forcing it to read packets off the wire that are not valid. By ?invalid?, we mean that the packet would not be accepted by the machine it is addressed to; a packet can be invalid because it is malformed in some way, or because it simply never reaches the actual destination. Regardless of the specific manner in which a packet is invalid, if an IDS mistakenly reads it, it can become permanently confused about the meaning of other network traffic.

To understand how insertion becomes a problem for network ID systems, it?s important to understand how they detect attacks. An IDS detects attacks in network traffic by employing pattern-matching algorithms to look for specific patterns of data in a packet or stream of packets. For example, an IDS might look for the string ?phf? in an HTTP request to discover a PHF CGI attack. An attacker that can insert packets into an IDS can prevent pattern matching from working. For instance, an attacker can send the string ?phf? to a web server, attempting to exploit the CGI vulnerability, but force the IDS to read ?phoneyf? (by ?inserting? the string ?oney?) instead.

One simple insertion attack involves intentionally corrupting the IP checksum. Every packet transmitted on an IP network has a checksum, which is used to verify whether the packet was corrupted in transit. IP checksums are 16 bit numbers that are computed by examining information in the packet. If the checksum on an IP packet does not match the actual packet, the host to which it is addressed will not accept it.

Two of the systems we tested (ISS RealSecure and Network Flight Recorder NFR) did not validate the IP checksum before processing captured packets. An attacker can trick either of these systems into reading an invalid packet simply by setting the IP checksum to an incorrect value; only the IDS will read such a packet.

Insertion attacks were the most commonly discovered vulnerabilities in all the ID systems we tested. In general, insertion attacks are enabled any time an IDS is less strict about accepting packets than the systems it protects. Because of this, it may seem logical to design a network IDS to be a strict as possible about accepting packets. Unfortunately, this design approach leads to an even more severe problem --- evasion attacks.

B. Evasion

An ?evasion? attack occurs whenever an IDS discards a packet that is accepted by the host that it is addressed to. An attacker employing evasion attacks is literally slipping packets under the nose of the IDS. Evasion attacks are devastating to the accuracy of an IDS; an evasion attack at the IP layer allows an attacker to attempt arbitrary attacks against hosts on a network, without the IDS ever realizing it.

One example of an evasion attack occurs when an attacker opens a TCP connection with a data packet. Before any TCP connection can be used, it must be ?opened? with a handshake between the two endpoints of the connection. A fairly obscure fact about TCP is that the handshake packets can themselves bear data. An IDS that doesn?t accept data in these packets is vulnerable to an evasion attack. The versions of AbirNet SessionWall-3 and WheelGroup NetRanger that we tested were both vulnerable to this attack. By placing entire HTTP requests inside of TCP SYN (connection initiation) packets, we were able to slip PHF attacks past the IDS.

C. Denial-of-Service

There are many different types of denial of service attacks that are valid against ID systems. The attacks we?ll discuss here all involve resource exhaustion --- the attacker identifies some point of network processing that requires the allocation of some sort of resource, and causes a condition to occur that consumes all of that resource. Resources that can be exhausted by an attacker include CPU cycles, memory, disk space, and network bandwidth.

The CPU processing capabilities of an IDS can be exhausted because the IDS spends CPU cycles reading packets, determining what they are, and matching them to some location in saved network state (for example, an IP fragment needs to be matched to the other fragments of the datagram it represents). An attacker can determine what the most computationally expensive network processing operations are, and force the IDS to spend all its time doing useless work.

ID systems require memory for a variety of things. TCP connection state needs to be saved, reassembly queues need to be maintained, and buffers of data need to be created for pattern matching. The system requires memory simply to read packets in the first place. As the system runs, it allocates memory as needed to perform network processing operations (for example, the receipt of an IP fragment means that the ID system will need to obtain memory to create and maintain an IP fragment queue for that packet). An attacker can determine which processing operations require the ID system to allocate memory, and force the IDS to allocate all its memory for meaningless information.

At some point, most ID systems will need to store logs of activity on disk. Each event stored consumes some amount of disk space, and all computers have a finite amount of disk space available. An attacker can create a stream of meaningless events and, by having them continually stored, eventually exhaust all disk space on the IDS, which will then be unable to store real events.

Finally, network ID systems track activity on the networks they monitor. For the most part, they are capable of doing this only because networks are very rarely used to their full capacity; few monitor systems can keep up with an extremely busy network. The ID system, unlike the end-systems, must read everyone?s packets, not just those sent specifically to it. An attacker can overload the network with meaningless information and prevent the ID system from keeping up with what?s actually happening on the network.

D. Complex Attacks

In many cases, it?s hard to fix one problem in a network IDS without introducing a new one. For instance, fixing a problem that allows an evasion attack can introduce insertion attacks (by making the IDS accept more packets, it?s possible to make it accept too many). A real-world problem makes this even more difficult.

Different operating systems deal with TCP/IP packets in slightly different ways. Although TCP/IP is a carefully standardized set of protocols, it?s possible for a system to disobey the standard slightly and still interoperate. An attacker can stretch the protocol to it?s limits to take advantage of quirks in operating systems.

In some cases, the standards don?t even specify exactly how a system should behave. For instance, the IP protocol uses a concept called ?fragmentation? to allow a packet to travel across many different types of networks with different limitations on packet sizes. Fragmentation allows routers to break packets up into smaller packets; the destination of the original packet ?reassembles? the fragments into the original packet.

Fragments can overlap each other. This happens when two fragments of the same packet contain the information for the same part of that packet. The IP standard does not mandate any particular way in which this must be resolved. Because of this, it?s possible for two different operating systems to reassemble two entirely different packets, given the same stream of fragments.

If the IDS doesn?t know exactly what operating systems it?s watching, it may not be able to reassemble fragment streams correctly, allowing an attacker to slip attacks past it. Unfortunately, we were unable to test this, because all of the systems we tested failed to reassemble fragments at all.

IV. Reviews of Actual ID Systems

A. RealSecure

RealSecure is Internet Security Systems, Inc.?s automated network intrusion detection system. We tested version 1.0.97.224 for Windows NT.

RealSecure appears to have two independent network monitor components. The first of these handles pattern matching within captured packets; the second provides a ?real-time playback? capability that allows administrators to watch the information being exchanged in a TCP connection in real-time.

This is important, because the ?playback? facility in RealSecure appears to be much weaker than the pattern matching facility. RealSecure session playback does not reorder TCP packets, and it doesn?t check sequence numbers. An attacker can cloud the playback window by sending a stream of nonsense packets with bad sequence numbers (a trivial insertion attack).

The most significant problem with RealSecure, as with all the other systems we tested, was that it did not handle IP fragmentation reassembly at all. An attacker can completely evade RealSecure by fragmenting every packet she sends across the network.

RealSecure also appeared to have serious problems with TCP reassembly when duplicate segments appeared on the network. RealSecure never detected an attack in any of the tests we ran that involved sending multiple segments with the same sequence number.

RealSecure does not appear to pay attention to TCP RST messages. We were able to desynchronize RealSecure by closing a connection with a client RST message, and then immediately reconnecting using the same parameters. RealSecure recognized attacks in streams even after their connection was reset. RealSecure also does not appear to pay attention to TCP SYN messages; we were able to desynchronize RealSecure from our connections by preceding them with arbitrary data segments with random sequence numbers.

Finally, RealSecure was vulnerable to all of our insertion attacks. It did not appear to check IP or TCP checksums, nor did it verify that the ACK bit was set on TCP data segments.

ISS is, at the time of this writing, in the process of releasing version 2.0 of their RealSecure product. We were unable to test this version, as it was not generally available at the time of this writing.

B. NetRanger

NetRanger is an automated network intrusion detection system by WheelGroup Corporation. NetRanger interfaces a passive network monitor with a packet filtering router, creating a ?reactive? IDS; the ability to respond in real-time to attacks by ?shunning? addresses (filtering them at the router) is a major feature of the system.

We had very limited access to the NetRanger system. The hardware requirement (and price) of this system made it impractical for us to obtain our own copy for testing; rather, we relied on the cooperation of an organization already using the product. Because of this, our tests were performed over the global Internet, and we were only able to perform certain tests (due to timing issues). Our test results for NetRanger still showed major problems. Like all the systems we reviewed, NetRanger (in the version we tested) is completely unable to handle fragmented IP packets. An attacker can evade NetRanger completely by fragmenting all her packets. NetRanger did not detect data in a SYN packet, so an attacker can evade many of NetRanger?s checks by putting crucial data in her initial SYN packet.

We were able to desynchronize NetRanger from our connections by preceding the connection with fake, randomly sequenced data. NetRanger failed to detect attacks in a connection, using the same parameters, that followed this. Finally, NetRanger was vulnerable to one of our insertion attacks (it doesn?t appear to validate TCP checksums). NetRanger did appear to reliably verify IP checksums.

A large portion of our tests were not performed against NetRanger. We can?t conjecture as to whether NetRanger is vulnerable to the attacks those tests measure. Hopefully, these tests can be run against NetRanger in the future. WheelGroup Corporation claims to have addressed many of our problems in a newer version of the product.

C. SessionWall-3

SessionWall is an automated network intrusion detection system by AbirNet. We tested the Windows NT version of AbirNet SessionWall-3. Although AbirNet appears to have real-time connection playback capabilities, we were unable to get it working in the evaluation copy we used for our tests.

Of all the ID systems we tested, AbirNet appeared have the most strict network monitoring system. SessionWall-3 did not record data for connections that weren?t marked by a three-way handshake. It stopped recording when a connection was torn down with an RST packet. This prevented our desynchronization tests from disrupting the system; however, the strictness of SessionWall?s implementation may be attackable as well (insertion of RST packets, for instance, could be used for evasion attacks).

SessionWall validated IP and TCP checksums, and did not accept data without the ACK bit set. It did not appear to wait for acknowledgment before accepting data, however.

We were able to desynchronize SessionWall-3 from our connections by injecting fake SYN packets into our stream; the SYNs were ignored by the endpoint, but SessionWall apparently resynchronized to them and lost pattern matching state. Like NetRanger, SessionWall-3 also failed to detect data in SYN packets. SessionWall incorrectly reassembles overlapping TCP segments, and is thus vulnerable to an evasion attack.

Like all the systems we reviewed, SessionWall-3 is completely unable to handle fragmented IP packets. An attacker can evade SessionWall-3 completely by fragmenting all her packets.

D. Network Flight Recorder

NFR is a network monitoring engine by Network Flight Recorder. Unlike the other systems we tested, NFR is not an automated network intrusion detection system; rather, NFR provides a network monitoring component that can be used in a variety of applications. NFR is user-programmable and extensible, and available in source code.

We reviewed NFR because its engine could be used as an automated network intrusion detection system. This is not the intent of the product, and our results do not have significant bearing on NFR?s non-security uses. Additionally, because NFR is completely programmable (the product is really an interpreter for a network programming language), it is possible for users of the product to address many of the concerns we bring up in our paper without modifying the underlying engine.

NFR was able to handle IP fragmentation; we verified this in an independent testing process that confirmed NFR?s ability to reassemble a fragmented UDP packet (we were able to perform this test because of NFR?s available source code). Unfortunately, NFR was unable to handle pattern matching in a TCP stream that was fragmented; it therefore ?failed? all of our fragmentation tests. NFR, in version beta-2, was vulnerable to all our insertion attack tests. It did not verify IP or TCP checksums, and accepted data without the SYN bit set. NFR detects data in SYN packets.

NFR does not immediately tear down a connection when an RST is seen. We were able to desynchronize NFR by sending spurious SYN packets in our connections, but were unable to successfully desynchronize it with any of our other tests. NFR did not correctly reassemble overlapping TCP segments, and is thus vulnerable to an evasion attack.

V. Discussion

Our tests revealed serious flaws in each system we examined. Every IDS we examined could be completely eluded by a savvy attacker. We have no reason to believe that skilled attackers on the Internet don?t already know and aren?t already exploiting this fact. Many of the problems we tested for were minor, and easily fixed. The very presence of such vulnerabilities leads us to believe that ID systems have not adequately been tested.

The ability to forge packets, and the ability to ?insert? packets into ID systems, makes it fairly trivial for an attacker to forge ?attacks? from arbitrary addresses. The ability to react to attacks by reconfiguring packet filters was a major advertised feature of many of the systems we tested. Our work shows that this capability can be leveraged against the system operators by an attacker; these facilities may do more harm than good.

Regardless of whether a problem is obviously solvable or not, it?s presence is significant to both IDS users and designers. Users need to understand that the manner they configure the IDS (and their network) has a very real impact on the security of the system, and on the availability of their network. Designers need to understand the basic problems we identify with packet capture, and must begin testing their systems more rigorously.

There are several things that can be done by IDS operators to enhance the overall security of the system as a whole. Additionally, IDS operators need to understand that the outputs of their systems must be read critically; ?session playback? data may not represent what?s actually occurring in a session, and the source addresses of attacks may not be valid at all.

One critically important step that must be taken before an IDS can be effectively used is to set up ?spoof protection? filters, which prevent attackers on the Internet from injecting packets with addresses forged to look like ?internal? systems into the network. Packet forgery can completely confuse network intrusion detection systems.

It?s important to understand that an attacker that successfully breaks into an IDS-protected network probably controls the IDS. An attacker with direct access to the network she?s attacking can forge valid-looking responses from systems she?s attacking. These forged packets can prevent the IDS from obtaining any coherent picture of what?s happening on the network. As soon as an IDS records a ?successful? attack on the network, administrators should assume that all bets are off, and further attacks are occurring without the knowledge of the IDS.

An attacker can fool ?session playback? facilities into playing arbitrary data back to the operators. Session playback may not accurately represent what?s happening inside of a connection. Real-time connection monitoring (when based on an ID system?s reconstruction of what?s happening in a TCP stream, rather than on printing and recording every packet on the wire) should not be trusted.

Finally, it?s of critical importance that ID system operators do not configure their system to ?react? to arbitrary attacks. An attacker can easily deny service to the entire network by triggering these reactions with faked packets; ID systems that reconfigure router filters are particularly vulnerable to this, as an attacker can forge attacks that appear to come from important sites (like DNS servers), and cause the IDS to cut off connectivity to these sites.