Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page. To copyotherwise, or republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.CCS’03, October 27-31, 2003, Washington, DC, USA.Copyright 2003 Sun Microsystems, Inc. All rights reserved.1-58113-738-9/03/0010...$5.00.DoS Protection for UDP-Based ProtocolsCharlie [email protected] PerlmanSun Microsystems [email protected] SommerfeldSun [email protected] IP packet reassembly requires resources, a denial of serviceattack can be mounted by swamping a receiver with IP fragments.In this paper we argue how this attack need not affect protocolsthat do not rely on IP fragmentation, and argue how mostprotocols, e.g., those that run on top of TCP, can avoid the need forfragmentation. However, protocols such as IPsec’s IKE protocol,which both runs on top of UDP and requires sending large packets,depend on IP packet reassembly. Photuris, an early proposal forIKE, introduced the concept of a stateless cookie, intended forDoS protection. However, the stateless cookie mechanism cannotprotect against a DoS attack unless the receiver can successfullyreceive the cookie, which it will not be able to do if reassemblyresources are exhausted. Thus, without additional design and/orimplementation defenses, an attacker can successfully, through afragmentation attack, prevent legitimate IKE handshakes fromcompleting. Defense against this attack requires both protocoldesign and implementation defenses. The IKEv2 protocol wasdesigned to make it easy to design a defensive implementation.This paper explains the defense strategy designed into the IKEv2protocol, along with the additional needed implementationmechanisms. It also describes and contrasts several other potentialstrategies that could work for similar UDP-based protocols.Categories and Subject DescriptorsC.2.0 [Computer-Communication Networks]: General---Security and Protection.General TermsAlgorithms, Performance, Design, Security, Reliability.KeywordsDoS, IPsec, IKE, fragmentation, protocol design, networksecurity, denial of service, buffer exhaustion.1. INTRODUCTIONOne of the major concerns in the design of IPsec keymanagement protocols has been to make them resistant todenial of service (DoS) attacks. Since IPsec implementationsare deployed in environments that are assumed to be hostile,they must be able to establish security associations even whileunder attack.The concept of stateless cookies as a protection against certainclasses of DoS (denial of service) attacks originated withPhoturis [8], an early key management protocol for IPsec. Thepurpose of stateless cookies is to defend against attackers thatsend traffic from fake source addresses, exhausting stateand/or computation resources at the victim node. The reasonfor the attacker sending from forged IP addresses is twofold: toavoid prosecution for mounting a denial of service attack, andto make it difficult for a firewall to screen out traffic from theattacker. In the stateless cookie design, when a node “Bob”receives a connection initiation request from a node “Alice”,Bob creates a number (called the “cookie”), unpredictable toAlice, returns that cookie to the IP address in the sourceaddress field in the IP header of the received packet, and keepsno state and does no additional computation. If the cookie isstateless, it must be recomputable by Bob, and is typically afunction of the source IP address from the received packet anda secret known only to Bob. If Bob asks Alice to return acookie before he is willing to consume significant resources,Alice must try again, this time returning the cookie. When Bobreceives a connect initiate request with a cookie, Bobcomputes whether that is the cookie he would have sent to thatIP address. If so, he is willing to devote state and computationto the connection from that IP address.2. FRAGMENTATION ATTACKS IN UDP-BASED PROTOCOLSAlthough in theory stateless cookies allow Bob not to devotestate or significant computation until he is assured that Alicecan receive at the address she claims to be coming from, inpractice there is a DoS threat that a straightforwardimplementation of a UDP-based protocol will be vulnerable to,if (like IKE) it sends large packets and depends on IPfragmentation. An attacker can take advantage of the fact thatIP fragment reassembly requires storing packet fragments ofpartially reassembled IP packets, which consumes memoryresources on the victim. Since the kernel reassembly queue islimited in size this sort of flooding will prevent legitimatepackets from being reassembled.It was a decision in the design of IKE [3] to run on top of UDP,to avoid the DoS attacks on TCP. Another decision was tokeep IKE simple and rely on IP fragmentation in order to senda large message. IKE messages can be large because they2contain structures such as certificates. The proposedsuccessors to IKE, including JFK [1], and IKEv2 [4], also havemade the decision to run on UDP and rely on IP fragmentationfor delivery of large messages.Protocols that run on top of TCP are not as vulnerable to thefragmentation attack, because TCP can avoid IP-levelfragmentation. TCP can do this because it is connection-oriented, and because it is designed so that it can send data inchunk sizes independent of the size of application messages.However TCP itself is prone to various DoS attacks, and thedecision to run on top of UDP was made with the intention tomake the protocol more robust against DoS attacks.But IKE’s decision to send large messages, and use UDP,make it particularly vulnerable to the fragmentation attackdescribed in this paper. This paper explains various strategiesthat a redesigned IKE, together with a DoS-resistantimplementation of the IP stack, can employ to defend againstsuch attacks. These strategies would be applicable to protocolssimilar to IKE which send large messages on UDP.The attack has not been addressed in the literature (except for theIKEv2-related internet drafts). In [7] many types of DoS attacksare discussed, but the attack in this paper is not included. Without adefense against this attack it is easy for an attacker to send IPfragments,