A copy of original Thesis in full (pdf file) - SERC

AN EXTENSION OF MULTI LAYER IPSECFOR SUPPORTING DYNAMIC QOS ANDSECURITY REQUIREMENTSA THESISSubmitted byArnab Kundufor the award of the degreeofMASTER OF SCIENCE (ENGG.)Supercomputer Education and Research CentreIndian Institute of ScienceBangaloreFebruary 2010

ACKNOWLEDGEMENTSI express my sincere gratitude to Professor N. Balakrishnan for his guidance andspecifically, for his emphasis on systematic approach, details and rigor in the processof research. I cherish the discussions I had with him and thank him for his advice andall the support throughout my research work.I am grateful to Dr. G. Athithan for his guidance and especially, for his constantencouragement for developing a deep passion towards research and emphasis on hardwork and rigorous experimentation. I thank him for his efforts in improvement of mywriting skills.I would also like to thank the faculty members of the institute with whom I hadfruitful interactions during my stay in the institute.I thank Mr. V. S. Mahalingam, Director CAIR, for giving me the opportunity andallowing me to use the facilities of the laboratory to carry out research.I thank the staff of the Supercomputer Education and Research Centre (SERC)and also the authorities of the Indian Institute of Science, Bangalore for their timelyhelp.I thank my colleagues in CAIR, with special thanks to Anupam for his supportand encouragement.

I am indebted to my parents and my wife for the care, affection and love they haveshowered on me and for their concern and support.- Arnab Kunduii

ABSTRACTKeywords: IP Security; Intermediate QoS and security service; Multi Layer IPSec;The IPSec protocol is a standard for providing confidentiality, integrity and originauthentication for communications in Internet scenario. However, the end-to-end protectionmodel of IPSec is in direct conflict with intermediate routers providing QoSand other performance enhancement and security related services.The problem isthat these services would not be able to access relevant information from the upperlayer headers that arrive fully encrypted from the source. The Multi Layer IPSec (ML-IPSec) protocol solves this problem partially by providing different levels of securityto different segments of IP datagrams. For this purpose, it uses manual configurationof security parameters at all intermediate nodes.As a consequence, this approachis not suitable for Internet scenario and mobile networking environments, where therequirement of prior knowledge of the intermediate nodes involved can not be metalways. The ML-IPSec also ignores the presence of IP datagrams with variable sizeIP and TCP headers, which may be required for the proper working of applicationsat the intermediate nodes.In our research, we propose an extension to the ML-IPSec protocol to address these two issues. By adding a dynamic key sharing protocolto the ML-IPSec, access to the upper layer headers is provided at the intermediatenodes on need basis. The extension also proposes to use the explicit or implicit headerlength field information for accommodating IP datagrams with varying header lengths.The extended ML-IPSec is implemented in the Linux kernel of IPSec processing, andexperimental validation involving standard intermediate applications is carried out.

The outcome of the experimental validation proves that the proposed extension toML-IPSec, the Dynamic Multi Layer IPSec, provides the functionality of on-the-flysharing of the required upper layer information with the intermediate nodes withoutany major overheads on packet size and processing delay.iv

TABLE OF CONTENTSAcknowledgementsAbstractList of TablesList of FiguresAbbreviationsiiiiviiiixxiii1 INTRODUCTION 11.1 Network Security Services . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Cryptographic Techniques . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Network Security and IPSec . . . . . . . . . . . . . . . . . . . . . . . . 51.4 Objectives of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . 91.5 Organization of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . 92 COMMUNICATION SECURITY PROTOCOLS 112.1 Layering at the TCP/IP protocol suite . . . . . . . . . . . . . . . . . . 112.2 Protocol layering and Communication Security Protocols . . . . . . . . 132.2.1 Application Level Security . . . . . . . . . . . . . . . . . . . . . 152.2.2 End-System level security . . . . . . . . . . . . . . . . . . . . . 162.2.3 Subnetwork Level Security . . . . . . . . . . . . . . . . . . . . . 172.2.4 Direct-Link level security . . . . . . . . . . . . . . . . . . . . . . 182.3 The IPSec protocol suite . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3.1 Security Association and Policies in IPSec . . . . . . . . . . . . 192.3.2 Authentication Header (AH) . . . . . . . . . . . . . . . . . . . . 212.3.3 Encapsulating Security Payload (ESP) . . . . . . . . . . . . . . 24

2.3.4 Mutual Authentication and key sharing . . . . . . . . . . . . . . 272.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 Problem of QoS and Network access control with IPSec 313.1 QoS Processing at intermediate nodes . . . . . . . . . . . . . . . . . . . 313.2 Security services at intermediate nodes . . . . . . . . . . . . . . . . . . 333.3 IPSec and intermediate device interoperability . . . . . . . . . . . . . . 343.3.1 Replacing IPSec with a transport-layer security mechanism . . . 343.3.2 Tunneling one security protocol within another . . . . . . . . . 353.3.3 Using a transport-friendly ESP format . . . . . . . . . . . . . . 363.3.4 Multi Layer IPSec (ML-IPSec) . . . . . . . . . . . . . . . . . . . 363.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404 An Extension to ML-IPSec protocol - Dynamic Multi Layer IPSec 414.1 Elements of DML-IPSec . . . . . . . . . . . . . . . . . . . . . . . . . . 424.1.1 DML-IPSec Zone . . . . . . . . . . . . . . . . . . . . . . . . . . 424.1.2 DML-IPSec Packets . . . . . . . . . . . . . . . . . . . . . . . . . 464.1.3 Security Associations . . . . . . . . . . . . . . . . . . . . . . . . 474.2 The DML-IPSec Protocol . . . . . . . . . . . . . . . . . . . . . . . . . 494.2.1 DML-IPSec Processing . . . . . . . . . . . . . . . . . . . . . . . 494.2.2 Key sharing Protocol . . . . . . . . . . . . . . . . . . . . . . . . 554.2.3 Handling of variable Zone sizes . . . . . . . . . . . . . . . . . . 654.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 665 Design and Implementation of DML-IPSec in Linux Environment 685.1 Implementation aspects of basic IPSec . . . . . . . . . . . . . . . . . . 695.1.1 Implementation of Protocols . . . . . . . . . . . . . . . . . . . . 695.1.2 User space utility . . . . . . . . . . . . . . . . . . . . . . . . . . 695.2 Implementation of the DML-IPSec protocol . . . . . . . . . . . . . . . 71vi

5.2.1 Implementation of Zones . . . . . . . . . . . . . . . . . . . . . 715.2.2 Implementation of security association . . . . . . . . . . . . . . 725.2.3 Implementation of the datagram processing at the endpoints . 745.2.4 DML-IPSec processing at the intermediate nodes . . . . . . . . 755.2.5 Security Databases configuration utility . . . . . . . . . . . . . . 775.2.6 Key Sharing Protocol . . . . . . . . . . . . . . . . . . . . . . . 795.3 Experimental validation of the DML-IPSec protocol . . . . . . . . . . . 815.3.1 Test bed setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 845.3.2 Validation with respect to intermediate services . . . . . . . . . 845.3.3 Overheads of DML-IPSec . . . . . . . . . . . . . . . . . . . . . 935.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 986 SUMMARY AND CONCLUSION 1006.1 Contributions of the work . . . . . . . . . . . . . . . . . . . . . . . . . 1026.2 Scope for further research . . . . . . . . . . . . . . . . . . . . . . . . . 103Bibliography 104vii

LIST OF TABLES2.1 The four Layers of the TCP/IP Protocol Suite . . . . . . . . . . . . . . . . 115.1 Applications, their use and the headers of the IP datagram accessed by them 825.2 Working of different intermediate application in different zone mappings . . 835.3 Packet Length Comparisons (Bytes) between IPSec and DML-IPSec withtwo zones for AH and ESP in transport and tunnel modes. . . . . . . . . . 945.4 Packet Length Overhead (Bytes) in IPSec and DML-IPSec protocol with twozones over standard IP protocol. The numbers in square brackets denote themin and max values possible for that particular case . . . . . . . . . . . . . 95

LIST OF FIGURES2.1 Processing of a HTTP Packet by different layers . . . . . . . . . . . . . . . 132.2 (i) Format of AH Header (ii) AH protocol in transport mode(iii) AH protocolin tunnel mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.3 IPSec in both modes of operations: (i) IPSec in tunnel mode of operationbetween IPSec Gateways Gateway1 and Gateway2. (ii) IPSec in transportmode of operation between clients C11 and C21. . . . . . . . . . . . . . . . 242.4 (i)ESP Header format (ii)ESP protocol in transport mode (ii)ESP protocolin tunnel mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.1 SSL Record Protocol Operation . . . . . . . . . . . . . . . . . . . . . . . 353.2 Zone and Zone map in ML-IPSec . . . . . . . . . . . . . . . . . . . . . . 373.3 ML-IPSec Protocol Operation . . . . . . . . . . . . . . . . . . . . . . . . 383.4 ML-IPSec Packet Format (i) Authentication Header (ii) Encapsulating SecurityPayload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.1 Different zone mappings of a HTTP Packet . . . . . . . . . . . . . . . . . 444.2 Zone Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454.3 DML-IPSec Packet Format (i) Authentication Header (ii) Encapsulating SecurityPayload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.4 Security association for DML-IPSec . . . . . . . . . . . . . . . . . . . . . 484.5 Outbound Processing at DML-IPSec endpoint . . . . . . . . . . . . . . . . 514.6 Inbound Processing at DML-IPSec endpoint . . . . . . . . . . . . . . . . . 534.7 Partial Inbound DML-IPSec Processing at intermediate node . . . . . . . . 544.8 Partial Outbound DML-IPSec Processing at intermediate node . . . . . . . 55

4.9 Key Sharing Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564.10 Zone map and Cryptographic message format during Key Sharing Protocol . 614.11 Key Sharing Protocol Example . . . . . . . . . . . . . . . . . . . . . . . . 645.1 SPD and SA in Linux Implementation of IPSec . . . . . . . . . . . . . . . 705.2 Illustration of interaction between user space utilities and Kernel . . . . . . 715.3 Zone structure and Zone specific DML-IPSec ESP SA . . . . . . . . . . . 725.4 SPD and SA structures in the DML-IPSec . . . . . . . . . . . . . . . . . . 735.5 Illustration of interaction between user space utilities and Kernel in DML-IPSec 785.6 High level Interaction of DML-IPSec implementation . . . . . . . . . . . . 815.7 Test bed Set up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 845.8 ESP packet corresponding to TCP request at the forwarding chain of firewallat IN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 865.9 Partially decrypted TCP packet corresponding to FTP request at the forwardingchain of firewall at IN . . . . . . . . . . . . . . . . . . . . . . . . 875.10 ESP packet corresponding to Web request at the forwarding chain of firewallat IN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 885.11 Partially decrypted TCP packet corresponding to Web request at the forwardingchain of firewall at IN . . . . . . . . . . . . . . . . . . . . . . . . 885.12 Ping Traceroute Output . . . . . . . . . . . . . . . . . . . . . . . . . . . 895.13 Wireshark output for the Ping Application on eth1 interface . . . . . . . . 905.14 Layout of captured ESP Packet on eth0 interface of IN . . . . . . . . . . . 915.15 Layout of partially decrypted ESP packet on eth1 interface of IN . . . . . . 92x

5.16 RTT of ping obtained in different communication scenarios between GW1and GW2. (1) Normal IP communication (2) Native IPSec ESP tunnel mode(3) ML-IPSec ESP tunnel mode without any type I zone (4) ML-IPSec ESPtunnel mode with one type I zone and one type II zone of static length (5)DML-IPSec ESP tunnel mode with one type I zone and one type II zone ofvariable length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 965.17 Comparison of RTT in presence of intermediate processing obtained in differentcommunication scenarios between GW1 and GW2. (1) ML-IPSec ESPtunnel mode with one type I zone and one type II zone of static length (2)DML-IPSec ESP tunnel mode with one type I zone and one type II zone ofvariable length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99xi

ABBREVIATIONSIPIPSecTCPTCP/IPUDPICMPIGMPATMHTTPFTPIETFRFCSMIMEPGPTLSMPLSESPAHIKEPEPQoSML-IPSecDML-IPSecOSI- Internet Protocol- Interney Protocol Secutiry- Transmission Control Protocol- Transmission Control Protocol/Internet Protocol- User Datagram Protocol- Internet Control Message Protocol- Internet Group Management Protocol- Asynchronous Transfer Mode- Hyper Text Transfer Protocol- File Transfer Protocol- Internet Engineering Task Force- Request For Comments- Secure Multipurpose Internet Mail Extensions- Pretty Good Privacy- Transport Layer Security- Multi Protocol Level Switching- Encapsulating Security Payload- Authentication Header- Internet Key Exchange- Performance Enhancement Proxy- Quality of Service- Multi layer IPSec- Dynamic Multi Layer IPSec- Open System Interfacexiii

SACSASADSPISPDICVIVRSADESAESCBCISAKMPTF-ESPEOPDHPKI- Security Association- Composite Security Association- Security Association Database- Security Parameter Index- Security Policy Database- Integrity Check Value- Initialization Vector- Rivest Shamir Adleman- Data Encryption Standard- Advanced Encryption Standard- Cipher Block Chaining- Internet Security Association and Key Management Protocol- Transport Friendly ESP- End Of Packet- Diffie Hellman- Public Key Infrastructurexiv

CHAPTER 1INTRODUCTIONToday governments, military, corporations, financial institutions and others exchangea great deal of confidential information using the Internet and Intranets. Protectionof such confidential information, ensuring data integrity and data origin authenticityare of paramount importance. The Internet was however originally designed forresearch and educational purpose, not for commercial applications [1]. As such, itssecurity relied on mutual trust and openness of the users, as well as their understandingof conduct considered appropriate on the network. As the Internet grew, the usercommunity expanded, and the existing security framework was found inadequate formodern day applications. This was mainly due to the lack of security services in theTCP/IP (Transmission Control Protocol/ Internet Protocol) [2–5] protocol suite. Thebroadcast nature of the lower layer protocols, absence of authentication mechanism atthe entire TCP/IP protocol suite, lack of protection to packet contents and easily predictableTCP sequence numbers of some TCP implementations are among the basicsecurity flaws of the TCP/IP protocol suite [6–9].In the commercial usage of Internet and TCP/IP, these problems manifest themselvesin a number of ways [10] : Eavesdropping attacks on a network can result inthe theft of private information such as credit card numbers and customer accountnumbers. Similarly, such attacks can result in the theft of services normally limitedto subscribers, such as information-based products. Data modification attacks can beused to modify the contents of certain transactions e.g., changing the payee on anelectronic check or changing the amount being transferred to a bank account. Spoofing1

attacks can be used to enable one party to masquerade as another party. Repudiationof transactions can cause major problems with billing systems and transactionprocessing agreements.1.1 NETWORK SECURITY SERVICESIn due course of time, safeguards were brought about to address the security relatedissues due to the inherent problems of the TCP/IP protocol suite. In the computercommunication context, the main security safeguards are referred to as security services[11, 12] . There are five generic security services :• Authentication Service: This service provides assurance of the identity of someentity (a person or a system).In other words, when some entity claims tohave a particular identity, an authentication service will provide a mechanismto validate this claim.Authentication is one of the most important securityservices, because all other security services depend upon it to some extent.Authentication is the means of countering the threat of masquerade which candirectly lead to compromise of any of the security objectives.• Access Control Service: This service protects against unauthorized use or manipulationof resource.Access control is a means of enforcing authorization.Access control directly contributes to achieving the security goals of confidentiality,integrity, availability and legitimate use. Authorization decisions governwhich initiator may access which targets for which purposes and under whatconditions. These decisions are reflected in an access control policy. Accessrequests are filtered through an access control mechanism which enforces theaccess control policy. Another aspect of access control is preventing sensitiveinformation from being transmitted through environments where it might be atrisk.2

• Confidentiality Service: This service protects against information being disclosedor revealed to unauthorized entities. Achieving a confidentiality objectiveideally requires protecting against disclosure of information through any informationchannel. In computer communications security, there are two types ofconfidentiality services. A data confidentiality service makes it infeasible to deducesensitive information from the content or size of a given data item. A trafficflow confidentiality service makes it infeasible to deduce sensitive informationby observing the traffic flows.• Data Integrity service:This service complements access control service andprotects against data being manipulated without authorization. An importantcharacteristics of a data integrity service is its granularity. There are three importantcases of authentication service. The first one, called connection integrityservice, applies to all data transmitted on a connection. The second one, calledconnectionless integrity service, applies to all data comprising one connectionlessitem. The third is a selective field integrity service, which applies only tosome field of a data unit.• Non-repudiation: This service protects against one party to a communicationexchange later falsely denying that the exchange occurred. Non-repudiation isfundamentally different from all other security services. Its primary purpose isto protect users of communications against threats from other legitimate users;rather than from unknown attackers or eavesdroppers. One important point tonote is that it does not eliminate repudiation. Rather it ensures the availabilityof irrefutable evidence to support the speedy resolution of any disagreementrelated to repudiation.A robust security solution for transaction processing should provide all the fivesecurity services. The security services are implemented using the cryptographic tech-3

niques as described next.1.2 CRYPTOGRAPHIC TECHNIQUESCryptographic techniques, such as encryption and digital signatures, are the buildingblocks in the implementation of all security services [12–15]. The most basic buildingblock is called a cryptographic system or cryptosystem. A cryptosystem defines a pairof data transformations. The first transformation is applied to an ordinary data item,known as plaintext and generates a corresponding unintelligible data item called ciphertext.The second transformation, applied to ciphertext, results in regeneration ofthe original plaintext. An encryption transformation uses as input both plaintext dataand an independent value known as encryption key. Similarly, the decryption transformationuses a decryption key along with the ciphertext to regenerate the plaintext.The transformations are most commonly called encryption and decryption respectively.There are two basic types of cryptosystems: symmetric cryptosystems and asymmetriccryptosystems. Symmetric cryptosystems are characterized by the fact that the samekey is used in encryption and decryption transformations. In contrast to symmetriccryptosystems, asymmetric cryptosystems use complementary pairs of keys for encryptionand decryption transformations. One key, the private key is kept secret like thesecret key in a symmetric cryptosystem. The other key, the public key , does not needto be kept secret. This two key approach can simplify key management by minimizingthe number of keys that need to be managed and stored in the network. The keydistribution system is also much simpler as it can use unprotected medium for thedistribution of the public keys. Data integrity and/or data origin authentication fora message can be provided as follows. The originator of the message generates, usingall the data bits of the message contents and a secret key, an Integrity Check Valuewhich is transmitted along with the message. The message recipient checks that the4

eceived message content and Integrity Check Values are consistent before acceptingthe message. A digital signature can be considered as a special case of a IntegrityCheck Value.The digital signature may need to be used to resolve a dispute betweenthe originator and recipient of a message. Symmetric encryption based or keyedhash algorithm based approach is usually inadequate for this purpose. Asymmetriccryptosystems provide more powerful digital signatures.1.3 NETWORK SECURITY AND IPSECIn order to provide the five fundamental security services, a number of cryptographicprotocols have been proposed. While these protocols are similar in the services theyprovide and in the cryptographic techniques they use, they vary in the manner in whichthey provide the services and in their locations with respect to the rest of the TCP/IPprotocol stack. The issue of where within the protocol stack to provide security iscontentious. Proponents of placing security lower in the stack argue that lower-layersecurity solutions can be implemented transparently to end users and application developers.Proponents of higher-layer solutions argue that application-layer solutionsallow application-specific security services (e.g., protecting selected fields within a protocol,or individual methods, as in HTTPS) [16].There exist protocols at different layers of TCP/IP protocol stack to provide thesesecurity services. Application level encryption viz. PGP [17,18] for secure mail transfer,TLS [19] based VPN for secure TCP communication, IPSec for providing IP layersecurity [20,21], MPLS based security [22] at data link layer are among these securitysolutions. Due to scalability and wide acceptance of the IP protocol and its applicationindependent character, the IPSec protocol has become a standard for providingInternet security.IPSec [20,23,24] provides security services at the IP layer by enabling a system to5

select required security protocols, determine the cryptographic algorithms to use forthe services and put in place any cryptographic keys required to provide the securityservices. Two protocols are used to provide the security: an authentication protocol designedby Authentication Header (AH) [23] and a combined encryption/authenticationprotocol designed by Encapsulating Security Protocol (ESP) [24]. The AH providesthe following security services: Connectionless Integrity, Data Origin Authenticationand Replay Protection. The ESP protocol provides confidentiality of data as well asof traffic flow on top of the services provided by the AH. IPSec also uses a separateprotocol, called Internet Key Exchange (IKE) [25–27] for establishing cryptographicparameters. Both AH and ESP support two modes of use : transport and tunnel mode.Transport mode provides security protections for upper layer protocols, whereas tunnelmode provides protection to the entire IP packet. This is achieved by encapsulatingthe entire IP packet inside a new IP packet. The cryptographic keys used in the IPSecsecurity services are shared only between the IPSec endpoints. All other intermediatenodes in a network, whether they are legitimate routers or malicious eavesdroppers canonly see the IP header when encryption is provided through ESP protocol. Even inthe case of AH, the payload of IP datagram cannot be tampered by any intermediatenode without being detected at the IPSec endpoint.This end-to-end model of security restricts performance enhancement and securityrelated operations of many intermediate networking and security devices as they can’taccess/modify the original IP header (in case of tunnel mode) and upper layer protocolheaders. These intermediate devices include routers providing Quality of Service(QoS), Performance Enhancement Proxies (PEP), Application level Proxy devices,Packet filtering firewalls etc [28, 29].The interoperability problem between IPSec and intermediate devices has beenaddressed in literature [30].Using transport layer security (TLS), splitting single6

IPSec tunnel into multiple tunnels, Multi layer IPSec (ML-IPSec) are a few of theproposed solutions. We observe that ML-IPSec provides better solution compared toother alternatives as it solves the interoperability issue without violating the end-toendsecurity paradigm for the data portion, unlike the case of splitted tunnel. TheML-IPSec also provides security to the entire IP datagram as opposed to the solutionlike TLS or Transport Friendly ESP (TF-ESP) where some important header fieldsare kept out of the scope of security services. The ML-IPSec protocol [31,32] providesa novel approach for solving these interoperability issues.The ML-IPSec protocolpartitions an entire IP packet into multiple zones and provides different security todifferent zones of the packet. The cryptographic information of the zones spanningover the upper layer protocol headers are made available to the legitimate intermediatenodes for their processing. The cryptographic keys of the zones spanning over dataportion are shared only among the ML-IPSec endpoints, thus retaining the end-to-endsecurity of original IPSec protocol for the data portion of an IP packet.Though the ML-IPSec protocol works well in a closed environment, the requirementof sharing security parameters and the requirement of prefixed zone sizes becomedifficult in large scale networks and in distributed environments. The main issues ofthe ML-IPSec protocol in this regard are as follows:1. Routing infrastructure has to be known prior to implementation and use, as eachrouter that requires access to higher layer protocol data will need to be manuallyconfigured before communication can commence. In some cases this will breakthe inherent reliability built into the Internet and it will not function correctlyover multiple routes unless the appropriate router in each separate route isknown and configured in advance. While this may not be a large problem ina geo-stationary satellite environment, where the satellite link terminates ata single terrestrial gateway, it could pose problems in a mobile or distributed7

environment where many base stations may be in use.2. The requirement of manual configuration at all the intermediate nodes makesthe assumption that ML-IPSec endpoints have access to and control over theintermediate routers that require partial access to the network payload. However,the ML-IPSec endpoints may not necessarily be trusted by an intermediaterouter providing QoS based service or an intermediate security device for manualconfiguration without prior arrangement.3. Security zones are designated as static byte ranges and need to be specifiedbefore communication ever commences.As a result, the exact length of theheaders need to be known prior to implementation. ML-IPSec will not functioncorrectly if the end nodes change the use of IP or TCP options, as the offsets andlengths of the headers will change, causing the zone mapping to fail. This willalso be an issue when ML-IPSec is used with multiple clients, each of which mayuse different TCP options depending on the capabilities and configuration of theoperating systems TCP/IP stack. This makes ML-IPSec extremely inflexible,particularly when it is used in tunnel mode.4. The zone mapping proposed in ML-IPSec is static in nature. This forces oneto configure the zone mapping before the commencement of the communication.It restricts the protocol from dynamically changing the zone mapping forproviding access to intermediate nodes without terminating the existing ML-IPSec communication. The ML-IPSec endpoints can off course, configure thezone mapping with maximum number of zones. This will lead to unnecessaryoverhead that increases with the number of zones. Again, static zone mappingcould pose problems in a mobile or distributed environment, where communicationpaths may change.8

1.4 OBJECTIVES OF THE THESISThis work aims at improving the ML-IPSec protocol for providing multilayer IPSeccapabilities without the need of static configuration and for IP packets with variablesize header lengths.The thesis first aims at providing dynamic configurability and cryptographic informationsharing between the IPSec endpoints and intermediate devices. This willenable intermediate nodes to access required header portions of IPSec packets of anexisting IPSec communication on need basis, rather than depending on the static configurationbefore the commencement of the communication. Another aim of the thesisis to accommodate TCP/IP packets with variable length headers. Through these enhancements,the thesis aims to address the issues of using ML-IPSec in mobile networkenvironments.The scope of the work is limited to Internet Protocol Version 4 (IPv4).1.5 ORGANIZATION OF THE THESISChapter 2 gives an overview of protocols to provide communication security at differentlayers of the TCP/IP protocol suite. It also describes the working of different componentsof the IPSec protocol suite viz. AH, ESP and IKE in detail. Chapter 3 providesa study on the interoperability issues between IPSec and intermediate devices. It alsodiscusses about different solutions including the ML-IPSec protocol proposed in theliterature to address these interoperability issues. Chapter 4 presents the extension tothe ML-IPSec protocol, called Dynamic ML-IPSec (DML-IPSec), for providing multilayer IPSec capabilities with the feature of on-the-fly access to upper layer protocolheaders to intermediate nodes. Chapter 5 gives the design and implementation detailsof DML-IPSec in Linux environment. It also provides experimental validationof the protocol. Chapter 6 summarizes the research work carried out as part of this9

thesis, highlights the contributions of the work and discusses the directions for furtherresearch.10

CHAPTER 2COMMUNICATION SECURITY PROTOCOLSThe TCP/IP protocol was conceived with the primary objective of communicationbetween two remote machines. The security aspects of communication became importantmuch later with the growth of Internet. As such, the TCP/IP protocol suite doesnot have provisions for the security features like confidentiality, authenticity and dataintegrity etc. as part of the network standard implementation. Research efforts haveresulted in several communication security protocols at different layers of the TCP/IPprotocol suite. In this chapter we discuss the different layers of the TCP/IP protocolsuite and follow it with a discussion on the communication security protocols at theselayers.2.1 LAYERING AT THE TCP/IP PROTOCOL SUITEThe TCP/IP protocol suite involves a whole family of protocols designated to performdifferent tasks and provide services [33]. These protocols belong to different layers ofthe TCP/IP protocol suite. The functionalities of the different layers are as follows:Table 2.1: The four Layers of the TCP/IP Protocol SuiteApplication Telnet, FTP,HTTP, etc.TransportNetworkLinkTCP and UDPIP,ICMP,IGMPEthernet,ATM etc.1. Application layer: It handles the details of a particular application. The standardimplementation provides support for different types of applications such11

as FTP, HTTP, Telnet and VoIP [34–36].2. Transport layer: It provides end-to-end flow of data, for the application layer. Itenables the communication between applications running in different terminalsand provides port numbers to different applications. The TCP and UDP aretwo major transport layer protocols.3. Internet layer: It handles the movement of packets around the network. Routingof packets takes place at this layer. It makes the communication among differentnetworks transparent.It allows the transport layer to see a unique virtualnetwork. The main protocol is IP, although there are other associated protocolsviz. ICMP, IGMP.4. Link layer: It manages the underlying network. This layer is network dependent.The Ethernet, ATM are typical examples in this layer.Table 2.1 shows the different protocols present at different layers of the TCP/IPprotocol suite. As the scope of thesis is restricted to providing security at IP Layer,only the Internet and Transport layer are relevant to us and are discussed.In the TCP/IP protocol suite, the IP protocol operating at the network layerprovides unreliable service. That is, it does its best job of moving a packet from itssource to its final destination, but there is no guarantee of the successful delivery of thepacket [33,37], TCP on the other hand, provides a reliable transport layer service overthe unreliable service of IP. The TCP protocol achieves this by performing timeoutand retransmission, in the event of the non-receipt of a packet within a stipulatedtime [33, 38–42].However, the UDP which is the other important transport layerprotocol offers connectionless and unreliable service.packet.Figure 2.1 depicts the processing of different layers of TCP/IP stack for a HTTP12

ApplicationDataHTTPHeaderHTTPPayloadApplicationLayerTCPLayerTCPHeaderHTTPHeaderHTTPPayloadIPLayerIP TCP HTTP HTTPHeader Header Header PayloadLinkLayerEthernetHeaderIPHeaderTCPHeaderHTTPHeaderHTTPPayloadEthernetTrailerFig. 2.1: Processing of a HTTP Packet by different layers2.2 PROTOCOL LAYERING AND COMMUNICATION SECURITY PRO-TOCOLSThe provisions of security protocols in a layered communication architecture, likeTCP/IP protocol suite, raise some important issues.Protocol layering results inencapsulation of one data item within another data item and connection is carriedout inside another connection. Hence there are major decisions to be made as to thelayer(s) at which data item or connection-based security should be provided. However,irrespective of the level at which the security is provided, the following basic servicesare essential:1. Authentication2. Authorization3. Confidentiality4. Integrity13

5. Nonrepudation6. Key managementFrom the communication protocol perspective, we can identify four different levelswhere the requirements of distinct security protocols arise. These four different levelsare listed below.1. Application level: Security protocols are application dependent.2. End-System level : Security protocols provide protection on end-to-end basis.3. Subnetwork level: Security protocols provide protection over a subnetwork whichis considered less trusted.4. Direct-link level: Security protocols provide protection over each link separately.The placement of security protocols at these different levels depends on severalfactors. In the following discussion, we list the major deciding factors for the placementof security protocols.• Traffic Multiplexing: The layering of the TCP/IP protocol suite provides multiplexingof several higher level services and applications through lesser numberof lower layer protocols. This traffic multiplexing provides some advantage inplacing security protocol at lower layers.• Application dependent Protection: If the security policy is dependent on userspecificinformation, providing security at higher layer is necessary.• Protocol Header Protection: Security protocol at any particular layer can’t providesecurity to it’s lower layer headers. Thus, in a scenario, where lower layerheaders carry sensitive information, placing the security protection at the lowerlayer is advantageous.14

• Number of protection points: Placing security at a very high level (i.e., applicationlevel) requires security to be implemented for every sensitive applicationin the system. Placing security at the lowest layer (i.e. direct-link level) requiresinstallation of security at every network link.Placing the security atmiddle layer (sub-network level) is a better choice as it requires significantlyless number of installation of the security protocol.• Route Knowledge: At a lower layer, more knowledge of different links and routesare available. In environments, where these information is useful for decidingon the security requirements, placing security at lower layer is important.Next, we discuss the properties of the communication security protocols at thedifferent levels and the corresponding layer of the TCP/IP protocol suite.2.2.1 Application Level SecurityThe application level security provides protection to the application layer of the TCP/IPprotocol suite. In many cases, the lower level alternatives provide better solutions withrespect to operational cost or lower number of equipment than the application levelsolutions. However, there exists many scenarios where application level security is theonly viable solution.One of which is the case, where the security service and thepolicies are application-specific either semantically or by virtue of being built into aparticular application protocol. For example, the security protocol for providing securefile transfer, has to know the complete working of the file transfer protocol. Similarly,if a security protocol has to provide security to some particular fields, as in the caseof electronic transaction where the PIN number has to be protected, the knowledgeof the application is mandatory for the working of the security protocol. Thus thesecurity protocol has to be implemented along with the application itself. Next is thecase, where security services traverse application relays. Here, the application involves15

more than two end systems, as in the case of secure mail transfer.A mail has totraverse multiple entities before reaching the final destination. The requirement is toprotect only the content of the mail while providing access to protocol headers forproper delivery of the mail.There are many popular application layer security protocols. Pretty Good Privacy(PGP),SMIME for secure mail transfer, HTTPS for secure HTTP connectionare among them. In all these application layer security protocols, authenticity andconfidentiality of the application layer are maintained.2.2.2 End-System level securityThe end-system level security protocols are required to address the scenario, where theend-systems are trusted but not the underlying network(s) and same security policy isenforced upon all communications involving a system regardless of applications. Thussecurity protections like confidentiality, authenticity can be applied on all traffic of thesystem for all applications.One point to note here is that the end-to-end communication security can beachieved by using the application level security protocol also. However, the followingfactors favor an end-system level solution over a comparable application level solution.• The end-system level solution makes security services transparent to the applications.• The end-system level solution provides better performance as it has the abilityto operate on larger data units.Management of security facilities, like keymanagement service, can be achieved on per-system basis rather than involvingindividual applications.• The upper layer protocol headers can be protected using end-system level security.16

In the TCP/IP protocol suite the end-system layer of security may be providedeither in the TCP layer or in the IP layer. The most popular protocol of providingsecurity at the TCP layer is Secure Socket Layer (SSL). The end-system layer securitycan be achieved by the IP layer security protocol, IPSec, also. The transport modeof the protocol provides the end-system level security at the IP layer. However, SSLhas gained more popularity for providing end-system level security at the TCP layer.The IPSec protocol suite in the tunnel mode of operation, has become a standard forproviding subnetwork layer security.2.2.3 Subnetwork Level SecuritySubnetwork level security provides communication security across one or more specificsubnetworks only, in contrast to the end-to-end security. In the following discussion,we describe the scenario, where providing subnetwork level of security is advantageousover the other levels of security.It is very common that subnetworks close to end-systems are trusted to the samelevel as that of the end-systems, because of their geographic proximity to the endsystems.Moreover, these sub-networks mostly comes under the same security administrationof the end-systems. This enables one to provide the subnetwork level securityat the boundary between trusted local subnetwork and the less trusted (external) subnetwork(i.e. at the gateway machines). In any network, the number of end-systemsusually far exceeds the number of network gateways. Thus providing the subnetworklevel system security at the gateway machines reduces the operational cost as well assecurity administration related complexities.Subnetwork level of security thus has an edge over end-system level security. InTCP/IP protocol suite, subnetwork layer security can be provided at the IP layer usingthe IPSec protocol in tunnel mode or at the data link layer in case of LAN. However,17

ecause of the wide acceptance of the IP protocol, IPSec in tunnel mode has becomea standard for providing gateway to gateway communication security [43].2.2.4 Direct-Link level securityThe appropriate scenarios for using direct-link level security are those with comparativelyfew untrusted links in an otherwise trusted network. Provision of security atthis level can be transparent to all higher layer protocols. Thus the security solutionis not tied to any communication architecture. (like TCP/IP or OSI or proprietaryprotocols). Hardware security devices can be attached to the communication deviceinterfaces. The main advantage of this solution is speed. However, this solution is notscalable and works well only on dedicated links.Direct-link level security can be provided at the physical layer, making the securityprotocol transparent to all upper layer protocols. For example, encryption may beprovided to the bit-streams passing through interface points. Direct-link level securitymay be provided at the data link layer also where the protection is provided at theframe level.This model is also useful in scenarios where all the machines are connected viadedicated links. If IP network is used instead of dedicated secure links, direct linklevelsolution at the data link layer security would not suffice.2.3 THE IPSEC PROTOCOL SUITEThe IPSec provides a standard, robust and extensible mechanism to ensure securityto IP and upper layer protocols. The protection takes the form of data integrity authentication,data confidentiality, anti-replay protection and limited traffic flow confidentiality.As discussed in the previous section, IPSec may provide end-system levelsecurity as well as subnetwork level security [44,45]. IPSec supports these two levels of18

security by using two modes of operation, i.e., transport and tunnel modes. It providestwo protocols, i.e., Authentication Header (AH) and Encapsulating Security Payload(ESP) for securing the IP datagrams. The AH provides data origin authentication,connectionless integrity and anti replay protection. The ESP provides all the securityfunctionalities of AH along with data confidentiality and limited traffic flow confidentiality.The IPSec protocols -AH and ESP- can be used to protect either an entire IPpacket or the upper layer protocols of IP payload. In transport mode, IPSec headeris inserted between the IP header and the upper layer protocol header to protect theentire upper layer protocol. In tunnel mode, the entire IP datagram to be protected,is encapsulated inside a new IP packet and the IPSec header is inserted between theouter and inner IP headers. The security services that IPSec provide, require sharedkeys for authentication and encryption. IPSec also provides the functionalities of bothmanual and dynamic negotiation of security parameters through a key managementprotocol called IKE.2.3.1 Security Association and Policies in IPSecBefore discussing the working of the two IPSec protocols -AH and ESP- we will definesome basic concepts used in IPSec protocol suite.A security Association (SA) is a relationship between two end-points of IPSec communicationthat describes how the entities will use security services to communicatesecurely. A SA consists of all the information needed to characterize and exchange securedcommunication. One point to note is that SA is an unidirectional relationship.Thus a pair of nodes require two separate SA to define the security relationship of theirbi-directional communications. The repository of the SAs is named as Security associationdatabase (SAD). A SA includes various pieces of information that the IPSecprocessing routines can use to determine whether the SA can be applied to a particular19

inbound or outbound IP datagram. These information, known as selectors of the SAdecide the scope of any particular SA. The fields in this SA selectors typically includethe following:• Source and Destination IP addresses• Name, either a user ID or system name• Transport layer protocol• Source and destination portsEach SA also contains cryptographic information for the IPSec processing routinesThese information include :• Data for providing authentication protection:authentication algorithm, keydetails etc.• Data for providing confidentiality protection: encryption algorithm, key, IV etc.• Data for providing anti-replay protection: sequence number counter, sequencenumber overflow flag, anti-replay counter, anti-replay window.• IPSec mode: Tunnel or Transport mode.• SA lifetime: in elapsed time or number of bytes protected.The inbound processing module of IPSec uses the SA entries from SAD to protectan IP datagram. When protected IP datagrams are sent, the sender needs to indicatewhich SA was used to encrypt the communication, so the receiver can use the sameSA to decrypt the IPSec protected IP datagram. This is mentioned using a field calledSecurity Parameter Index (SPI) present at IPSec headers. Thus, the sole responsibilityof SADB is to determine the particulars about IPSec header and cryptographicprotections on the outbound IP traffic and decapsulation of the incoming IPSec packetat the inbound traffic.20

Another important database in IPSec processing is the Security Policy Database(SPD). The SPD fulfills different roles for outbound and inbound processing.Foroutbound traffic, the SPD defines the action to be taken on the outgoing IP datagramwhereas the inbound SPD determines whether the current packet can be accepted bythe receiver machine or not. Each SPD rule consists of one or more selector fields asmentioned in the discussion about SA and the action corresponding to the rule.At the outbound processing, the possible actions can be any one of the following• Drop the packet.• Send out the packet without any protection.• Apply IPSec protection: In this case, SPD contains an entry to the correspondingSA.In case of inbound processing, the IPSec routine first decapsulates the incoming IPpacket using the SA entries selected by the SPI and IP address fields of the incomingIPSec packet. A record of the SAs that were used and the order in which they wereapplied is maintained and passed to SPD processing routine.The SPD processingroutine of the inbound processing ensures that each SA that protected the packet isapplied in the proper order. After successful verification, an appropriate rule is appliedto the packet.2.3.2 Authentication Header (AH)The Authentication Header (AH) of the IPSec protocol suite provides the followingtypes of protections.• Connectionless Integrity guarantees that the receiver has received the exact messagesent by the sender without any tampering happening in midway. The wordconnectionless signifies that the protocol does not ensure the orderly delivery ofthe datagram as the IP protocol itself is a state-less protocol.21

• Data Origin Authentication ensures that the message was actually sent by theapparent originator of the message and not by another user masquerading asthe original message originator.• Replay Protection ensures that the same packet is not delivered multiple timesand the packets delivered are not grossly out of order. This capability must beimplemented at the sender; the receiver may optionally enable its use.Next HeaderPayload len Reserved (set to zero)Security Parameter Index(SPI)Anti−replay sequence numberAuthentication Data(i)IP HeaderOuter IPheader(new)AH HeaderAH HeaderUpper protocol headersand dataAuthenticated FieldsInner IPheader(original)Authenticated FieldsUpper protocol headersand data(ii)(iii)Fig. 2.2: (i) Format of AH Header (ii) AH protocol in transport mode(iii) AHprotocol in tunnel mode.Figure 2.2 illustrates the AH header format. The header comprises of six fields.Five of the fields have fixed length, for a total length of three 32 bit words; the sixthfield is variable length. The individual fields are as follows :• Next header is a 8 bit field which indicates the payload that follows the AH [46].• Payload length field is the size of the AH header in 32 bit words minus 2.• Reserved is a field for future use, currently all the bits are set to 0.• SPI along with the destination address in the outer IP header is an index toSA at the receivers end.• Sequence number is a monotonically increasing number that keeps track of thenumber of messages sent from sender to receiver using the current SA. It is usedto prevent replay attacks.22

• Authentication data is a variable length field which contains ICV (integrity checkvalue) of the message contents needed to authenticate the packet. The ICV iscomputed by hashing the contents of the message. The mandatory keyed hashalgorithms for IPSec AH are HMAC-MD5 and HMAC-SHA1 [47–51].AH protocol can provide authentication both at end-system level and as well asin subnetwork level depending on the mode of operation. AH protocol supports twomodes of operations. These modes are as follows:• The transport mode of AH provides end-system level security. In this mode ofoperation, the communication endpoints must be the IPSec endpoints. The AHheader is inserted into the IP datagram by placing it immediately after the IPheader (and any OPTIONs) and before the next upper layer protocol header.Figure 2.2(ii) illustrates the layout of an AH packet in transport mode.Asdepicted in the figure, the AH protocol provides authentication security overthe entire IP datagram.• The tunnel mode of AH provides subnetwork level security.In this mode ofoperation, a security gateway is used to provide protection for multiple hosts ona network. An additional IP header whose source address is that of the securitygateway is placed at the beginning of the packet. The address of the original IPheader is kept intact and it becomes part of the payload of the new IP header.The destination IP address of the outer IP header is modified to the IP addressof the destination gateway machine, if the destination network is also protectedby a security gateway. Figure 2.2(iii) shows the position of the AH header inan IP datagram in tunnel mode. AH follows the new IP header and precedesthe original IP header.As shown in the figure, AH provides authenticationprotection to the entire IP datagram including the new IP header, the AHheader and the original IP datagram.23

LocalNetworkClient11Client12IPSec IPSec TunnelIPSecGateway 1 Gateway 2PublicNetworkClient21Client22LocalNetworkClient1n(i)Client2mClient11IPSec Transport mode SecurityClient21LocalNetworkPublicRouter 1NetworkRouter 2LocalNetwork(ii)Fig. 2.3: IPSec in both modes of operations: (i) IPSec in tunnel mode of operationbetween IPSec Gateways Gateway1 and Gateway2. (ii) IPSec in transport mode ofoperation between clients C11 and C21.Fig 2.3 depicts the different networking scenarios where the different modes ofIPSec protocols can be used for providing sub-network level and endsystem level ofsecurity.2.3.3 Encapsulating Security Payload (ESP)The AH provides several crucial security services to an IP datagram, but it doesnot provide confidentiality to the IP datagram. The Encapsulating Security Payload(ESP) protocol of IPSec provides confidentiality to IP datagram along with the securityprotections offered by AH.The following types of protection can be provided through the use of ESP:• Confidentiality guarantees that the message contents are not available in clearto possible eavesdroppers.• Traffic analysis protection (Tunnel mode only) provides the assurance that aneavesdropper cannot determine the ultimate source and destination involved incommunication.24

The ESP header can also provide the following types of protections offered by AH:• Connectionless integrity• Data origin authentication• Replay protectionSecurity Parameters index(SPI)Anti−replay sequence numberPayload Data (Sepcial unencrypted data + encrypted data)IPHeaderESPHeaderUpper protocol headersand packet dataESPTrailerESPauth dataPadding (0−2555 bytes)Authentication DataPad lengthNext HeaderEncrypted fieldsAuthenticated fields(ii)(i)OuterIPHeaderESPHeaderInnerIPHeaderUpper protocol headersand packet dataESPTrailerESPauth dataEncrypted fieldsAuthenticated fields(iii)Fig. 2.4: (i)ESP Header format (ii)ESP protocol in transport mode (ii)ESP protocolin tunnel mode.Figure 2.4(i) illustrates the ESP header format. The header compromises sevenfields, two of which are optional. The individual header fields are as follows:• SPI is the index into the receivers SA database.• Sequence Number field is the number of messages sent from the sender to thereceiver using the current SA.• Payload Data field is a variable-length field that fulfills the main purpose ofESP header of providing confidentiality. If the IP header is to be provided confidentialityprotection, this field contains an encrypted version of the message’scontents.25

• Padding is an optional field with a maximum length of 255 bytes. Padding isrequired to ensure that the data’s length is an integral multiple of the encryptionalgorithm. In that case, the effective length of the plaintext is the sum of thelengths of the unencrypted data, padding, the pad length field and the nextheader field.The mandatory encryption algorithms for IPSec ESP are DES-CBC [52] andnull encryption algorithm [53–55]. The later one does not provide encryptionprotection. As ESP must provide at least one of authentication and encryption,null encryption algorithm should not be used along with null authenticationalgorithm [56].If the message is to be authenticated, padding is required to ensure that theauthentication data field begins on a 4-byte boundary.• Pad length is a one byte field that represents total number of bytes of paddingcontained in the payload data field.• Next header represents the type of header that follows the ESP header.• Authentication data field is an optional field that contains the ICV, if the packetis to be provided with authentication and integrity protection.The mandatory keyed hash algorithms for IPSec AH are HMAC-MD5 andHMAC-SHA1 and the null authentication algorithm [50, 51].The ESP header normally consists of four parts. These four parts are as follows:1. The initial ESP header portion consists of SPI and sequence number.2. The data consists of the original IP packet payload and the original IP headerin case of tunnel mode.3. The ESP trailer consists of padding, if any, pad length and the next headerfield.26

4. The ESP authentication data consists of the authentication data, if any.The ESP header can also be used in either transport mode or tunnel mode similar tothe AH protocol . Figure 2.4 (ii) and (iii) illustrates the placement of the differentportions of ESP header in both modes.2.3.4 Mutual Authentication and key sharingIPSec uses a separate protocol called Internet Key Sharing (IKE) for providing keysharing after mutual authentication among the peers [25–27,57]. The main goal of IKEis to negotiate an IPSec SA with a peer. That is accomplished through a two-phaseprotocol. Phase 1 establishes an Internet Security Association and Key ManagementProtocol (ISAKMP) SA which is a secure channel through which the actual IPSec SAnegotiation can take place. Phase 2 negotiates a pair of one-way SAs: one for inboundand another for outbound communications.1. IKE Phase 1:ISAKMPThe establishment of the ISAKMP SA can be accomplished through the completionof one of several different phase 1 exchanges, also referred to as modes:Main Mode, Aggressive Mode and Base Mode. A phase 1 exchange has threegoals.• To negotiate security parameters: The initiator and the responder of Phase1 of IKE must agree on the the values and settings of the security parametersthat will govern the phase 2 communication. They also must negotiate,which authentication method would be used to authenticate the peers.• To authenticate identities: The peers authenticate each others identitybased on some additional information.27

• To establish a shared secret: Once the peers have agreed on the methodsand the parameters to be used to generate phase 1 shared secret, anexchange of message transfer is used to generate the shared secret.Three authentication methods are used in IKE: preshared secret key, digitalsignature and public key encryption. Any one of these three methods is usedin the phase 1 of the IKE to authenticate peers. Preshared secret key authenticationrelies on information (the preshared secret) shared between the IKEoriginator and responder. The other two authentication methods require eachpair to posses its public-private key pair.The negotiation of security parameters is achieved by sending possible securityproposals from the initiator, and the receiver acknowledges the selected proposalas counter-proposal message. In phase 1, the initiator proposes one or more alternativecollection of attributes. This takes the form of a single SA payload,containing a single proposal, which in turn contains one or more transformationpayloads. Each transformation payload defines the exact algorithm details. Theresponder selects one of the proposal and sends that proposal back to the sender.This chosen proposal becomes the basis of ISAKMP negotiation. The attributesthat are open to negotiate are the following: Encryption algorithm, Hash algorithm,Authentication method, Group description type, Keyed pseudo-randomfunction and Life duration.The above mentioned functionalities i.e. Authentication of peers and negotiationof security parameters are accomplished through the completion of one of severaldifferent phase 1 modes.2. The phase 2 of IKE: Quick ModeOnce the phase 1 negotiation is complete, an ISAKMP SA, which is a pro-28

tected channel, is established between the pairs. The phase 2 exchange uses thisISAKMP SA to negotiate IPSec SA. The only phase 2 IKE exchange that hasbeen defined so far is Quick mode. The phase 2 quick mode has the followinggoals:• To negotiate security parameters. The initiator and the responder mustagree on the values and settings of security parameters that will govern theoperation of the negotiated IPSec SA.• To prevent replay of phase 2 negotiations.• To generate keying materials for the IPSec SA.A phase 2 Quick mode exchange consists of three messages. The first two proposalsare the proposal from the initiator and counter-proposal from responderrespectively, similar to phase 1 of IKE. These two exchanges negotiate the IPSecSAs policy and parameters and generate keying material in an authenticatedmanner that protects against a replay of a previous negotiation.The last message, from the initiator to the responder, concludes the exchangeand reassures the responder that the responders proposal was received.2.4 SUMMARYIn this chapter, we have described different layers of the TCP/IP protocol suite and thefunctioning of the individual layers. We have also discussed about the four differentlevels of security protocols along with the scenarios, where these security protocolsmay be implemented.We then discuss about communication security protocols atdifferent layers of he TCP/IP protocol suite. We observe that security protocols atdifferent levels and layers of the TCP/IP protocol suite have their own advantages anddisadvantages. The level at which the security is to be provided should be judiciously29

decided based on these considerations.Next, we review the IPSec protocol suite. We have described the security databaseorganization and their role towards the IPSec processing.IPSec protocols (AH and ESP) definition and processing.a brief discussion on the Internet Key Exchange protocol.We have explained theWe have also presentedWe observe that IPSecprovides several security protection to IP datagrams. However, the encapsulation ofupper layer protocol headers viz., transport, application layer headers and even theoriginal IP header in case of tunnel mode, restricts several intermediate devices fromperforming QoS and security related processing. In the next chapter, we discuss theinteroperability issue between IPSec protocol and QoS and security related processingat intermediate devices.30

CHAPTER 3Problem of QoS and Network access control with IPSecThe IPSec protocol suite has become a standard for providing security services forInternet communications. It protects the entire IP datagram, so that no intermediatenetwork node in the public Internet can access or modify any information above the IPlayer in an IPSec-protected packet. However, recent advances in Internet technologyhave introduced a rich new set of services and applications, like traffic engineering, TCPperformance enhancements, transparent proxying and caching, all of which requireintermediate network nodes to access certain parts of an IP datagram, usually theupper layer protocol information. This access is required to perform flow classification,constraint-based routing, or other customized processing. Some security devices toorequire access to transport and upper layer headers for providing traffic analysis andfirewalling capabilities. This is in direct conflict with the IPSec mechanisms.3.1 QOS PROCESSING AT INTERMEDIATE NODESFundamentally, end-to-end protection model of IPSec and its strict layering principleare unsuitable for an emerging class of new networking services and applications.Unlike in the traditional Internet, intermediate routers are beginning to play moreand more active roles. They often rely on some information about the IP datagrampayload, such as certain upper layer protocol header fields, to make intelligent routingdecisions. In other words, routers have need to participate in the processing oflayers above IP. In this regard, we discuss several intermediate applications used forproviding Quality of Service.31

• Traffic EngineeringThe flow information in IPv4 is encoded in both the IP header and the upperlayerprotocol headers, such as TCP or UDP port numbers.Therefore, anymechanism that discriminates between flows inside the network (generally calledflow classification) will need to access the upper-layer protocol headers. If thisis done inside the network (as opposed to classification at end-points), it maypotentially conflict with IPSec. Flow classification is essential in providing richclasses of services and quality-of-service (QoS) support.These include flowbasedand class-based queuing, random early detection (RED), router-basedcongestion control and policing, integrated services [resource reservation protocol(RSVP)] and differentiated services [multi protocol level switching (MPLS)],etc [58–60].• Application-Layer ProxiesSome Internet routers can provide application-layer services for performancegains. For example, an intermediate router can become a transparent web proxywhen it snoops through the TCP and HTTP header of an IP datagram todetermine the web page request, and serve it with the web page from the localcache. It is transparent to end-users but boosts the responsiveness, because thedelivery paths for web request and data between the intermediate router andthe web site server are eliminated.• Transport-aware link layer mechanismsInternet has accommodated a very wide range of link technologies, but certaintransport protocols like TCP have not achieved optimal performance, when operatedover a path that includes lossy wireless links or long-delay satellite links.For example, to significantly improve the TCP performance over a wireless link,32

the base station at the lossy link must be aware of the TCP state information ineach passing flow, and deliberately delay or drop certain types of TCP packets.Such link-layer mechanisms for TCP performance improvement (often referredto as TCP Performance Enhancing Proxies or TCP-PEP) require intermediatenodes to access and sometimes modify the upper layer protocol headers [61–64].3.2 SECURITY SERVICES AT INTERMEDIATE NODESThe intermediate nodes not only provide services for improving the overall performanceof the communication, they also perform several security related functionalities. Theseincludes traffic analysis, access control through firewalling etc [65].Network engineers and administrators often need the ability to monitor and analyzetraffic from a network to perform a variety of tasks like diagnosis, intrusiondetections, firewalling etc. These tasks often require the ability to access upper-layerprotocol headers within packets, but the advocated proliferation of IPSec encryptioncan prevent such analysis, or limit its granularity (for example, individual nodes orflows are inseparable in an IPSec tunnel between two gateways). The fact that suchanalysis is both necessary and essential means that we should find a way of accommodatingits prerequisites.In case of providing firewall service for controlling network access, the firewallmay require to access transport layer and application layer headers depending on thesecurity policy.For example, for providing stateful firewall, access to TCP headeris required, and application level firewalls require access to application layer protocolheaders.The end-to-end security of IPSec restricts the access to the upper layerheaders in these cases too.33

3.3 IPSEC AND INTERMEDIATE DEVICE INTEROPERABILITYOver the last decade or more, researchers have proposed many solutions to address theissue of interoperability between IPSec and several intermediate devices required forproviding QoS or Security services. In this section, we describe some of the relevantsolutions.3.3.1 Replacing IPSec with a transport-layer security mechanismThis approach uses a transport-layer security mechanism as an alternative to IPSec toprovide security services. The transport-layer mechanism such as secure socket layer(SSL) or transport layer security (TLS) operates above TCP payload.SSL is not a single protocol, rather two layers of protocols.The SSL RecordProtocol provides security to TCP payload. Three higher layer protocols, i.e., SSLhandshake protocol, SSL change cipher spec protocol and SSL alert protocol are usedin the management of SSL exchanges. The SSL Record Protocol provides the followingsecurity services to the TCP payload:Confidentiality: The handshake protocols define a shared secret key that is usedfor encryption of TCP payload using symmetric key cryptographic algorithm.Message Identity: The handshake protocols also define a shared secret key that isused to form a keyed message authentication code of the TCP payload data.Figure 3.1 depicts the overall operation of the SSL Record Protocol. The Recordprotocol takes an application message, fragments the data into blocks, compresses thedata, applies MAC, encrypts, adds SSL record header and transmits the resulting unitin a TCP segment. Received data are decrypted, verified, decompressed, reassembledand delivered to the application level.SSL encrypts the TCP data while leaving the TCP header in unencrypted andunauthenticated form so that intermediate nodes can make use of the TCP state in-34

Application DataFragmentCompressADD MAC¢¡¢¡¢¡¢¢¡¢Encrypt£¡£¡£¡£¡£ ¤¡¤¡¤¡¤¡¤¤¡¤¡¤¡¤¡¤ £¡£¡£¡£¡£¤¡¤¡¤¡¤¡¤ £¡£¡£¡£¡£Add SSL RecordHeader¥¡¥¡¥¡¥¡¥ ¦¡¦¡¦¡¦¡¦¦¡¦¡¦¡¦¡¦ ¥¡¥¡¥¡¥¡¥¦¡¦¡¦¡¦¡¦ ¥¡¥¡¥¡¥¡¥¦¡¦¡¦¡¦¡¦ ¥¡¥¡¥¡¥¡¥Fig. 3.1: SSL Record Protocol Operationformation encoded in the TCP header. In fact, many Internet applications alreadyimplement security with SSL or TLS, such as most web browsers (using HTTPS protocol)and mail programs (using SIMAP and SPOP). However, letting the entire TCPheader appear in clear text exposes several vulnerabilities of the TCP session to avariety of TCP protocol attacks (in particular traffic analysis), because the identityof sender and receiver are now visible without confidentiality protection.Further,SSL/TLS works only on TCP, and not on user datagram protocol (UDP), thus, therange of applications supported is smaller than with IPSec [66].3.3.2 Tunneling one security protocol within anotherAnother solution for interoperability advocates for tunneling one security protocolinside another. This solution proposes to use some transport layer security protocol (forexample SSL/TLS) for providing end-to-end security of the transport layer payload.The transport layer header can be protected using IPSec protocol in tunnel mode. Thecryptographic parameters used in IPSec protection can be shared with the intermediatenodes for their processing keeping the cryptographic parameters used in SSL/TLS35

secret between the endpoints only.However, the IPSec protocol not only encryptsthe transport layer header but also the entire transport layer payload.Thus thetransport layer payload gets encrypted twice. Moreover, the intermediate nodes requireto decrypt/encrypt the entire IP datagram just to access the transport layer headersresulting in unnecessary overheads.3.3.3 Using a transport-friendly ESP formatThe Transport-Friendly-ESP (TF-ESP) proposed by Bellovin [67] modifies the originalESP packet format to include limited TCP state information, such as flow identificationand sequence numbers, in a disclosure header outside the scope of encryption (butauthenticated). This approach works well for some TCP PEP mechanisms which donot require any write access on TCP header fields but fails in other cases. Further,the unencrypted TCP state information is made available universally, even to untrustworthynodes, which creates vulnerabilities for possible attacks. In addition, TF-ESPis not flexible enough to support all upper-layer protocols.3.3.4 Multi Layer IPSec (ML-IPSec)The ML-IPSec [30,32] uses a multilayer protection model in place of a single end-to-endmodel. Unlike IPSec where the scope of encryption and authentication applies to theentire IP datagram, this scheme divides the IP datagram into zones. It applies differentprotection schemes to different zones. Each zone has a set of security associations, aset of private keys that are not shared with other zones, and a set of access controlrules defining which nodes in the network path have access to the zone.A zone is any portion of an IP datagram under the same security protectionscheme. The granularity of a zone is one octet. The entire IP datagram is partitionedinto non-overlapping zones, except for the IP header in the transport mode. A zone36

IP DatagramZone 1(2 Sub Zones)Zone 2(2 Sub Zones)Zone 3( Sub Zone)Fig. 3.2: Zone and Zone map in ML-IPSecneed not be a contiguous block in an IP datagram, but each contiguous block is calleda sub-zone. A zone map is a mapping relationship from octets of the IP datagram tothe associated zones for each octet. There are some important points to note aboutthe zone definition of ML-IPSec.• The ML-IPSec zone map is static for the entire lifespan of the ML-IPSec communication.It can not be reorganized to provide service to some new intermediatenode.• The ML-IPSec zones are specified on physical byte level. Thus the accommodationof TCP/IP datagrams with variable size OPTION field is not possible.Figure 3.2 depicts a sample zone map of an IP datagram with three zones.The ML-IPSec redefines the concept of security association of standard IPSec protocolas the intermediate nodes participate in defining the security parameters fordifferent zones. It uses a single SA for each of the zones and finally a composite SA(CSA) is derived for the protection of any IP datagram. The protocol uses manualmechanism for establishing these SAs before the commencement of DML-IPSec communication.Another import point to note is that these SAs remain constant for theentire lifespan of the communication.When ML-IPSec protects a traffic stream from its source to its destination, itfirst partitions the IP datagram into zones and applies zone-specific cryptographic37

Senderany node TCP PEP node any node RecevierIPHdrESPHdrIPHdrESPHdrIPHdrESPHdrIPHdrZone1IPHdrZone1IPHdrTCPHdrTCPHdrTCPHdrTCPdataZone2 Zone2 Zone2TCPdataFig. 3.3: ML-IPSec Protocol Operationprotections. When the ML-IPSec protected datagram flows through an authorized intermediategateway, a certain zone of the datagram may be decrypted and/or modifiedand re-encrypted, but the other zones will not be compromised. When the datagramreaches its destination, the ML-IPSec will be able to reconstruct the entire datagram.Fig 3.3 depicts the operation of ML-IPSec protocol. The IP datagram is protected intunnel mode of operation. The original IP header and TCP header constitutes the firstzone and the cryptographic information for this zone is shared between the ML-IPSecendpoints and intermediate nodes, TCP PEP agents here. The other zone is spannedover TCP payload and secured in end-to-end manner between ML-IPSec endpoints.The same security protocols, AH and ESP, are used in ML-IPSec with relevantchanges to support multiple zones. Both AH and ESP support transport mode andtunnel mode of operation. In case of AH, the authentication header is further dividedto contain zone specific authentication information. In case of ESP, the payload portionis divided into zones. Each zone is encrypted separately after padding (if required).Finally the authentication trailers of all the zones are appended after the last encryptedzone. Figure 3.4 describes the format for both the headers for a case in which the ML-IPSec protocol involves three zones.38

Nxt Hdr Payload Len ReserveredSecurity Parameter Index (SPI)Sequence NumberAuthentication Data for zone 1Authentication Data for zone 2Authentication Data zone 3(i)Security Parameter Index (SPI)Sequence NumberEncrypted Payload for zone 1(Variable length)PadPad Length Next HeaderEncrypted Payload for zone 2(Variable length)PadPad LengthEncrypted Payload for zone 3(Variable length)Pad Pad LengthAuthentication Trailer for zone 1Authentication Trailer for zone 2Authentication Trailer for zone 3Fig. 3.4: ML-IPSec Packet Format (i) Authentication Header (ii) EncapsulatingSecurity PayloadThe ML-IPSec protocol works well in a closed environment; however the protocolsuffers from the following deficiencies [68]:(ii)• The ML-IPSec requires complete knowledge about all intermediate routers beforethe commencement of the communication. This can pose problems in Internetscenarios and also in mobile networking environments.• All the intermediate routers need to be manually configured for ML-IPSec zonesand cryptographic information. This requirement causes scalability problems forthe deployment of the ML-IPSec solution.• Security zones are designated as static byte ranges and need to be specifiedbefore communication commences. As a result, the exact length of the headersneed to be known prior to implementation. The ML-IPSec will not functioncorrectly if the end nodes change IP or TCP option fields.• The static zone mapping of ML-IPSec forces one to configure the zone mappingbefore the commencement of the communication. It restricts the protocol from39

dynamically changing the zone mapping for providing access to new intermediatenodes without terminating the existing ML-IPSec communication.3.4 SUMMARYIn this chapter we have discussed the interoperability issues between several intermediatenodes and IPSec protocols. We also described the possible solutions proposed inthe literature to address the issues. We observe that the ML-IPSec protocol providesa novel solution of providing multi layer security to an IP datagram. This approachsolves the issue of the access to the upper layer headers to intermediate nodes. However,the manual key sharing mechanism and static zone mapping of the protocol givesrise to some important issues with respect to scalability of the solution. In the nextchapter, we describe our extension to the ML-IPSec protocol, called DML-IPSec toaddress all these issues.40

CHAPTER 4An Extension to ML-IPSec protocol - Dynamic Multi LayerIPSecThe IPSec protocol was designed for providing security for communications over theIP layer primarily between two end-points. Over time, the protocol had to address theissue of giving access to upper layer protocol headers at intermediate nodes while protectingthe communication between end-points. The ML-IPSec protocol was proposedfor resolving this issue. However, the current version of ML-IPSec protocol fails toaddress the requirement of dynamic key sharing between the intermediate and sourcenodes. It is also not designed to support variable zone sizes. A partial solution tocater to variable zone sizes has been proposed in [68]. However, this solution maynot give access to application layer headers of any generic application in intermediatenodes. Additionally, it may sometimes lead to suboptimal zone partitioning resultingin unnecessary overheads both in forms of packet size and processing delay. Also thediscussion lacks implementation details and performance related issues.Our extension to the ML-IPSec protocol, called Dynamic Multi Layer IPSec (DML-IPSec) aims to solve these issues. The DML-IPSec tries to achieve this by re-definingsome of the IPSec fundamentals such as the packet structure, security associationetc. It also redefines the ML-IPSec zones and zone mapping. A new key sharingprotocol enables dynamical sharing of cryptographic information between the sourceand intermediate nodes. Finally a solution to the variable zone size issue of ML-IPSecprotocol is implemented through on-the-fly derivation of the zone lengths.This chapter is organized as follows.Section 4.1 presents the basic definitions41

and terminologies in the context of the DML-IPSec protocol. Section 4.2 presents thedetails of DML-IPSec protocol.4.1 ELEMENTS OF DML-IPSEC4.1.1 DML-IPSec ZoneIn ML-IPSec protocol, an IP packet is partitioned into non-overlapping segments calledzones. A zone need not be contiguous, in which case, all contiguous portions of a zoneform a subzone of that zone. Each zone is given a cryptographic protection that isindependent of the other zones by using a separate set of cryptographic parameters.A zone map is a mapping relationship from octets of the IP datagram to the correspondingzones. The zone map defines the zone boundaries for all the zones andthese boundaries remain constant throughout the lifespan of the ML-IPSec SecurityAssociations (SAs). For each zone, all octets of all subzones are concatenated (in theorder they appear in a datagram) and then encapsulated into the ML-IPSec PayloadData field. During in-bound processing, original IP datagrams are reconstructed fromthe decrypted zones using the zone mapping.It can be observed that though the zones themselves may not be contiguous, howeverthe position of the different protocol headers are non-overlapping and contiguousinside an IP datagram.This observation justifies the zones to occupy contiguoussegments in an IP datagram in our definition of DML-IPSec protocol. Thus, in theDML-IPSec protocol, zone is defined as a set of non-overlapping and contiguous partitionsof an IP packet. This is in contrast to the case of ML-IPSec, where a zone mayconsists of many contiguous portions called subzones. All the zones are provided withcryptographic protection independent of other zones.We also propose to classify zones into two separate types depending on the accessibilityrequirements at the intermediate nodes. The first type of zone, called type42

I zone, is defined on headers of IP datagram and is required for examination andmodification by intermediate nodes. Thus, we can define different type I zones for IP,transport, application layer headers or in suitable combinations of these headers. Thesecond type of zone, called type II zone, is meant for the payload portion and is keptsecure between the endpoints of IPSec communications. The single type II zone startsimmediately after the last type I zone and spans till the end of the IP datagram. So,the scope of the single type II zone depends on that of the last type I zone. In ourapproach, an IP datagram is partitioned into several type I zones followed by a finaltype II zone. Fig. 4.1 shows the case of an HTTP packet in ESP tunnel mode. Wemay define three type I zones, one each for inner IP header, TCP header, and HTTPheader respectively and the type II zone is defined on the HTTP payload. Alternatively,we may define only a single type I zone consisting of inner IP header and TCPheader and in this case the whole TCP payload (including HTTP header and HTTPpayload ) forms the single type II zone. However, this partitioning depends on thelevel of access required by the intermediate nodes. For example, if no intermediateprocessing is required during the entire DML-IPSec session, the single type II zonemay cover the entire portion of the datagram secured by DML-IPSec. In that case notype I zone exists in the IP datagram and this scenario resembles the case of a normalIPSec packet.This new definition of a zone not only facilitates all granularities of access to theprotocol headers by the intermediate nodes, but also enable us to define logical zoneboundaries in contrary to byte level physical zone boundary of the ML-IPSec.Asdiscussed earlier, a zone consists of headers or payload, the boundaries of which mayvary depending on the context. Hence the flexibility obtained in the definition of logicalzone boundaries can be used for resolving the issue of accommodating IP packets withvariable length headers.43

IPHDRESPHDRIPHDRType IZone 1TCPHDRHTTPHDRHTTPPayloadType I Type I Type IIZone 2 Zone 3 Zone 1(i)IPHDRTCPHDRHTTPHDRHTTPPayload(ii)IPHDRESPHDRIP TCPHDR HDRType I Zone 1HTTP HTTPHDR PayloadType II Zone 1(iii)Fig. 4.1: Different zone mappings of a HTTP PacketThe DML-IPSec protocol also defines zone map as the mapping from the octetsof the IP datagram to different zones of the DML-IPSec protocol. However, unlikethe case of ML-IPSec, zone map of DML-IPSec does not specify any zone boundary.Unique integer numbers are used to represent the scope of each zone. In case of zonesspanning over multiple protocol headers, the integer values for the corresponding zonesare concatenated to represent the scope of the zone. All the zone sizes can dynamicallyvary according to the concerned IP datagram. As per our definition of type I zones, itis the combination of protocol headers. If any of these protocol headers have variablesize (as in case of IP and TCP headers with OPTION fields), the corresponding typeI zone size varies. The length of the single type II zone depends on that of the lasttype I zone. Moreover, the payload portion of any protocol may vary depending onapplication requirement. So, the length of the single type II zone also varies. The DML-IPSec protocol derives the zone lengths dynamically from the header length fields ofthe protocol headers. This feature of DML-IPSec zones is important for addressingthe issue of variable header lengths. Another important feature of DML-IPSec zoneis that the zone maps need not necessarily be constant throughout the lifespan of theDML-IPSec security association. The key sharing protocol may modify any existing44

zone map for providing service to some intermediate nodes.One important feature of the zone mapping of the DML-IPSec is the ability ofdynamic reorganization of the type I zones without the need of renegotiation of thecryptographic parameters for the type II zone. In other words, two DML-IPSec endpointsmay configure only one type II zone without any type I zone to start the initialcommunication. If some intermediate node requests the access to some upper layerheader, the zone maps are reorganized and the SAs for the type I zones are decided.The current cryptographic parameters used for the sole type II zone are used for thenew type II zone after the reorganization of zones.If any other reorganization ofthe zones is required, at some later time, the current cryptographic information forthe type II zone is used for the type II zone constructed after the reorganization ofzones. This flexibility enables one to start a DML-IPSec session with single type IIzone and update the zone information on need basis. In this way, the overhead due tounnecessary partitioning of an IP packet into zones is avoided.No_of_Zones 4Zone 1Type IScope 1Zone 2Type IScope 2Zone 3Type IScope 3Zone 4Type IIScope 3−EOPNo_of_Zones 2Zone 1Type IScope 12Zone 2Type IIScope 2−EOP(ii)(i)Fig. 4.2: Zone MapFig 4.2 depicts the entries of the zone map for the case described in fig 4.1. Thezone map depicted in the fig 4.2(i) corresponds to the HTTP header packet with 3type I zones and single type II zone as shown in fig 4.1(i). The scope of the first zone45

is restricted to the inner IP header and hence represented as ”Scope 1”. The scopefield of zone 2 is specified as ”Scope 2” indicating that the scope of the zone is overthe next header which is the TCP header. The scope of 3rd zone is represented as”Scope 3” to specify the scope over the third header which is the application header.The scope of the sole type II zone begins after the application layer header and spanstill the End-Of-Packet. It is represented as ”Scope 3-EOP. The zone map depicted asfig 4.2(ii) represents the zone map for the HTTP packet with two zones shown in thefig 4.1(iii). The single type I zone is defined over the inner IP header and transportlayer header and represented as ”Scope 12” in the zone map. The sole zone II spansover the transport layer payload and its scope is represented as ”Scope 2-EOP” in thefigure.4.1.2 DML-IPSec PacketsThe DML-IPSec supports the AH and ESP protocols of the conventional IPSec withsome modifications.In case of AH, the authentication header is almost similar toIPSec.The only difference is that the authentication data for the whole packet isfurther divided into zone specific authentication data. The packet format of the AHis exactly similar to that of ML-IPSec.The DML-IPSec ESP payload is partitioned into zone specific segments.Thedata for each zone, together with padding, padding length, and next header field(applicable for the single type II zone only), are collectively referred to as the ciphertext block for that zone. The size of each cipher text block can be determined by thealgorithm definition. Each zone may have a separate authentication trailer. The sizeof the authentication trailer depends on the algorithm used for that particular zone.In DML-IPSec ESP, the authentication trailer of each zone (if present) is appendedimmediately after the corresponding encrypted zone data, in contrary to ML-IPSec,46

where all the authentication trailers are appended together after the encrypted dataof the last zone.Nxt Hdr Payload Len ReservedSecurity Parameter Index (SPI)Sequence NumberAuthentication Data for Type I zone 1Authentication Data for Type I zone 2Authentication Data for Type II zone(i)Security Parameter Index (SPI)Sequence NumberType I Zone 1 Pay Load(Variable length)PadAuthentication TrailerPadType II Zone Pay Load(Variable length)Pad LengthType I Zone 2 Pay Load(Variable length)Authentication TrailerPad LengthPadPad Length Next HeaderAuthentication Trailer(ii)Fig. 4.3: DML-IPSec Packet Format (i) Authentication Header (ii)Encapsulating Security PayloadFig 4.3 depicts the layout of the AH and the ESP headers according to the DML-IPSec definition with two type I zones and one type II zone.4.1.3 Security AssociationsThe concept of security association (SA) is fundamental to IPSec. An SA is a relationshipbetween two end-points of IPSec communication that describes how the entitieswill use security services to communicate securely. However in DML-IPSec, severalintermediate nodes may participate in defining these security protections to the IPdatagram. Moreover, the scope of one particular set of security protection is valid onsingle zone. So we define a single SA for each zone of an IP datagram. We finallycombine all these individual zonal SAs to represent the security relationship of theentire IP datagram. This combined security relationship consists of one zone map anda set of zone lists. The zone map contains the common information for all zones alongwith the logical zone definitions, whereas the individual zone list consists of securityinformation related to that particular zone only.For example, fields like sequence47

number counter, anti-replay window, DML-IPSec protocol mode are of common interestto all the zones and are present in the zone map. The zone specific cryptographicinformation like encryption algorithm, encryption key, authentication algorithm andauthentication key are present at corresponding zone specific lists.The DML-IPSec endpoints have the complete knowledge of the entire zone map aswell as that of all zone specific lists. The intermediate nodes have the knowledge of therelevant type I zone specific information. The cryptographic information related tothe type II zone is kept hidden from any intermediate node. The key sharing protocolis responsible for sharing this partial zone information with the intermediate nodes.The two endpoints of a DML-IPSec communication may re-negotiate the SA definitionfor providing some service to the intermediate node having the capability ofpartial DML-IPSec processing.No_of_Zones 3 No_of_Zones 2Zone 1 Type I Zone 1 Type IScope 1Scope 1Zone 2 Type I Zone 2 Type IScope 2Scope 2Zone 3 Type IIScope 2−EOPAnti replay WindowSequence NumberMode : TunnelSPI ValueAnti replay WindowSequence NumberMode : TunnelSPI ValueZone 1Enc Alg0 : DESKey : K1Auth Alg0 : A1Key : K2Zone 2Enc Alg0 : DESKey : K3Auth Alg0 : A1Key : K4Zone 3Enc Alg0 : DESKey : K5Auth Alg0 : A1Key : K6Zone 1Enc Alg0 : DESKey : K1Auth Alg0 : A1Key : K2Zone 2Enc Alg0 : DESKey : K3Auth Alg0 : A1Key : K4DML−IPSec End PointsDML−IPSecIntermediate nodeFig. 4.4: Security association for DML-IPSecFig 4.4 depicts the security association both in DML-IPSec end points and intermediatenodes. This scenario corresponds to the case of ESP tunnel mode with two48

type I zones, where the first type I zone spans the inner IP header and the second typeI zone on transport layer header. The single type II zone spans the transport layerpayload. As depicted in the Fig 4.4, the end-points of the DML-IPSec communicationhave the complete knowledge of the zone maps as well as the zone lists for all the zones,whereas the intermediate node has information only about the two type I zones.4.2 THE DML-IPSEC PROTOCOLThe DML-IPSec protocol has two basic components. The first one is for the processingof packets at the endpoints as well as several intermediate nodes.The secondcomponent is a protocol for sharing of cryptographic parameters to the intermediatenodes. In this section we first describe the DML-IPSec processing followed by the KeySharing Protocol. Finally we discuss the handling of variable sized zones.4.2.1 DML-IPSec ProcessingThe DML-IPSec processing is different for the endpoints and intermediate nodes. Theendpoints of a DML-IPSec communication involve two types of processing. The firstone, called Outbound processing, is responsible for generating DML-IPSec packetsfrom an IP datagram according to the security policy and security association. TheInbound processing is responsible for generating the original IP datagram from a DML-IPSec packet using the security policies and security associations.• Outbound ProcessingWhen the source node sends out an IP datagram, the following steps take place.Derivation of ZonesThe very first operation at the outbound processing is the derivation of thezones and zone lengths. The number of zones and scope of each zone is derived49

from the information of the zone map. This zone map and zone specific SAdatabases are populated either by the initial configuration of a DML-IPSeccommunication or by the key sharing protocol. The length of each zone mayvary from IP datagram to IP datagram. In case of type I zones, the headerlength field of the protocol headers are used to determine the length of eachzone; otherwise the implicit length definition is used to derive the zone lengths.The zone boundary of the type II zone is derived from the total length of theIP datagram and the cumulative lengths of all type I zones.Zone Specific EncryptionThe next step involves encryption of the relevant data in each zone. At thebeginning of the encryption module, the sender adds any necessary padding toeach zone’s Payload Data field, to meet the encryption algorithm’s block sizerequirement, if any, and to align it on a 4-byte boundary according to the DML-IPSec ESP format. The sender then encrypts the resulting plaintext (payloaddata, padding, pad length, and next header in case of single type II zone) usingthe key, the encryption algorithm, and the encryption mode indicated by thezonal SA and cryptographic synchronization data, if any.Zone Specific Authentication CalculationAfter the zone specific encryption, the sender calculates the authentication valuesfor the zones (if required) and appends the same after the encrypted dataof the corresponding zone. The cryptographic information required for the calculationis derived from the corresponding SA entry.Fig 4.5 depicts the outbound processing of an IP datagram with two type Izones followed by a type II zone for ESP protocol. As explained in the abovediscussion, the zone map is used to derive the zone length followed by zone wise50

IP DatagramZone MapZone 1 Zone 2 Type II ZoneEncryptEncryptEncryptSA 1AuthenticationAuthenticationSA 2AuthenticationSA 3IP HDRESP HDRFig. 4.5: Outbound Processing at DML-IPSec endpointencryption. Finally the zone specific ICV’s are computed on encrypted zonedata.• Inbound ProcessingOn inbound processing, the receiver performs the following steps on an IP datagram.Derivation of Zone BoundaryThe derivation of zone boundaries is significantly different from that of outboundprocessing. The main difference here is that the length fields of zones remainencrypted. After receiving a DML-IPSec datagram, the receiver processes thedatagram according to the zone map.The receiver starts decrypting type Izones till it decrypts the header length field of the header. In case of a typeI zone spanning over multiple protocol headers, partial decryption is used tillthe length of all the protocol headers is derived. The length of the single typeII zone is derived using the total length of the IP datagram and other type Izone lengths. The partial decryption of the type I zones are done on temporarybuffers, as the encrypted data would be required in authentication verification51

step of the inbound processing that follows the present step of zone boundaryderivation.Authentication verificationThe authentication verification in DML-IPSec is zone specific.The cryptographicinformation regarding authentication calculation algorithm are fetchedfrom SAs corresponding to zones.The authentication values are calculatedin the same way as that of the outbound processing, and the values are thenmatched against the authentication values stored in the authentication field ofeach zone.If the match is perfect for all values, the authentication verificationsucceeds and the datagram is processed by next step, else the datagram isdropped.Zone Specific DecryptionThe final step of the inbound processing is the zone specific decryption of theIP datagram.This is almost opposite to that of zone specific encryption atthe outbound processing. The zone specific decryption details are fetched fromcorresponding SAs. After the decryption of all the zones, the original IP datagramis re-constructed by trimming fields like pad, IV (if present) etc.Theinitial portions of the type I zones which are used for partial decryption for thederivation of zone boundaries are not decrypted again in this step. Rather thedecrypted values from the temporary buffer position are copied and only thenecessary portions of type I zones are decrypted.Fig 4.6 depicts the inbound processing of an DML-IPSec ESP packet with twotype I zones followed by a type II zone. As depicted in the figure the inboundprocessing consists of derivation of zone boundaries using partial decryption followedby authentication verification and zone specific decryption to reconstruct52

IP DatagramDecryption Decryption DecryptionAuthenticationVerificattionAuthenticationVerificattionAuthenticationVerificattionDerivation of zone lengthsPartialDecryptionPartialDecryptionSA 1 SA 2 SA 3Zone MapIP HDRESP HDRFig. 4.6: Inbound Processing at DML-IPSec endpointthe original IP datagram.• Partial Datagram Processing at Intermediate nodesThe intermediate nodes process an incoming IP datagram depending on thepresence of the security parameters for that particular DML-IPSec communication.In the absence of the security parameters, the IP datagrams are forwardedto the destination and at the same time the key sharing protocol gets executed.After successful execution of the key sharing protocol, all the incomingIP datagrams get partially decrypted according to the security association andzone mapping at the inbound processing module. After the inbound processing,the partially decrypted IP datagram traverses through the networking stack ofthe intermediate node. Before the datagram leaves the intermediate node, it isprocessed by the outbound module to reconstruct the DML-IPSec packet.Inbound Processing at Intermediate nodesThe inbound processing module is almost similar to that of DML-IPSec endpoints.The zone boundaries of the relevant type I zones are first calculated53

followed by ICV verification and decryption of the necessary type I zones. Onexecution of these steps the inbound module gets partially decrypted data forcustomized intermediate processing. The inbound module constructs a legitimateIP datagram using this partially decrypted datagram so that it can traverseTCP/IP stack of the intermediate node. This requires modification and updationof relevant fields of the newly constructed IP datagram viz., total length,checksum etc. Another point to note is that the DML-IPSec header and zonespecific IV, if any, after the processing is not discarded; rather the inboundprocessing module appends the header after the legitimate IP payload and recomputesthe length fields accordingly.IP DatagramDecryptionDecryptionAuthenticationVerificattionAuthenticationVerificattionDerivation of zone lengthsPartialDecryptionPartialDecryptionSA 1 SA 2Zone MapIP HDRESP HDRFig. 4.7: Partial Inbound DML-IPSec Processing at intermediate nodeFig 4.7 depicts the partial inbound processing of a DML-IPSec ESP packet withtwo type I zones and one type II zone. Both the type I zones are decryptedto form the upper layer headers of the intermediate IP datagram, whereas thewhole type II zone gets copied to the payload portion of the intermediate IPdatagram as it is.54

Outbound Processing at Intermediate nodesThe outbound processing module is similar to that of DML-IPSec endpoints.Here the zone specific SAs are derived using the header fields and saved dataof the partially decrypted datagram to regenerate the type I zones. Finally thetype II zone is concatenated to reconstruct the DML-IPSec packet.IP DatagramMaster Zone MapZone 1 Zone 2EncryptEncryptSA 1AuthenticationAuthenticationSA 2IP HDRESP HDRFig.node4.8: Partial Outbound DML-IPSec Processing at intermediateFig 4.8 depicts the partial outbound processing of a DML-IPSec ESP packet withtwo type I zones and one type II zone. Both of the type I zones are constructedfrom the upper layer headers of the intermediate IP datagram and the typeII zone is constructed from the payload portion of the partially decrypted IPdatagram.4.2.2 Key sharing ProtocolThe key sharing protocol is responsible for dynamically providing all cryptographicinformation to intermediate nodes.When an intermediate node fails to process aDML-IPSec packet due to the absence of required security parameters, the key sharingprotocol is invoked between the source of DML-IPSec packet and the intermediatenode. At the end of successful execution of the protocol, the relevant security asso-55

ciations are made available to the intermediate node. Whenever the current securityparameters are changed or get expired, the corresponding action is being taken at theintermediate node also. The realization of these functionalities require the notificationof the events and exchange of data using appropriate messages. We propose touse ICMP messages for all the communications required for the execution of the keysharing protocol. In the following section, we explain the working of the protocol indetail.Source NodeESP PacketIntermediate NodeForward (2)Destination NodeInvocation (1)New Zone Map (3)Phase IZone map ack (4)Auth Challenge (5)Phase IIAuth Response (6)Phase IIIShared Secret (7)Shared Secret Ack (8)Phase IVZone & crypo info (9)Zone & crypo info Ack (10)ESP Packet(11)Intermediate Processing and ForwardFig. 4.9: Key Sharing Protocol• Invocation of the protocolWhenever a DML-IPSec packet traverses through an intermediate node, that requiresaccess to some of type I zones, the inbound security database is searched.If no entry is present in the database, the key sharing protocol is invoked. As56

the first step of the protocol, a header inaccessible message is sent from theintermediate node to the source of the DML-IPSec packet (mentioned as 1 infig 4.9). The intermediate node also mentions the protocol headers that it requiresto access the body portion of this message.Unique integer codes areused for identifying different protocol headers according to the same format asdescribed in the zone map related discussion of the previous section.Uponreceiving this header inaccessible message, the source of the DML-IPSec communicationcommences the first phase of the protocol to decide whether theservice to the intermediate node can be provided or not.• Zone reorganization phaseThis first phase of the protocol is responsible for deciding the zone mapping toprovide access to intermediate nodes. This phase performs the following steps.First, the source of the DML-IPSec communication decides whether the access tothe requested protocol headers can be granted using the existing zone mapping ofthe DML-IPSec communication. If the present zone map can serve the requestedaccess, the next phase of the protocol commences. Otherwise the subsequentsteps of this current phase continue to reorganize the zone mapping.Next, the source of DML-IPSec communication decides the new zone mappingrequired for the requested access. If the requested access is possible according toa new zone map, the new zone map is sent to the destination of the DML-IPSecconnection by the source (indicated as 3 and 4 in fig 4.9). Otherwise an errormessage is sent to the intermediate node and key sharing fails. After receivingthis message the intermediate node may decide to request for access to a newset of headers. After this renegotiation message transfer, both the source anddestination update the security association and zone mapping databases.Itmay also be required to update the zone map and security associations of other57

existing intermediate nodes. In that case these intermediate nodes update theirsecurity databases.The cryptographic information for the new type II zone is derived from theexisting security association for the type II zone. Only the new type I zone mapsand cryptographic information regarding the new type I zones are communicatedin this step. This ensures that the default security parameters, derived from themanual or automatic key exchange procedure are kept constant for the type IIzone. Because of this particular zone re-negotiation phase, the end nodes of theDML-IPSec communication may start with initial configuration of only one zone(same as normal IPSec). Whenever a request, if any, from some intermediatenode comes, the zone maps are reorganized through renegotiation.We explain the zone reorganization phase with an example. Suppose, two endpointsof a DML-IPSec session commence the communication with a single typeII zone spanning over entire IP datagram using ESP in tunnel mode. The zonemap and the security databases also are configured with entry for a single typeII zone having scope over the entire datagram (indicated as 0-EOP in the zonemap ). If some intermediate node requires access to the inner IP and transportlayer header, it sends a header inaccessible message to the source nodeindicating the header requirement as Scope 12 to request access of the innerIP and transport layer header. Upon receiving this message, the source nodechecks the zone map to find out that the access is not possible according to thecurrent zone map and decides to execute the zone reorganization. The sourcenode reorganizes the zone map with one type I zone spanning over inner IP andtransport layer header and a type II zone over transport layer payload. Thecryptographic parameters used for the previous type II zone are used for thenew type II zone. The source node also decides the cryptographic parameters58

for the sole type I zone. After this zone reorganization, the new zone map andthe cryptographic information for the sole type I zone are sent to the destinationnode. Upon receiving this zone reorganization message, the destination nodealso updates its zone map and the security databases and acknowledges to thesource node.• Authentication PhaseThis phase of the key sharing protocol commences, if the service requested bythe intermediate node can be granted by the source of the DML-IPSec communication.The main responsibility of this phase is to verify the identity ofthe intermediate node to the source of DML-IPSec session.If required theDML-IPSec source may provide the authentication of itself to the intermediatenode. We propose the usage of a simple challenge-response type protocol forthe purpose of authentication. The protocol uses public key cryptography forimplementing the authentication process. All the concerned nodes, DML-IPSecendpoints and intermediate nodes, are to be provided with the public keys ofall others. In the first step, the source of DML-IPSec session may select somerandom number R, encrypt it using the public key of the concerned intermediatenode and send it to the intermediate node (indicated as 5 in fig 4.9). Uponreceiving this authentication request, the intermediate node decrypts R usingit’s private key and encrypts the same with the public key of the source node.This simple challenge-response protocol can prove identity of the intermediatenode to the source node (indicated as 6 in fig 4.9). Similar challenge responseprotocol may be used to prove identity of source node to the intermediate node.If the identity of the intermediate node is proved to the source node, the lattersignals the commencement of the next phase, else an appropriate error messageis sent to the intermediate node and the key sharing fails.59

• Shared secret establishment phaseThis phase is responsible for the establishment of a temporary shared secretbetween the source and intermediate nodes. This shared secret is to be usedas key for encrypting the actual message transfer of the DML-IPSec securityparameters at the next phase of the protocol using some symmetric encryptionalgorithm. This shared secret depends on the symmetric encryption algorithmused in the next phase of transfer of security parameters. Both the nodes mayperform some key establishment protocol like Diffie Hellman [69] to derive ashared secret. The source node may select the key and send the same to theintermediate node after encrypting with the public key of the intermediate node(indicated as 7 and 8 in fig 4.9).One point to note here is that the actualtransfer of the security parameters may involve zone map and cryptographicinformation of varying size. So the usage of symmetric cryptographic algorithmis desirable at the final phase.• Security parameter sharing phaseThe final phase of the protocol is solely responsible for actual transfer of thesecurity parameters from the source to the intermediate nodes. This phase isalso responsible for updation of security and policy databases of the intermediatenodes. This phase consists of the following steps:First, the source node constructs the message consisting of necessary informationfor the updation of the security databases of the intermediate nodes. Itgenerates a master list M containing the zone map and other common informationlike protocol, mode, SPI, DML-IPSec endpoints and DML-IPSec protectednetwork addresses. The source node also prepares individual zone lists for allrelevant type I zones. This zone specific list contains cryptographic informationsuch as encryption algorithm, encryption key, authentication algorithm, authen-60

tication key etc for that particular zone. Next,the master list and zone lists areconcatenated to form the actual message. This message is padded to meet theencryption algorithm block size (if required) and appended with the pad length.Finally this padded message is encrypted using the shared secret established atthe previous phase. This encrypted message is sent to the intermediate node(indicated as 9 in fig 4.9).Upon receiving this message, the intermediate node decrypts this concatenatedmessage and parses it to segregate the master zone map and zone specific lists.The policy database is updated for the processing of the DML-IPSec packets.SA entries are created for each of the zone specific lists. Both the outboundand inbound security databases are updated using these information.Aftersuccessful updation of the databases, the intermediate node acknowledges thesame to the source node (indicated as 10 in fig 4.9).M: {ESP,TUNNEL,gw1,gw2,nw1,nw2,spi,1}L1 {12,e1,ek1,a1,ak1}Zone List for Single type I ZoneM: {ESP,TUNNEL,gw1,gw2,nw1,nw2,spi,2}L1 {1,e1,ek1,a1,ak1}L2 {2,e2,ek2,a2,ak2}Zone List for two type I ZonesFig. 4.10: Zone map and Cryptographic message format during KeySharing ProtocolWe use the fig 4.10 to explain the message transfer at the final phase of theprotocol. As shown in the diagram, the first case transmits zone details correspondingto ESP tunnel mode with only one type I zone. The scope of the singletype I zone spans over inner IP header and TCP header and protected with encryptionalgorithm e1 with key ek1 and authentication algorithm a1 with keyak1. The second case corresponds to ESP tunnel mode with two type I zones.61

The scope of the first type I zone spans over inner IP header with encryptionalgorithm e1 and key ek1 and authentication algorithm a1 along with key ak1.The second type I zone is defined over transport layer header with encryptionalgorithm e2 and key ek2 and authentication algorithm a2 along with key ak2.• Revocation of Security ParametersOne additional responsibility of the key sharing protocol is the revocation orchange of the security parameters. The revocation of the security parametersis required, when the current security relationship of a DML-IPSec expires.Whenever the current security parameter expires or the existing DML-IPSecterminates, the source node communicates the same to the intermediate nodes.Special ICMP code is used to transfer the revocation message to the intermediatenode.The source node constructs the message representing the DML-IPSecconnection descriptor and encrypts the same with public key of the intermediatenode. This connection descriptors consist of security policy related fields suchas gateway addresses, mode, protocol, SPI etc. Upon receiving this message theintermediate nodes update their security databases, and all security associationsrelated to the current connection are deleted.• Zone Reorganization at the Intermediate Nodes:As described in the zone reorganization phase of the key sharing protocol, theend points of a DML-IPSec session may decide to renegotiate the existing zonemap definition for providing service to a new intermediate node. In that case, theinformation related to these changes has to be communicated to already existingintermediate nodes.The key sharing protocol carries out this responsibilityusing the following steps:First, a special ICMP error message is sent to the currently existing intermediate62

node to indicate this zone re-negotiation.Upon receiving this message, theintermediate node gets involved in key sharing protocol. Here the source nodeand the concerned intermediate node first get engaged in the establishment ofa temporary shared secret. Using this shared secret, the source node securelysends the new zone map to the intermediate node.We elaborate the working of this reorganization with an example. Let us assumethat the end points of DML-IPSec session initiates the communication withonly one type II zone in tunnel mode ESP. Afterwords an intermediate node,say IN1 requests for access to both inner IP header and transport layer header.To serve this request, the zone maps of the DML-IPSec are modified to onetype I zone spanning over inner IP header and transport header at the firstphase of key sharing protocol. At the last phase of the key sharing protocol,these zone and security information are sent to the intermediate node IN1. Itmay happen that after some time another intermediate node, say IN2 requestsfor access to inner IP header only.In that case the DML-IPSec endpointsmay reorganize the zone map to two type I zones, where the first type I zonecovers the inner IP header and the second type I zone spans the transport layerheader. This information is sent to IN2 at the last phase of the key sharingprotocol. Moreover, this information about zone re-negotiation is communicatedto IN1 also using the zone reorganization procedure at the intermediate node asdiscussed at the previous section.• An Example of Key SharingWe explain the working of the key sharing protocol with an example as depictedin Fig 4.11. The key sharing protocol uses currently unused ICMP type 1 forall types of communication with different code values. In our subsequent discussion,we represent ICMP message with type t and code c as . The63

ESP packetHeader inaccessible1,1Authentication Request1,2Authentication Response1,3Shared Key (K)1,4Acknowledgement1,5Searches database andFails to find entryForwards the packetSecurity Parameter1,6Acknowledgement1,7ESP packetSearches database anddecrypts the header zonesRecreate the ESP packetand forwardsSource IPSecGatewayIntermediate deviceDestination IPSecGatewayFig. 4.11: Key Sharing Protocol Exampletype signifies message type, for example for key sharing between intermediatenode and source node or the zone reorganization phase. The code specifies theparticular operation specified by that message. As shown in the figure, afterreceiving the first ESP packet the intermediate node sends header inaccessiblemessage as . The requested header access is mentioned in the body portionof that message. As the service requested by the intermediate nodecan be granted using current zone map of the source node, the next message isthe authentication challenge. The source node sends the challenge message as. Upon receiving this challenge message, the intermediate node sends theresponse to the challenge as and that marks the end of second phase. It isfollowed by the third phase of establishment of a shared secret. Here the sourcenode selects the encryption key K for the final phase and encrypts that withpublic key of the intermediate node. This information is sent as . Next,64

intermediate node acknowledges the same to source node as and the currentphase ends. In the final phase, the source node generates the necessaryzone map and zone lists and encrypts the same with the key K established inthe previous phase. This encrypted message is sent to the intermediate node as. The final message is from the intermediate node to the source toacknowledge the receipt of security parameters. After these message transfers,the intermediate node is able to handle the DML-IPSec packets for intermediateprocessing as shown in the case of the second ESP packet in the figure.4.2.3 Handling of variable Zone sizesThe zone size in ML-IPSec is fixed and the zone boundaries are represented by bytelevel definition. This restricts the access to zones with variable sizes at intermediatenodes.The lengths of the zones may vary as the length of IP and TCP headersvary between 20 bytes and 60 bytes depending upon the presence of the OPTIONfields.These OPTION fields are used by the IP and TCP protocols for carryingsynchronization and other messages. For example, the OPTION fields in IP header isused in the case of source routing option of IP packet routing, marking the addressof traversed path at the traceroute option of ping program etc. The OPTION fieldsin TCP headers are used for the synchronization of TCP window, MTU etc.Theaccess to these information carried by the OPTION fields may be required at severalintermediate nodes for their functioning. Thus, providing access to these variable sizezones of an IP datagram is important.In this section, we discuss the handling of variable zone size of the DML-IPSecprotocol in detail using the previous discussion of the overall processing of the protocol.As discussed in previous sections, we define zone as a contiguous portion of IPdatagram. Moreover, we have defined type I zones as protocol headers or combination65

of continuous protocol headers. In this way, it is possible to provide access to all upperlayer headers to the intermediate nodes without compromising on the end-to-endsecurity of the actual payload.During outbound processing, we derive the lengths of the type I zones from theprotocol header length fields as discussed in the previous section.During inboundprocessing, partial decryption of the protocol headers is employed to decide the lengthfield of the headers. After the partial decryption of the protocol headers, the lengthsof the zones are derived.For illustration purpose, we define one particular scenario, where we use ESP intunnel mode and we define two zones.The only type I zone covers the whole ofinner IP header and corresponding transport layer header, whereas the single type IIzone covers the transport layer payload. We use DES algorithm for the encryptionof the first zone and AES with 256 bits key for the encryption of the second zone.During inbound processing of both intermediate node and DML-IPSec endpoints, forhandling the variable size header length, we first decrypt the first two DES blocks,(first 16 octets). Upon successful decryption of these two blocks we derive the actuallength of IP header from the header length field, whereas the protocol field of the IPheader defines the next protocol definition. In case of UDP and ICMP the transportlayer header lengths are derived directly. In case of TCP we continue the block byblock decryption until we decrypt the 16th octet of TCP header, where we get theTCP header length value.4.3 SUMMARYIn this chapter, we have described the details of the Dynamic Multi Layer IPSec(DML-IPSec) protocol.We have redefined some of IPSec and ML-IPSec conceptsfor the DML-IPSec protocol. We have redefined the concept of zone so that packets66

with variable header length can be accommodated during DML-IPSec processing. Inorder to provide seamless access to intermediate nodes, we have classified zones astype I and type II zones.Type I zones are defined as protocol headers or propercombination of protocol headers. The type I zones are allowed to be accessed by theintermediate nodes.The other type of zone, called type II zone is defined on thepayload portion and kept secure between the endpoints of the DML-IPSec sessions.We have also redefined the security association in DML-IPSec. A single IP datagrammay be processed using different cryptographic parameters at different zones.Wepresent the key sharing protocol for sharing cryptographic information and allied zonespecific parameters between the DML-IPSec endpoints and intermediate nodes. Thiskey sharing protocol is invoked, whenever a DML-IPSec protected datagram passesthrough an intermediate node with partial DML-IPSec processing capabilities. Wealso address the issue of zones with variable lengths in case of IP datagrams withOPTION fields. In the next chapter, we discuss the implementation details of theDML-IPSec protocol in Linux environment.67

CHAPTER 5Design and Implementation of DML-IPSec in LinuxEnvironmentThe DML-IPSec protocol as described in the previous chapter addresses the issuesof key sharing with the intermediate nodes and that of accommodating packets withvariable length headers. The access to upper layer header information at the intermediatenodes is an issue when the upper layer protocol headers are encrypted as in thecase of ESP. Therefore, the DML-IPSec protocol assumes significance in case of ESPprotocol compared to that of the authentication only case of AH protocol. Hence, theimplementation is done for the ESP protocol. In this chapter, we discuss the designissues and provide the implementation details of the DML-IPSec protocol in LinuxOS. The implementation is carried out in Linux kernel version 2.6.23 [70] of RHEL 4.The existing implementation of IPSec in Linux kernel is modified to provide the multilayer processing capability. The ICMP based socket programming is used for differentmessage transfers required for the key sharing protocol. A user level interface in theform of a pseudo device driver is also implemented to update the kernel level securitydatabases from the user space key sharing utilities.We first briefly mention the implementation aspects of the IPSec protocol suitealong with special emphasis on the ESP protocol. Next, we present the implementationdetails of the DML-IPSec ESP protocol. Finally we provide the details of the test-bedsetup and the results validating the design and implementation of DML-IPSec.68

5.1 IMPLEMENTATION ASPECTS OF BASIC IPSECThe Linux kernel series 2.6.X has the implementation of IPSec protocols i.e. AH andESP built in the IP stack. User space utilities (setkey, racoon) are provided to configurethe kernel space configurations of IPSec [71]. The main modules of the native IPSecimplementation [72–74] are described in the following sections.5.1.1 Implementation of ProtocolsThe ESP and AH protocols are implemented along with the implementation of IP.The data structures for representing Security Policies (SPs) and Security Association(SAs) are crucial for the implementation of IPSec. Fig 5.1 depicts the various XFRMstructures used for the representation of the SPs and SAs.The Security Policy Database (SPD) is represented using xfrm-policy structure.The data structure for SPD consists of bundle of SAs implemented as xfrm-dst structures.The bundle of SAs consists of individual SA entries implemented as xfrm-statestructures. This xfrm-state structure holds all necessary information for a particulartransformation. This structure contains information like replay window, sequencenumber, and cryptographic information. The sharable information about encryptionand authentication algorithm are stored in xfrm-algo structure objects, while privatedata e.g., key, Initialization Vector for a particular transformation are maintained asesp-data structure object of the xfrm-state structure.5.1.2 User space utilityUser space utility setkey is used for updating the structures representing the SPs ofthe SPD. The structures representing SAs can be updated either manually by usingthe setkey utility or by automatic key exchange through the racoon utility. Racoonuses the IKE protocol for the automatic configuration of the SAs.69

struct xfrm_algo {char alg_name[64];int alg_key_len;............}struct xfrm_policy{struct xfrm_policy*next;u32 priority;struct dst_entry *bundles;u16 family;u8 type;u8 action;u8 xfrm_nr;................struct xfrm_tmplxfrm_vec[XFRM_MAX_DEPTH];}struct dst_entry{struct xfrm_state*xfrm;struct dst_entry*child;}NULLstruct xfrm_state{/*Cryptographic Information*/struct xfrm_algo*aalg;struct xfrm_algo*ealg;................void*data;/*Other Information*/struct xfrm_replay_state replay;u8 replay_window;................}struct xfrm_algo {char alg_name[64];int alg_key_len;............}struct esp_data{/* Confidentiality */struct {u8*............key;}conf;/* Integrity. */struct {u8*............key;} auth;}Fig. 5.1: SPD and SA in Linux Implementation of IPSecFigure 5.2 specifies a flow until kernel applies IPSec-SA to a packet. The numbersindicated in the figure corresponding to the events are explained below.(1) The administrator sets a policy by populating the structures representing SPsusing setkey.(2) The kernel refers to SPD in order to make a decision of applying IPSec to apacket.(3) If IPSec is required, then kernel gets the key for IPSec-SA from SAD.(4) If it fails, then kernel sends a request to get the key to racoon (in the case ofabsence of manual SA configuration).(5) The racoon exchanges the key by using IKE with the other end of the IPSeccommunication.(6) The racoon updates the structures representing SAs.(7) The Kernel can send a packet after providing IPSec security according to SAs.70

Setkey(1)(4)Racoon(6)IKE(5)Remote IPSec MachineSPD(2) (3)KernelSAD(7)Fig. 5.2: Illustration of interaction between user space utilities andKernel5.2 IMPLEMENTATION OF THE DML-IPSEC PROTOCOLWe implemented the multi-layer IPSec functionalities inside the native Linux implementationof IPSec protocol. The SA structure was updated to hold the necessarysecurity association information for multiple zones instead of a single SA of the normalIPSec. The zone mapping for different zones were implemented along-with thekernel implementation of security associations. The inbound and outbound modules ofthe IPSec endpoints were re-implemented to incorporate multi-layer IPSec capability.We also implemented necessary modules for providing partial IPSec processing capabilitiesat the intermediate nodes. The implementation details of the major modulesof the DML-IPSec protocol are described next.5.2.1 Implementation of ZonesThe concept of zone is not present in the native implementation of the IPSec-ESPprotocol. We implemented zones along with the security associations. Each zone hasa set of cryptographic parameters that may vary from zone to zone. The zone specificcryptographic information are implemented by modification of security association asexplained in the following.every ESP communication.The DML-IPSec protocol also defines a zone map forThis zone map contains information regarding number71

of zones and the scope of each zone.This zone map is consulted during the keysharing protocol and also at the beginning of packet processing module to determinethe zone boundaries of the datagrams. We implement this zone map as a DML-zonemapstructure as depicted in Fig 5.3. This DML-zone-map contains the number ofzones and the scope of all the zones are mentioned in the array of integer values.Finally, the xfrm-state structure corresponding to a ESP tunnel holds an entry for theDML-zone-map structure.#ifdef DML_IPSECstruct DML_zone_map{__u8* scope; // Scope for Zones#ifdef DML_IPSECstruct DML_esp_data{struct DML_esp_data * next;/*SA for Next Zone*/struct esp_data *value;/*Private data for that Zone*/struct xfrm_algo * aalgo,*ealgo,*calgo;__u8 no_of_zones;/*Common algo specific datafor that zone*/// Number of Zonesint Scope; /*Scope of this Zone*/};#endif};#endifFig. 5.3: Zone structure and Zone specific DML-IPSec ESP SA5.2.2 Implementation of security associationThe DML-IPSec protocol defines end-to-end security policies between two endpoints asthe information regarding any intermediate node is not relevant for packet processingat the endpoints. The security policies at the DML-IPSec endpoints are similar tonormal IPSec. Thus the security policy definition resembles that of normal IPSec. Onthe other hand, the security association information is significantly different in DML-IPSec than the standard IPSec, as the zone specific cryptographic information has tobe maintained along with every security association.In normal ESP implementation every IPSec communication has a set of securityparameters represented by the structure xfrm-state. This structure in turn stores the72

algorithm specific entries in xfrm-algo structure and the information about key, IV etcin esp-data structure. The xfrm-state structure also holds information about replaywindow and sequence number.In the DML-IPSec implementation, the xfrm-statestructure is segregated into two parts. The first one contains common informationfor all zones. The other part holds zone specific information through a pointer to anew data structure, called DML-esp-data, that holds cryptographic information for aparticular zone. As depicted in Fig 5.4, this structure has entries for algorithm specificxfrm-algo structures and also private data represented as esp-data structures.struct xfrm_algo {charalg_name[64];intalg_key_len;............}struct xfrm_algo {charalg_name[64];intalg_key_len;}............struct xfrm_policy{struct dst_entry *bundles;................}struct dst_entry{struct xfrm_state*xfrm;................struct dst_entry*child;}struct xfrm_state/*Cryptographic Information*/void *zonal_SA;/*Other Information*/u8replay_window;struct xfrm_replay_state replay;struct *DML_zone_map Map;................struct DML_esp_data{struct xfrm_algo *aalgo,*calgo,*ealgo;struct DML_esp_data*next;struct esp_data *value;}struct DML_esp_data{struct xfrm_algo *aalgo,*calgo,*ealgo;struct DML_esp_data*next;struct esp_data *value;}NULLstruct DML_zone_map{__u8* scope;__u8 no_of_zones;}struct esp_data{/* Confidentiality *//* Integrity. */}struct esp_data{/* Confidentiality *//* Integrity. */}NULLFig. 5.4: SPD and SA structures in the DML-IPSecFig. 5.4 shows the XFRM structures for ESP with one type I zone and one typeII zone. As shown in the figure, the SPD structure xfrm-policy points to SA bundlestructure dst-entry that contains SA structure xfrm-state. The xfrm-state structureholds zone map structure DML-zone-map. The zone specific cryptographic informationis stored in the structure DML-esp-data. Therefore, the number of instances of this73

structure is equal to the number of zones. As there are two zones, one type I and onetype II zone in the case of Fig. 5.4, two instances of the structure DML-esp-data ispresent. Here the first one stores cryptographic information for single type I zone andthe second one holds the cryptographic information for the type II zone.5.2.3 Implementation of the datagram processing at the endpointsIn DML-IPSec ESP implementation, at the outbound processing module, the xfrmpolicystructures representing the SPs are first searched linearly till a match occurs.If the policy is ESP, the SA is derived from the dst-entry SA bundle. Next, the DMLzone-mapstructure is scanned from the xfrm-state structure.Using the zone mapdefinition, the outgoing IP datagram is partitioned into multiple zones.Next, theDML-esp-data list of the xfrm-state structure is linearly accessed to provide cryptographicsecurity to zones. The common information for all the zones are fetched fromthe entries containing common information in xfrm-state structure.In the inbound processing of DML-IPSec ESP, first the xfrm-state structure representingSAs is derived using the selectors containing SPI and the IP address. InDML-IPSec, different cryptographic parameters are applied on different zones. Moreover,the derivation of the zone boundaries depends on the encrypted header portions.In DML-IPSec, first the zone definition is derived from the DML-zone-map structureof the xfrm-state structure and partial decryption on the type I zones is employed forthe derivation of exact zone lengths. The DML-esp-data list of the xfrm-state structureis linearly accessed to provide cryptographic information specific to zones. Thisdecryption is carried out by copying the encrypted ESP data into temporary buffer asthe authentication verification has to be performed on the encrypted zone data. Afterthe derivation of the zone boundaries, all the zones are decrypted and the originalIP datagram is re-constructed back. After the decryption of the ESP packet, SPD is74

eferred through the selector field of the SA. On successful policy verification, the IPdatagram traverses through the IP stack of the inbound machine.5.2.4 DML-IPSec processing at the intermediate nodesThe implementation of DML-IPSec processing at intermediate nodes is different fromthat of the endpoints. The inbound module decrypts the necessary type I zones togenerate a partially decrypted IP datagram so that the upper layer headers can beexamined by several networking applications of the intermediate nodes.The outboundmodule processes the partially decrypted IP datagram to reconstruct the originalDML-IPSec packet. The security policy and association databases are implementedas XFRM structures as discussed in the previous section. The kernel level structuresrepresenting the SPs and SAs are updated by the key sharing protocol.During the inbound processing, the structures representing the SPs are searchedlinearly. If the policy for an incoming DML-IPSec packet is partial DML-IPSec processing,the structures representing SAs for that transformation are fetched from theSPs. The structure representing the SA has cryptographic information for all necessarytype I zones represented as DML-esp-data structures as mentioned in the previoussections. Partial decryption of headers is employed to derive the zone boundaries followingwhich the necessary type I zones are decrypted. After successful decryption ofnecessary type I zones, the IP datagram is constructed using these partial decryptedzones. One important point to notice is that, information like sequence number of ESPheader, cryptographic synchronization information like initialization vector of differentzones varies from IP datagram to datagram. So, these information need to be savedper packet basis at the inbound processing module of the intermediate node.TheDML-IPSec implementation saves these information in the newly generated partiallydecrypted IP datagram itself. After decrypting necessary type I zones, the ESP header75

and IVs of the decrypted zones are appended after the partially decrypted IP datagramand the length field and checksums are re-computed.The partially decrypted ESP packets are encrypted back to reconstruct the DML-IPSec ESP packet at the outbound module of the intermediate nodes. Similar to theinbound module, the policy database is searched to find out the policy. In case ofpartial DML-IPSec processing, the security association entry is fetched. Unlike theDML-IPSec endpoints, where cryptographic synchronization data and ESP header sequencenumbers are derived from kernel variables and random numbers, the outboundmodule of intermediate nodes need to use the values used by the source of the packet.The appended ESP header and synchronization values from the partially decryptedIP datagram are used along with the cryptographic information saved at the zonal SAsof the outbound module to reconstruct the original DML-IPSec ESP packet.The intermediate nodes have to process an ESP packet which is defined betweentwo different DML-IPSec endpoints. Hence, the security policy database is defined fortwo other machines. The key sharing protocol is responsible for updating the securitydatabases accordingly.Another important point is that both the inbound and outbound security policyand association database entries have to be created by the key sharing protocolitself. The security association databases of both the inbound and outbound moduleshave to be populated with same cryptographic information. However, the policydatabases may vary considerably.For example, ESP tunnel mode datagrams withtype I zone spanning over inner IP header may have different entries at the outboundpolicy database than the inbound policy database.76

5.2.5 Security Databases configuration utilityThe security policies of DML-IPSec communication are dependent only on the DML-IPSec endpoints and not on the intermediate nodes. The security association on thecontrary is dependent on the information about intermediate processing as it decidesthe cryptographic parameters for different portions of the IP datagram. The DML-IPSec ESP implementation uses the default IPSec utility setkey for the updation ofthe xfrm-policy structures representing the SPs. The default IPSec configuration file isused for updating the SPD at the time of defining secure ESP communication betweentwo DML-IPSec endpoints.However, the updation of the SA entries are different from that of native IPSecimplementation, as DML-IPSec require to create zone specific SA entries at the xfrmstatestructure. The type II zone is secured between the endpoints, whereas the type Izones might be accessed by the intermediate nodes. Thus, the intermediate nodes donot participate in the establishment of SA corresponding to type II zone. The DML-IPSec protocol also uses the default IPSec mechanisms (setkey and racoon utilities) fordefining the SA between two endpoints, i.e. the SA for the type II zone. However, theSA for type I zones which are specifically introduced in DML-IPSec processing needsto be updated through a separate configuration file and an interface. The configurationfile contains the security parameters corresponding to type I zone SAs. The informationin the configuration file is communicated to the kernel space through a pseudo characterdevice driver [75].This device driver is also used by the key sharing protocol forupdating zone specific SA configuration at the xfrm-state structures, representing SA.The esp-init-state function of the native kernel implementation, invoked by the defaultIPSec configuration utilities, is modified to create necessary entries for all type I zonesand sole type II zone at the xfrm-state structure representing SA.Figure 5.5 shows the steps involved when kernel applies zonal SA according to77

Setkey DML−IPSec utility Key Sharing Protocol Racoon IKE Remote IPSec Machine(6)(2)(10)(5)(1)Pseudo Character(11)Device deriver(8) (7)SPD(3)Kernel(9)(12)(4)SADFig. 5.5: Illustration of interaction between user space utilities andKernel in DML-IPSecDML-IPSec definition.The numbers indicated in the figure shows the sequence ofevents that are described below:1. The administrator sets a policy to structures representing SPs by using setkey.2. The DML-IPSec user space utility updates the type I zone specific informationto the pseudo character device driver.3. Kernel refers to SPD in order to make a decision of applying IPSec to a packet.4. If DML-IPSec is required, then kernel get the Key for zonal SAs from SAD.5. If it fails to get the key, then kernel sends a request to get the zone II Key toracoon (in the case of absence of manual SA configuration).6. The racoon exchanges the Key by using IKE with the other end of the IPSeccommunication.7. The racoon updates the structures representing SAs.8. The pseudo character device driver is consulted by the SA updation functionand zonal SAs are created.9. The kernel can send a packet after providing DML-IPSec security according toSAs.78

10. Key sharing protocol sends the new zone specific information to the pseudocharacter driver11. Pseudo device driver updates the SAs according to new zonal information12. The kernel can send a packet after providing DML-IPSec security according toSAs negotiated by key sharing protocol.5.2.6 Key Sharing ProtocolThe key sharing protocol is one of the major components of the DML-IPSec protocol.In the following discussion, we present the implementation details of the key sharingprotocol. The key sharing protocol consists of some user space utilities, correspondingkernel space components and a mechanism for communication between the two. Theuser space utilities carry out the invocation of the protocol, updation of the securityassociation databases and ensure the consistency of zone mapping and zone specificcryptographic information through all concerned nodes of the DML-IPSec communication.At the kernel level, pseudo character device driver is implemented to populatethe kernel space data structures with the data received from the user space.The main goal of the key sharing protocol is to share the zone mapping and cryptographicinformation from the source of DML-IPSec communication to an intermediatenodes. In this regard, the key sharing protocol implements a series of message transfersbetween the source and intermediate node of DML-IPSec communication. Themessages are transferred using ICMP packets. The protocol employs currently unusedtype and code field of the ICMP protocol [5,76] to carry out the communication.The payload carries the actual message corresponding to the pair of theICMP packet. The receiver of the ICMP packet decodes the message according to the pair of the ICMP header. The payload of the ICMP messages are ofvarying size as it may contain encrypted zone map, encrypted authenticated data etc.79

The sender module has to update necessary header fields like length, checksum etc forconstructing a legitimate ICMP packet.Apart from sharing the zone mapping and cryptographic information betweenDML-IPSec source and intermediate nodes, the key sharing protocol also communicatesthe same between the source and destination nodes during the zone reorganizationphase. The key sharing protocol implements the zone reorganization messagesusing ICMP packets.For authentication, the protocol uses pari library implementation of RSA encryptionalgorithm [77, 78]. Linux operating system default PRNG rand is used for thegeneration of PRNG used in the protocol. We have used DSA encryption algorithmfor the encryption of the zone specific cryptographic information at the final phase ofkey sharing. However all these cryptography related methods are implemented in amodular way so that replacement by user-specific routines can be made seamlessly.After successful key transfer, the key sharing protocol needs to update the securitydatabases of the Linux kernel. In the case of end nodes, Security Association Databasehas to be modified in accordance with the zone map definition configuration files.Here the pseudo character device driver is used to update kernel level data structureas discussed in the previous discussion. However, in the case of intermediate nodes,the security association as well as Security Policy Database need to be updated. Thesecurity parameter information is first parsed to segregate the policy database andassociation database information. The key sharing protocol has to update both theinbound and outbound security databases. All the relevant information for updatingthese databases are communicated at the final phase of the key sharing protocol itself.Fig 5.6 depicts the overall high level interaction of the DML-IPSec implementation.As shown in the diagram, the user space key sharing protocol uses the pseudo characterdevice driver [75] for communicating to the kernel level implementation of DML-IPSec80

KERNEL SPACESA BundleSASPDZonal SAZonal SAPseudo CharacterDevice DriverNULLNULLDML−IPSecConfiguration andKey sharing ProtocolSetkey andRacoon utilityUSER SPACEFig. 5.6: High level Interaction of DML-IPSec implementationprocessing.5.3 EXPERIMENTAL VALIDATION OF THE DML-IPSEC PROTO-COLThis section presents experimental validation of the DML-IPSec protocol. We studyseveral intermediate applications that require access to different fields of the IP, TCP/UDPheaders. The standard IPSec protocols-AH and ESP-restrict the functioning of theseapplications at intermediate nodes as they require access to the transport layer andupper layer headers. For example, in case of tunnel mode ESP, access to the inner IPheader is difficult as it remains encrypted. The access to the fields of the TCP andUDP headers is also difficult because of similar reasons. The details of the conflictsbetween intermediate processing and IPSec security is discussed in chapter 3. We willbriefly mention the header fields required for them in the Table 5.1. Next, we present81

three zone mappings according to DML-IPSec ESP definition and show that the problemof accessing the headers at intermediate nodes can be overcome by using suitablezone mapping.Table 5.1: Applications, their use and the headers of the IP datagram accessedby themApplication Responsibility Header Fields usePerformance enhancement proxy Performance Enhancement for TCP IP, TCP HdrAppl. Proxy Performance gain for Web IP, TCP and HTTP HdrRouters with Traffic Eng. Providing QoS based service IP, TCP or UDP HdrPacket filtering Firewall Firewalling IP,TCP,UDP HdrApplication layer Firewall Firewalling IP,TCP, UDP,application HdrTraffic analysis Load/traffic control IP,TCP, UDP,application HdrWe consider three different zone mappings for ESP tunnel mode, designated bydifferent cases as follows:• Case I : Single type II Zone.– Type I zone : Nil– Type II zone : Spans entire inner IP datagram• Case II: Two zones with one type I and one type II zones– Type I zone: Single type I zone spans inner IP header and transport header.– Type II zone : Spans entire transport layer payload.• Case III: Four zones with maximum of three type I zones and one type II zone– First type I zone spans the inner IP header.– Second type I zone spans the transport layer header.– Third type I zone spans the application layer header.(For standard applicationsotherwise void)82

– Type II zone : Spans entire application layer payload for standard applicationor the entire transport layer payload.In the table we mention about the accessibility of header fields and thus workingof different intermediate applications for different zone mapping mentioned above.Table 5.2: Working of different intermediate application in different zonemappingsHeader FieldsApplication that can functionCase I No access No application can functionCase II Inner IP and Transport hdr PEP,Routers with QoS,Packet Filter FWCase III Inner IP,transport and application hdr PEP,Appl Proxy,Routers,Firewall,Traffic analysisFrom table 5.1 we observe that all of the intermediate services require to accessoriginal IP header and transport layer header and some of them require access toapplication layer header too. Table 5.2 shows that all these requirements can be metby one of the three typical zone mapping listed above as type II and III. It is to benoted that all these services are capable of processing packets with variable size TCPand IP header. But in the presence of the existing ML-IPSec implementation, theyfail to get access to the OPTION fields.In the next section, we provide experimental verification of the claims of the DML-IPSec protocol with three different applications running in the intermediate nodes.First, we use packet filtering firewall to show the accessibility of the inner IP andtransport layer header followed by application level firewall, which requires accessto IP, TCP and HTTP header. Next, we validate our claim of handling sessions withvariable zone sizes by providing access to OPTION fields of IP packets. The IP packetshaving option fields have variable sized zones. We demonstrate it using a ping programwith record routing feature. The presence of the IP address of the intermediate node at83

the record route output validates our claim of accessibility of OPTION field of inner IPheader at intermediate node. In the absence of the DML-IPSec protocol, the firewallused to drop packets for not being able to retrieve the required headers. Similarly,the record route output did not show the IP addresses of the intermediate nodes thatdemonstrated the inaccessibility of the OPTION fields of the IP datagram.5.3.1 Test bed setupClient(C11)IPSecGateway(GW1)IntermediateDevice( IN )IPSecGateway(GW2)Client(C21)140.1.4.1 140.1.4.30 140.1.4.31 140.1.4.13 140.4.4.13 140.4.4.31 140.4.4.30 140.4.4.1(eth0) (eth1) (eth0) (eth0) (eth1) (eth0) (eth1) (eth0)Fig. 5.7: Test bed Set upFigure 5.7 shows the testbed setup for the implementation of DML-IPSec. Wehave configured two Linux servers as the ESP gateway (labeled as GW1 and GW2 infigure). We have configured another Linux server (labeled as IN) as the intermediatenode in between GW1 and GW2 that run the different intermediate applications.All these Linux servers are installed with RHEL-4 with the kernel version 2.6.23.The gateway machines GW1 and GW2 have the DML-IPSec implementation for ESPprotocol, whereas the intermediate node IN have the capabilities for partial DML-IPSec processing. We have connected two client machines at the gateway machines(labeled as C11 and C21). The IP addresses of the clients and servers are shown inFig 5.7.5.3.2 Validation with respect to intermediate servicesIn this section, we provide the details of the experiments on DML-IPSec protocol andthe corresponding results in the presence of the following intermediate applications.For all these subsequent experiments, we configure a DML-IPSec ESP tunnel between84

GW1 and GW2. The tunnel is defined for the entire subnet of the GW1 and GW2that also covers the clients C11 and C21. The relevant applications described next arerun in the intermediate node IN.Packet Filtering FirewallWe carried out experiments for validating our solution with respect to statefulpacket filtering using iptables firewall [79, 80] at IN for FTP session between C11and C21. We have chosen FTP as it uses TCP protocol in its transport layer. TheFORWARD chain of the firewall is configured to allow only FTP packets between C11and C21. The default policy is set to DROP packets implying that in the event ofthe firewall not being to access the FTP port information encoded in the port fieldsof the TCP header, the packet will be dropped. We have also configured the iptablesfirewall to log all the traffic flowing through the forward chain. The GW1 and GW2are configured with two zone DML-IPSec ESP in tunnel mode, whereas the single typeI zone spans the inner IP header and transport layer header. The single type II zonespans over the transport layer payload. The first zone is protected with DES algorithmwith HMAC-MD5 authentication and second zone is protected with AES encryptionand HMAC-MD5 authentication.At the first phase of our experimentation, we disable the execution of the user levelmodule of the key sharing protocol at IN. Now we initiate a FTP request from C21 toC11. The FTP session fails to establish.At the next phase, we start the key sharing protocol with the request for the typeI zone spanning over IP and Transport layer header.After successful execution ofthe key sharing protocol, we again initiate a FTP session from C21 to C11. Here thesession is established.In the first phase, the FTP session could not establish as the iptables firewallmodule is not able to access the inner IP and TCP headers of the FTP packet. Fig-85

ure 5.10 depicts the screendump of the relevant kernel log message entries at the nodeIN corresponding to the ESP packets seen at the intermediate node. We can noticeSRC, DST and PROTO fields in all the three packets. The SRC and DST representIP addresses of GW2 and GW1 respectively. ESP value at the PROTO field signifiesthat these packets are of ESP protocol from GW2 to GW1. From the SRC and DSTfields, it can be observed that all the packets are sent only from GW2 to GW1. Thesepackets corresponds to the ESP encapsulated TCP SYN packets from C21 to C11.These packets are retransmitted a number of times due to the non-receipt of TCP acknowledgmentpackets corresponding to the TCP SYN packets. The acknowledgmentpacket not received as the firewall could not decrypt the ESP encapsulated TCP SYNpackets, resulting in dropping the SYN packets at IN itself.In the second case, we configure the key sharing module to request the parametersfor zone request up to transport layer header.The FTP access is granted as theaccess to the inner IP and TCP header could now be possible at the iptables firewallmodule of IN. Figure 5.9 depicts the screendump of the relevant kernel log messageentries at the node IN. We can see TCP packets flowing between C11 and C21 inboth directions as the intermediate node can decrypt the type I zone. The first threepackets correspond to the 3-way TCP handshake between C21 and C11, whereas lasttwo packets represent FTP traffic transfer(Port No 21).Fig. 5.8: ESP packet corresponding to TCP request at the forwardingchain of firewall at INApplication layer firewall86

Fig. 5.9: Partially decrypted TCP packet corresponding to FTP requestat the forwarding chain of firewall at INThe following experiment validates our claim of accessibility of application layerheaders at the intermediate nodes. We configure string match extension module of theiptables firewall for allowing application layer header access at the intermediate node.We configure a web server in the C11 with address c11.org. We initially configure thepolicy as drop all packets at IN and then add required rules for allowing only HTTPtraffic from c11.org.In the first case, with the key sharing module disabled, the access to C11 from C21is denied. Figure 5.10 depicts the screendump of the kernel log message entries at INcorresponding to this case. Three ESP packets can be seen at the intermediate node.These messages correspond to the ESP encapsulated SYN packets of the 3-way TCPhandshake from C21 to C11 that are executed prior to the GET packet of the HTTPprotocol.Since the ESP packets can’t be partially decrypted, the firewall moduleDROPs these packets.In the second case, we configure the key sharing module to request parameters forzones up to application layer header for HTTP traffic. The web access is granted asthe access to the HTTP header (carrying the web address c11.org) is possible at theiptables firewall module of IN. Figure 5.11 depicts the screendump of the kernel logmessage entries relevant to this case. Here we can see TCP packets flowing between87

C11 and C21 as the intermediate node can decrypt the type I zone. The first threepackets correspond to the TCP handshake prior to the HTTP GET message transfer,whereas last two packets represent web traffic transfer (Port No 80) containing theweb access as a part of the HTTP header.Fig. 5.10: ESP packet corresponding to Web request at the forwardingchain of firewall at INFig. 5.11: Partially decrypted TCP packet corresponding to Web requestat the forwarding chain of firewall at INSessions with variable zone sizesOur next experiment validates the ability of the DML-IPSec protocol to handlevariable zone size datagrams. One scenario in which an IP datagram can have variablezone size is when the IP header has the OPTION fields present. Depending on thenumber of such OPTIONs, the size of zone corresponding to the IP header also varies.We use ping application with the traceroute option (ping -R ip-addr) between C21and C11 to demonstrate the capability of handling variable zone sizes. The tracerouteoption when enabled inserts the IP addresses of the intermediate nodes through which88

Fig. 5.12: Ping Traceroute Outputthe packet passes through before reaching its destination. This requires access to theOPTION fields of the IP headers of the ICMP packets as the addresses are insertedin the OPTION fields. We initialize tunnel mode ESP communication between GW1and GW2 with two zones where the single type I zone spans inner IP header andtransport layer header and type II zone covers transport layer payload. We executethe ping application from C21 for C11. The result of the execution is depicted in theFig 5.12. At every line, the round trip path for the packet is displayed. As seen inthe Fig 5.12, the first output does not have the IP address of the intermediate nodeIN, i.e., 140.1.4.13 in the list of IP addresses traced in the route of the ping packet.But, second packet onwards the entries for IN are present in the output of the pingapplication, as the key sharing protocol gets executed immediately after receiving thefirst ESP packet at IN.We further explain the scenario with the help of Fig 5.13. This is the screen dumpof wireshark packet capturing utility [81] running at the intermediate node duringthe execution of the ping application discussed above.The first two ESP packetscorrespond to the first output of the ping application when the intermediate node IN89

Fig. 5.13: Wireshark output for the Ping Application on eth1 interfacedoes not have the cryptographic information for processing the upper layer headersfor making entry for own IP address. The IP addresses displayed at the start nodeare carried through the OPTION field of the inner IP header of the DML-IPSec ESPpacket in tunnel mode. The ICMP messages 3-9 in Fig 5.13 correspond to key sharingprotocol execution. These messages are ICMP type 1 messages (used in the key sharingprotocol) that are not defined in the Wireshark application. Hence, they are markedas obsolete or malformed ICMP packets. On completion of the key sharing protocol,between IN and GW1 the cryptographic parameters are loaded in the intermediatenode IN. The packet number 13 in Fig 5.13 is an ICMP message generated afterpartial decryption of an ESP message generated between GW1 to GW2 correspondingto a ping reply from C11 to C21. As we are capturing the IP packets on the eth1interface of IN, we can see the ESP packets coming from the GW2 corresponding tothe ping request from C21 as seen in packet number 12 in Fig 5.13. But, the ESP90

packet from GW1 corresponding to the ping reply from C11 gets partially decryptedby the DML-IPSec processing module before being captured on the eth1 interface.Thus, we can observe the partially decrypted ping reply as seen in packet number 13in Fig 5.13.Fig. 5.14: Layout of captured ESP Packet on eth0 interface of INWe use the same ping application for describing the layout of the partially decryptedIP packets at the intermediate node IN. We execute a ping application atC11 for C12. The figure 5.14 depicts the output of wireshark application at the eth0interface of IN, whereas the figure 5.15 depicts the output of wireshark applicationat the eth1 interface. The screenshot is taken after the execution of the key sharingprotocol. The DML-IPSec is configured for ESP in tunnel mode with two zones sameas the previous experiment of traceroute. The first zone spans over inner IP headerand transport header and secured with DES algorithm. The second zone spans theentire transfer layer payload and encrypted with AES algorithm with 128 bits key91

Fig. 5.15: Layout of partially decrypted ESP packet on eth1 interfaceof INlength. Both the zones are secured with encryption only mode without the presenceof authentication.We executed the ping application from C11 with 32 bytes of ICMP payload. Thismakes a 60 byte IP packet (20 bytes IP header + 8 bytes ICMP header + 32 bytesof ICMP payload). This ICMP packet is processed by GW1 and produces a two zoneDML-IPSec ESP packet in tunnel mode as an IP packet of 132 bytes. The single typeI zone (zone1) spans the inner IP header and transport layer header and the type IIzone (zone 2) spans the transport layer payload. The ESP packet thus formed consistsof 132 bytes of data. The ESP packet consists of outer IP header of 20 bytes followedby 8 bytes of ESP header, 40 bytes of zone1 data and 64 bytes of zone 2 data. Thezone1 data is partitioned as 8 bytes of IV and 32 bytes of encrypted data.(28 bytes ofinner IP header and ICMP header is padded to meet 8 byte block size requirement ofDES algorithm ). The zone2 is partitioned as 16 bytes of IV and 48 bytes of encrypted92

data. (32 bytes of ICMP data and 1 byte next header field is padded to meet 16 byteblock size requirement of AES algorithm).Figure 5.15 is the capture at the eth1 of the GW1. At IN, the ESP packet depictedin Fig 5.14 is partially decrypted and the zone1 is decrypted. After that, the externalIP header, ESP header and the IV for the zone 1 is removed and we form an IP packetwhere the IP and ICMP headers of the original ICMP message from C11 is used.This can be seen in the IP header and ICMP header portion of the figure 5.15. Onepoint to note is that we have not discarded the external IP header, ESP header, IVfor zone1 as these information will be required for generating back the original ESPpacket. Rather, we append these headers after the encrypted zone 2 portion. We alsodon’t discard the padding portions of the decrypted zone1. We update the IP headerand ICMP header to construct an ICMP packet with the original ESP header lengthas can be seen in figure 5.15. This can be observed by comparing the Total Lengthfields at the Fig 5.14 and Fig 5.15.5.3.3 Overheads of DML-IPSecThe DML-IPSec protocol allows the intermediate devices to access the various protocolheaders as well as handles sessions with variable zone sizes. However, it introducesadditional processing overhead and also increases the packet size as compared to IPSecand ML-IPSec. We first analyze the effect of increase in the packet length followed bythat of extra processing overhead.DML-IPSec increases the packet size compared to the standard IPSec. Such anincrease in IP packet size also takes place in the case of ML-IPSec. The increase in IPdatagram length depends on the number of zones. As the number of zone increasesthe overhead also increases because of the presence of multiple authentication trailers,zone wise padding, presence of separate IVs for different zones. For example, when93

compared to standard IPSec, DML-IPSec with two zones will have bigger cipher textbecause of the presence of two separate IVs and separate padding for two zones. Moreover,the presence of two separate authentication trailer will increase the overhead.We have enumerated the effect of increase in packet length due to the DML-IPSecprotocol using a datagram with IP header length IL, Transport layer header length TLand transport layer payload of length n. We have computed the packet length in fourdifferent cases for IPSec and for the corresponding cases of DML-IPSec. Each of thesefour cases consists of AH and ESP in transport and tunnel modes. For IPSec we haveconsidered HMAC-MD5 for authentication and AES for encryption. For DML-IPSec,we have considered two zones. In the case of transport mode, the first zone representsthe transport layer header and the type II zone spans the transport layer payload. Inthe case of tunnel mode, the sole type I zone covers inner IP header and transportlayer header whereas the sole type II zone spans over transport layer payload. Boththe zones are protected with authentication and encryption. The authentication forboth the zones is provided using HMAC-MD5. The encryption is provided using DESand AES for zone1 and zone2 respectively. Table 5.4 shows the overhead in case of IPto IPSec and IP to DML-IPSec in all four cases.Table 5.3: Packet Length Comparisons (Bytes) between IPSec andDML-IPSec with two zones for AH and ESP in transport and tunnel modes.IPSecDML-ESP with two zonesAH Transport Mode 24+IL+TL+n 36+IL+TL+nAH Tunnel Mode 44+IL+TL+n 56+IL+TL+nESP Transport Mode 36 +IL+⌈(TL +n +2)/16⌉*16 56+IL+⌈(TL+1)/8⌉*8+⌈(n+2)/16⌉ *16ESP Tunnel Mode 56+⌈(IL+TL +n +2)/16⌉*16 76+⌈(IL+TL+1)/8⌉*8+⌈(n+2)/16⌉*16We observe that, the use of IPSec protocol adds an overhead ranging from 24 to 73bytes. The DML-IPSec AH adds an extra 12 bytes overhead whereas the DML-IPSec94

Table 5.4: Packet Length Overhead (Bytes) in IPSec and DML-IPSec protocolwith two zones over standard IP protocol. The numbers in square bracketsdenote the min and max values possible for that particular caseIP to IPSec IP to DML-IPSec with two zonesAH Transport Mode 24 36AH Tunnel Mode 44 56ESP Transport Mode [38,53] [59,81]ESP Tunnel Mode [58,73] [79,101]ESP increases the overheads by maximum 28 bytes. For an average IP datagram of 500bytes, this is only a 3% to 5% increase. The DML-IPSec protocol does not introduceany extra overhead compared to ML-IPSec. The same overheads shown in table 5.4are applicable for ML-IPSec also. We consider the same datagram with IP headerlength IL, Transport layer header length TL and transport layer payload of length nfor ML-IPSec ESP processing in ESP tunnel mode with two zones same as the caseof DML-IPSec described above. The IP datagram will get partitioned in exactly sameway of the DML-IPSec and thus introduce same overheads in packet size. However,the ML-IPSec protocol would not be able to handle IP datagrams with variable lengthIP and transport layer headers.We conduct experiments to study the performance on the test bed shown in Fig 5.7.We execute ping program from C11 to C21 and use the Round Trip Time (RTT) tocompare the processing delay in the following network configurations between GW1and GW2 as described below.1. IP: The two gateways are configured without security protocol.2. IPSec : This is native ESP implementation provided with RHEL 4 Linux distribution.We have configured tunnel mode ESP between GW1 and GW2 withAES algorithm with 128 bit key and without any authentication.3. ML-IPSec-1-zone: This is our ML-IPSec ESP implementation in tunnel mode95

and has a single type II zone with AES encryption and without authentication.4. ML-IPSec-2-zones: This is our ML-IPSec ESP implementation in tunnel modewith 2 zones where the single type I zone spans over the IP and transport layerheader whereas the single type II zone spans over the transport layer payload.We use DES encryption for type I zone and AES for type II zone. This is animplementation where we use fixed zone sizes with the first zone spanning overinitial 28 bytes of IP datagram whereas the second zone spans over the rest ofthe portion.5. DML-IPSec-2-zones: This is our DML-IPSec ESP implementation in tunnelmode with variable zone length. The type I zone covers the IP and transportlayer header and the entire transport layer payload is covered by the type IIzone. The length of the IP header can vary depending on the OPTION fieldand hence the size of zone1. Zonal cryptographic information is same to theprevious case.1.5time in sec −−−>10.501 2 3 4 5Fig. 5.16: RTT of ping obtained in different communication scenariosbetween GW1 and GW2. (1) Normal IP communication (2) NativeIPSec ESP tunnel mode (3) ML-IPSec ESP tunnel mode without anytype I zone (4) ML-IPSec ESP tunnel mode with one type I zone andone type II zone of static length (5) DML-IPSec ESP tunnel mode withone type I zone and one type II zone of variable length96

Figure 5.16 depicts the round trip time of ping application with 56 bytes of ICMPpayload sent from C11 to C21. We have taken the mean RTT value over 10 iterationsof ping program where each iteration consists of 100 ping packets. We can see around50% increase in RTT from normal IP to native IPSec. Next we can see around 12%increase in RTT over native IPSec in case of ML-IPSec with only one type II zone.This increase is due to the extra processing and access to data structures involvedin ML-IPSec. There is around 30% increase in RTT for ML-IPSec with two staticzones compared to one static zone. This increase is due to invocation of two differentencryption algorithms and also because of partioning of single IP datagram into twozones. Finally, we notice 10% increase in RTT when we move to DML-IPSec with twodynamic zones from two static zones in ML-IPSec. This is because of the on-the-flyderivation of the zone length related processing delay.The experiment measures the processing delay due to the dynamic zone boundaryfeature of DML-IPSec protocol. The intermediate node in all five cases functions onlyas a forwarding device without doing any DML-IPSec partial processing and otherintermediate processing. Any extra processing activity by an intermediate node mayadd to further delays. Next, we try to figure out the processing delay incurred dueto the partial DML-IPSec processing along with packet filtering firewalling operation.We execute ping from C11 to C21 with 56 bytes of ICMP payload in the followingnetworking configurations:1. ML-IPSec-2-zones: This is our ML-IPSec ESP implementation in tunnel modewith 2 zones where the single type I zone spans the IP and transport layer headerwhereas the single type II zone spans the transport layer payload. We use DESencryption for type I zone and AES for type II zone. This is an implementationwhere we use fixed zone sizes with the first zone spanning over initial 28 bytesof IP datagram whereas the second zone spans over the rest of the portion.97

2. DML-IPSec-2-zones: This is our DML-IPSec ESP implementation in tunnelmode with variable zone length. The type I zone covers the IP and transportlayer header whereas the entire transport layer payload is covered by the typeII zone. Zonal cryptographic parameters are same as the previous case.In the first experiment, we observed the increase in RTT in case of ML-IPSecwhereas, the second experiment compares the same for DML-IPSec. Both the experimentsconsists of two phases. In the first phase of the first experiment, we configurean ESP tunnel between GW1 and GW2. The intermediate node IN is not configuredwith the cryptographic protection. The firewall at IN is configured to allow ESP trafficbetween GW1 and GW2. In the second phase of the first experiment, we configurean ESP tunnel between GW1 and GW2. The intermediate node IN is configured todecrypt the data contained the single type I zone. The firewall at IN is configured toallow ICMP packets between C11 and C21.The second experiment is almost similar to the first with only a difference that weuse our DML-IPSec implementation with dynamic zones instead of static ML-IPSec.The RTT values in these two experiments are depicted in fig 5.17 as 1 and 2. Theblack colored bar represents the RTT in the absence of intermediate IPSec processingwhereas the other one represents the RTT in presence of partial IPSec processing atIN. We observe around 20% of RTT increase in case of intermediate node processingfor the first experiment. This increase is around 24% in the second experiment, whereDML-IPSec with variable zones are used.5.4 SUMMARYIn this chapter, we have described the design and implementation details of the DML-IPSec protocol inside Linux kernel. We have provided the details about the new datastructures and their relationship with the existing kernel data structures. We have also98

1.81.61.4time in sec −−−>1.210.80.60.40.201 2Fig. 5.17: Comparison of RTT in presence of intermediate processingobtained in different communication scenarios between GW1 and GW2.(1) ML-IPSec ESP tunnel mode with one type I zone and one type IIzone of static length (2) DML-IPSec ESP tunnel mode with one type Izone and one type II zone of variable lengthprovided experimental validation of our implementation in the context of intermediateapplications. Next, we have provided an analysis about the packet size overhead andthe overhead due to the change in packet processing of DML-IPSec.99

CHAPTER 6SUMMARY AND CONCLUSIONThe research work begins with a study of security requirements in the TCP/IP networksand a review of approaches for providing communication security at differentlayers of the TCP/IP protocol stack. The IPSec protocol suite, has become a standardfor providing IP layer communication security services. The IPSec protocol suite, however,conflicts with the working of several intermediate applications for the purposeof performance enhancement and security of the overall network. TCP-PEP, Applicationlevel proxy, firewalls are some of these intermediate applications. Using transportfriendly ESP, TLS instead IPSec, Splitting Single IPSec tunnels into multiple tunnels,using Multi layer IPSec(ML-IPSec) are among the proposed solutions in literature toaddress this issue of conflict. The ML-IPSec protocol solves this interoperability problemwithout violating the end-to-end security for the data or exposing some importantheader fields unlike the other solutions. The ML-IPSec protocol applies different securityto different portions of an IP datagram represented as zones. The intermediatenodes can decrypt/encrypt the relevant header portions of an IP datagram and performtheir functionalities. However, the ML-IPSec protocol suffers from the problemof requirement of pre-configuration of cryptographic parameters and zone mapping fordifferent zones of an IP datagram. The requirement of static zone boundary can notcater TCP/IP packets with variable header sizes.Our extension to the ML-IPSec protocol, called Dynamic Multi Layer IPSec (DML-IPSec) proposes a multi layer variant with the capabilities of dynamic zone configurationand sharing of cryptographic parameters between IPSec endpoints and inter-100

mediate nodes.It also accommodates IP datagrams with variable length headers.The DML-IPSec supports the AH and ESP protocols of the conventional IPSec withsome modifications required for providing separate cryptographic protection to differentzones of an IP datagram. This extended protocol redefines zone, zone map andSecurity Association (SA) and also redefines IPSec processing.The DML-IPSec protocol has two basic components. The first one is for processingof datagrams at the endpoints as well as intermediate nodes. The second componentis the key sharing protocol. The DML-IPSec endpoints are responsible for generatingDML-IPSec datagrams from IP datagram and vice versa using the zone mappingand zone specific cryptographic information. The intermediate nodes are responsiblefor partial encryption/decryption of upper layer headers for intermediate processing.The key sharing protocol for sharing zone related cryptographic information amongthe intermediate nodes is responsible for dynamically enabling intermediate nodes toaccess zonal information as required for performing specific services relating to qualityor security.The key sharing protocol consists of four phases for reorganization ofzones, authentication verification of the intermediate node, establishment of sharedsecret and actual transfer of the cryptographic information to the intermediate nodeusing the shared secret.The DML-IPSec for ESP protocol is implemented according to the new definitionof zones along with the key sharing protocol. RHEL version 4 and Linux kernel version2.6.23.14 are used in this implementation for IPv4.Detailed experiments are carried out for validating the solution with respect tofirewalling services.Stateful packet filtering using iptables is employed along withstring match extension. The DML-IPSec protocol introduces some processing overheadand also increases the datagram size as compared to IPSec and ML-IPSec. It increasesthe datagram size compared to the standard IPSec.However, this increase in IP101

datagram size is present in the case of ML-IPSec as well. The increase in IP datagramlength depends on the number of zones. As the number of zones increases this overheadalso increases. We have used RTT of ping program to determine the processing delaydue to DML-IPSec processing at endpoints and intermediate nodes. We observe around10% increase of RTT time due to the DML-IPSec processing.6.1 CONTRIBUTIONS OF THE WORKThe contributions of the research work carried out as a part of this thesis are as follows:First, it analyzes the problems of Intermediate services in the presence of endto-endsecurity protection of the IPSec protocol.It also reviews the solutions forthis problem of interoperability between IPSec and intermediate services viz. TCP-PEP, routers providing QoS, firewalling devices etc. The issues in using Multi LayerIPSec(ML-IPSec), especially in mobile environments are highlighted.Next, an extension to ML-IPSec is proposed to sort out these issues. Called DML-IPSec, this extension redefines some of the IPSec and ML-IPSec primitives and processingto achieve this. A new protocol for dynamically sharing cryptographic parametersto intermediate nodes is also proposed. It is capable of dynamic modification of zonesand sharing of cryptographic parameters between endpoints and intermediate nodesusing a key sharing protocol. The DML-IPSec also accommodates datagrams with variableheader lengths. The above mentioned features enable any intermediate node todynamically access required header portions of any DML-IPSec protected datagrams.Consequently they make the DML-IPSec suited for providing IPSec over mobile anddistributed networks. A complete implementation of DML-IPSec ESP protocol is alsocarried out as a part of the research work.Finally, experimental validations of the the research work are provided. It alsoprovides experimental data for the calculation of overheads due to the extra processing102

of DML-IPSec and also that of packet size. The DML-IPSec provides the dynamicsupport for QoS and security services without any significant extra overhead comparedto that of ML-IPSec.6.2 SCOPE FOR FURTHER RESEARCHThe research work carried out focuses on providing Dynamic Multi Layer IPSec onIPv4 networks. Work can be directed towards the adaptability of the current solutionfor IPv6 scenario. The extension headers of the IPv6 header are responsible for manyQoS and routing functionalities. In ESP tunnel mode scenario, all these extensionheaders remain encrypted between the IPSec endpoints.Thus, intermediate nodesmay require to dynamically access these extension headers for performing QoS androuting related functionalities. Moreover, IPv6 headers may contain any number ofextension headers.Thus the requirement of accommodation of IPv6 packets withvariable extension headers is also important.103

BIBLIOGRAPHY[1] D. Clark, “The Design Philosophy of the DARPA Internet Protocols,” ACM SIGCOMMComputer Communication Review, vol. 18, pp. 106–114, Aug. 1988.[2] J. Postel, “Internet Protocol,” RFC: 791, 1981.[3] J. Postel, “Transmission Control Protocol,” RFC: 793, 1981.[4] J. Postel, “User Datagram Protocol,” RFC: 768, 1980.[5] J. Postel, “Internet Control Message Protocol,” RFC 792, Sep. 1981.[6] S. Bellovin., “Security Problems in the TCP/IP Protocol Suite,” ACM SIGCOMMComputer Communication Review, vol. 19, pp. 32–48, Apr. 1989.[7] O. Adeyinka, “Internet Attack Methods and Internet Security Technology,” Proc. of theSecond Asia International Conference on Modelling and Simulation, vol. 19, pp. 77–82,2008.[8] M. de Vivo, G. O. de Vivo, and G. Isern, “Internet security attacks at the basic levels,”ACM SIGOPS Operating Systems Review, vol. 32, pp. 4–15, Apr. 1998.[9] R. K. Marco de Vivo, Gabriela O. de Vivo and G. Isern, “Internet vulnerabilities relatedto TCP/IP and T/TCP,” ACM SIGCOMM Computer Communication Review, vol. 29,pp. 81–85, Jan. 1999.[10] A. Bhimani, “Securing the commercial Internet,” Communications of the ACM, vol. 39,pp. 29–35, June 1996.[11] W. Ford, Computer Communications Security – Principles, Standard Protocols andTechniques. PTR,Prentice Hall, 1994.[12] C. Kaufman, R. Perlman, and M. Spenciner, Network Security – Private Communicationin a Public World. PTR,Prentice Hall, 2002.[13] R. A. DeMillo, N. A. Lynch, and M. J. Merritt, “Cryptographic Protocols,” Proceedingsof the fourteenth annual ACM symposium on Theory of computing, vol. 19, pp. 383–400,1982.[14] R. DeMillo and M. Merritt, “Protocols for Data Security,” IEEE Jnl on Computer,vol. 16, no. 2, pp. 39–51, 1983.[15] A. Menezes, P. van Oorschot, and S. Vanstone, Handbook of Applied Cryptography. CRCPress, 1996.[16] V.L.Voydock and S.T.Kent, “Security Mechanisms in high-level Network Protocols,”ACM Computing Surveys, vol. 15, pp. 135–171, June. 1983.104

[17] J. Callas, L. Donnerhacke, H. Finney, and R. Thayer, “OpenPGP Message Format,”RFC: 2440, Nov. 1998.[18] P. Zimmermann, PGP User’s Guide. The MIT Press, 1995.[19] T. Dierks and C. Allen, “The TLS Protocol : Version 1.0,” RFC: 2246, Jan. 1999.[20] S. Kent and R. Atkinson, “Security Architecture for the Internet Protocol,” RFC 2401,1998.[21] B. Gleeson, A. Lin, J. Heinanen, T. Finland, G. Armitage, and A. Malis, “A Frameworkfor IP Based Virtual Private Networks,” RFC 2764, Feb. 2000.[22] E. Rosen, A. Viswanathan, and R. Callon, “Multiprotocol Label Switching Architecture,”RFC: 3031, Jan. 2001.[23] S. Kent and R. Atkinson, “IP Authentication Header,” RFC 2402, Nov. 1998.[24] S. Kent and R. Atkinson, “IP Encapsulating Security Payload,” RFC 2406, Nov. 1998.[25] D. Harkins and D. Carrel, “The Internet Key Exchange (IKE),” RFC 2409, Nov. 1998.[26] D. Maughan, M. Schertler, M. Schneider, and J. Turner, “Internet Security Associationand Key Management Protocol (ISAKMP),” RFC 2408, Nov. 1998.[27] R. Perlman and C. Kaufman, “Key exchange in IPSec: analysis of IKE,” InternetComputing, IEEE, vol. 4, pp. 50–56, Nov.-Dec. 2000.[28] I. Faynberg, S. Kasera, G. Sundaram, S. Mizikovsky, and T. Woo, “Intermediary-Based Transport Services, Performance Optimization Mechanisms, and Relevant Requirements,”INTERNET-DRAFT : draft-faynberg-intermediary-transport-00.txt, Feb.2002.[29] P. Srisuresh, J. Kuthan, J. Rosenberg, A. Molitor, and A. Rayhan, “Middlebox communicationarchitecture and framework,” RFC : 3303, Feb. 2002.[30] Y. Zhang, “A Multilayer IP Security Protocol for TCP Performance Enhancementin Wireless Network,” IEEE Journal on Selected Areas In Communications, vol. 22,pp. 767–776, May 2004.[31] M. Karir, “IPSEC and the Internet,” Masters Dissertation, Number: CSHCN MS 1999-9, 1999.[32] Y. Zhang and B. Singh, “A multi-layer IPsec protocol,” Proc. Usenix Security Symp,pp. 213–228, Aug. 2000.[33] W. Stevens, The TCP/IP Illustrated Vol I : The Protocols. Addision-Wesley, 1995.[34] J. Postel and J. Reynolds, “File Transfer Protocol (FTP),” RFC : 959, Oct. 1985.[35] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee,“Hypertext Transfer Protocol – HTTP/1.1,” RFC : 2616, June 1999.105

[36] J. Postel and J.Reynolds, “TELNET Protocol Specification,” RFC : 854, May 1983.[37] R. Perlman, Interconnections: Bridges, Routers, Switches, and Internetworking Protocols.Addison-Wesley, 2000.[38] M. Allman, V. Paxson, and W. Stevens, “TCP Congestion Control,” RFC : 2581, Apr.1999.[39] S. Floyd, “Limited Slow-Start for TCP with Large Congestion Windows,” RFC : 3742,March 2004.[40] W. Stevens, “TCP slow start, congestion avoidance, fast retransmit, and fast recoveryalgorithms,” RFC : 2001, 1997.[41] M. Chan and R. Ramjee, “TCP/IP performance over 3G wireless links with rate anddelay variations,” Proc. of ACM Mobicom, pp. 71–82, Sept. 2002.[42] B. Guha and B. Mukherjee, “Network security via reverse engineering of TCP code:Vulnerability analysis and proposed solutions,” Proc. IEEE Infocom ’96, vol. 2, pp. 603–610, March 1996.[43] R. Oppliger, “Security at the Internet layer,” IEEE JNL on Computer, vol. 31, pp. 43–47, Sep. 1998.[44] N. Doraswamy and D. Harkins, IPSec, 2nd Edn. Pearson Education, 2002.[45] S. Frankel, Demystifing the IPSec Puzzle. Artech House Publishers, 2001.[46] J. Reynolds and J. Postel, “Assigned Numbers,” RFC 1700, Oct. 1994.[47] R. Rivest, “The MD5 Message-Digest Algorithm,” RFC : 1321, April 1992.[48] M. Oehler and R. Glenn, “HMAC-MD5 IP Authentication with Replay Prevention,”RFC : 2085, Feb. 1997.[49] P. Metzger and W. Simpson, “IP Authentication using Keyed SHA,” RFC : 1852, Sep.1995.[50] C. Madson and R. Glenn, “The Use of HMAC-MD5-96 within ESP and AH,” RFC2403, Nov. 1998.[51] C. Madson and R. Glenn, “The Use of HMAC-SHA-1-96 within ESP and AH,” RFC2404, Nov. 1998.[52] C. Madson and N. Doraswamy, “The ESP DES-CBC Cipher Algorithm With ExplicitIV,” RFC 2405, Nov. 1998.[53] P. Karn, P. Metzger, and W. Simpson, “The ESP DES-CBC Transform,” RFC : 1829,Aug. 1995.[54] P. Karn, P. Metzger, and W. Simpson, “The ESP Triple DES Transform,” RFC : 1851,Sep. 1995.106

[55] R. Glenn and S. Kent, “The NULL Encryption Algorithm and its Use with IPsec,” RFC2410, Nov. 1998.[56] N. Ferguson and B. Schneier, “A Cryptographic Evaluation of IPSec,” online athttp://www.counterpane. com/ipsec.html, Apr. 1999.[57] H. Orman, “The OAKLEY Key Determination Protocol,” RFC 2412, 1998.[58] B. Braden, D. Clark, J. Crowcroft, B. Davie, S. Deering, D. Estrin, S.Floyd, V. Jacobson,G. Minshall, C. Partridge, L. Peterson, K. Ramakrishnan, S. Shenker, J.Wroclawski, andL. Zhang, “Recommendations on queue management and congestion avoidance in theInternet,” IETF, RFC 2309, 1998.[59] S. Floyd and V. Jacobson, “Random early detection gateways for congestion avoidance,”IEEE/ACM Trans. on Networking, vol. 1, pp. 397–413, Aug. 1993.[60] S. Floyd and K. Fall, “Promoting the use of end-to-end congestion control in the Internet,”IEEE/ACM Trans. on Networking, vol. 7, pp. 458–472, Aug. 1999.[61] A. Bakre and B. R. Badrinath, “I-TCP: Indirect TCP for mobile hosts,” Proc. 15th Int.Conf. Distributed Computing Systems (ICDCS), pp. 136–143, May 1995.[62] H. Balakrishnan, S. Seshan, E. Amir, and R. H. Katz, “Improving TCP/IP performanceover wireless networks,” Proc. 1st Annu. Int.Conf. Mobile Computing and Networking(MobiCom95), pp. 2–11, 1995.[63] H. Balakrishnan, V. Padmanabhan, S. Seshan, and R. Katz, “A comparison of mechanismfor improving TCP performance over wireless links,” IEEE/ACM Trans. on Networking,vol. 5, pp. 756–769, Dec. 1997.[64] B. Bakshi, P. Krishna, N. H. Vaidya, and D. K. Pradhan, “Improving performanceof TCP over wireless networks,” Proc. 17th Int. Conf.Distributed Computing Systems(ICDCS-17), pp. 365–373, May 1997.[65] S. McCreary and K. C. Claffy, “Trends in wide area IP traffic patterns:A view fromames Internet exchange,” ITC Specialist Seminar, Monterey, CA.[Online]. Available:http://www.caida.org/outreach/papers/2000/AIX0005/, vol. 7, pp. 458–472, Sep. 2000.[66] A. Alshamsi and T. Saito, “A Technical Comparison of IPSec and SSL,” Proc. of the19th International Conference on Advanced Information Networking and Applications(AINA05), 2005.[67] S. Bellovin, “Transport-friendly ESP (or layer violation for fun and profit),” IETF-44TF-ESP BOF, March 1999.[68] J. Singh and B. Soh, “A Critical Analysis of Multilayer IP Security Protocol,” Proc ofThird International Conf. on Information Tech. and Application, 2005.[69] E. Rescorla, “Diffie-Hellman Key Agreement Method,” RFC 2631, June 1999.[70] The Linux Kernel Archives: www.kernel.org .107

[71] D. McDonald, C. Metz, and B. Phan, “PF-KEY key management API, version 2,” IETFRFC 2367, July 1998.[72] D. P. Bovet and M. Cesati, Understanding the Linux Kernel, Second Edition. O’Reilly,December 2002.[73] D. Bovet and M. Cesati, Understanding the Linux Networking Kernel, Second Edition.O’Reilly, December 2002.[74] W. Stevens and G.R.Wright, The TCP/IP Illustrated Vol II : The Implementation.Addision-Wesley, 1995.[75] J. Corbet, A. Rubini, and G. Kroah-Hartman, Linux Device Drivers, Third Edition.O’Reilly, February 2005.[76] S. Hares and C. Wittbrodt, “Essential Tools for the OSI Internet,” RFC 1574, Feb.1994.[77] R. Rivest, A. Shamir, and L. Adleman, “A Method for Obtaining Digital Signaturesand Public-Key Cryptosystems,” Communications of the ACM, vol. 21, pp. 120–126,Feb. 1978.[78] C. Batut, K. Belabas, D. Bernardi, H. Cohen, and M. Olivier, Users Guide to the PARIlibrary (version 2.3.1) http://pari.math.u-bordeaux.fr.[79] The netfilter.org iptables project. http://www.netfilter.org/.[80] O. Andreasson, Iptables Tutorial 1.2.2. http://iptables-tutorial.frozentux.net/iptablestutorial.html.[81] G. Combs, Wireshark : Network Protocol Analyzer for Unix and Windows. Availableat www.wireshark.org.108