17 September 1992 The ROAD to a New IP* by David H. Crocker, Brandenburg Consulting Copyright © 1992 by Interop Company TCP and IP have been in full operational use since 1983. There are tens of thousands of IP networks and perhaps millions of users. By any reasonable measure, IP counts as a massive suc cess. Unfortunately, the phenomenal number of users is stressing the design limits of IP: the 32-bit IP Host Address space simply is too small for long-term growth -- although it was more than ample when chosen, 15 years ago. This article explores the nature of the limitation and the efforts to move from the current IP version 4 to a new IP version 7. (Versions numbers 5 and 6 have been used elsewhere.) Disclaimer #1: This topic has recently engendered significant debating and politicking within the Internet community, including a crisis of confidence that has shaken the core of IETF, this summer. This article attempts to present the technical issues independent of the political and emotional concerns, hoping to aid the reader in considering the nature of the trade-offs offered by various proposals. It is essential that the choice be made by an informed Internet community, since the choice will not be easy or obvious, but it will affect the future of the Internet. General online discussion about this topic is conducted on the mailing list: big-internet@munnari.oz.au but the reader is reminded to subscribe by sending to big- internet-request. Disclaimer #2: The author of this article is an author of one of the proposals. Disclaimer #1 notwithstanding, the author will attempt to avoid the temptation to indulge his certain bias towards his own excellent proposal... Is there really a problem? An address space of 32-bits allows reference to billions of hosts. (Well, actually host interfaces.) Since current consumption of IP addresses is only in the realm of one or a few million, it might appear that there is no pressing concern. In fact, there are two different problems. One is an immediate crisis and the other is likely to become one within a few years. The first is the size of the information base that must be maintained within IP routers, for making data-forwarding decisions. The latter is simple exhaustion of the IP address space. Internet growth is approximately 100% every 12 months. While growth in North America has slowed, growth in Europe is explosive and the Pacific Rim is starting a similar curve. All of this is within the usual markets of business and technical users. If new markets open, such as use of IP in consumer products, a significant fraction of the world's population become candidates for IP addresses. Note that IP, with subnet addressing, divides an address into: Network : Sub-network : Host And uses the Class A/B/C mechanism, mixed with address masks, to vary where the boundaries occur between these sub-fields. All of this fits within 32 bits and network numbers are assigned sequentially, with no relationship between any two network numbers, producing a "flat" address space. No matter what games are played within those bits, there is a limit to the maximum number of addresses that can be assigned. Routing table size In the Internet backbone, routers are required to know about all of the networks that can be reached. These routers do not have to know about the different sub-networks or hosts, but they do need to know about each and every network in the entire Internet. This is due to the "flat" address space of IP network numbering. The fact that two networks are topological neighbors and that packets to the two may travel most of the Internet using the same path is of no benefit when constructing routing tables. The two network numbers are unrelated, requiring calculation of routes for both of them. The computational cost of calculating these paths separately has become a serious burden. So a way needs to be found to aggregate table entries. This requires a fundamental change in the nature of IP network addresses, from the current, flat style, to one that has some useful structure. The usual assumption is that a hierarchical scheme will be most appropriate. Address space While there is no topological information in the encoding of IP network numbers, there is a scheme for distinguishing "large" networks from smaller ones, via the Class A, B, and C mechanism. The most popular addresses are Class B, since they permit reasonably large networks and there are a substantial number (approximately 32,000) of such network numbers available. Very few Class A addresses are possible and Class C addresses are useful only for the smallest networks. Hence, the first wall that IP will hit is an exhaustion of Class B address. While there may be a way to get around this wall, the second wall, assignment of all possible IP addresses, eventually will be encountered. There is debate about the urgency of these 2 walls. However, it appears that modifications to the assignment of Class B addresses, and modifications to backbone router protocols, will allow deferral of the first concern. To expand the IP address space, enough bits need to be added to allow for reasonable administration and for a scheme which provides some meaningful assistance in relating topological neighbors. For example, a datagram from Japan destined for Stanford University and another from Japan destined for Berkeley ought to travel most of the same path along the Internet, and most of the routers along the way ought to need one -- not two -- entries in their routing tables. This is only possible if some portion of the two addresses is the same. As long as we've got this thing open... IP works well, but improvement always is possible. One line of thought is that this forced change to addressing provides an opportunity to repair or improve other aspects to IP. For exam ple, the space available for IP options sometimes is viewed as too constrained. Similarly, the mechanism for specifying con straints upon handling (Type of Service) have never been viewed as adequate and have not seen significant use. The remainder of this article considers the challenges of balanc ing competing concerns, the selection process that is being pur sued, and the nature of the routing-, addressing- and header format-related issues. It ends with a very brief summary of the major proposals that are before the Internet community. The article is frankly cursory in discussing most of the issues. The current debate ranges wide and deep, in considering fundamental aspects of the IP infrastructure. There is a great deal of activity and a great many messages, Internet Drafts and RFCs. This article attempts to summarize the issues and proposed solutions, but the reader is encouraged to join the relevant mail ing lists and read the relevant documents. Trading immediacy for carefulness Opinions about the timing of total address space exhaustion vary between 2 years and 20 years. However, a plausible, near-term estimate is 5 years. This means that the Internet has until 1997 to develop, test and install a new addressing scheme. Conservative project management thinking would attend the dif ficulties of fielding an entirely new technology in such a large community and would seek to have the new scheme fully deployed in 3-4 years. This leaves very little time to develop and test a solution. As those who create and distribute products and services well know, the basic development of a capability is often the smallest part of an effort. Testing, manufacturing, training and instal lation often consume considerable time and resources. By any reasonable measure, it will take 2-3 years to deploy a solution. This leaves us with only 1-2 years to develop, test and stabilize a solution. A choice needs to be made immediately. This creates a difficult pressure between immediacy and inno vation. Some of the alternatives being considered sound quite appealing, but have minimal or fluid specifications, no oper ational experience, and may represent significant differences from current IP experience. The challenge, for these alter natives, is to demonstrate that the degree of benefit that will accrue from choosing them is sufficiently great to justify the risk of delaying their deployment. Or else, to convince the community that no extra time is required. The challenge for the "simpler" and more conservative alternatives is to convince the community that they offer fundamental safety and cost-effec tiveness. The biggest danger is one of "creeping feature-ism" with more and more requirements being added to the project. It is easy to succumb to this, since we do not get many opportunities to change the infrastructure. However, each additional requirement makes the total task that much more complicated and risky. At a minimum, it virtually guarantees delays before the total package of changes can be deployed into the Internet. Technically the only clear requirement is to enhance IP's routing and addressing structure. Everything else is beyond the immediate crisis. Previous efforts Concern about IP address exhaustion and routing table size explosion has created a sense of crisis within the IETF commu nity. Almost 2 years ago, a special effort, called the ROAD (ROuting and ADdressing) group was formed to consider solutions. It gravitated towards one option, but did not see quick adoption of its recommendation. But time had passed and urgency grew. There has been pressure to select a solution immediately, without extensive exploration and development of options. The Internet Engineering Steering Group (IESG) divided the concerns into short- term, mid-term and long-term. Class-B exhaustion and routing table size explosion fall into the first category. IP address space exhaustion falls into the mid-term timeframe. The IESG feels that other issues of general enhancement to IP, such as quality of service, security/authentication, mobility, resource allocation, accounting, and high packet rates can been deferred for "long term" consideration. There is reasonable consensus that the proposal called Classless Inter-Domain Routing (CIDR) [2] will be adequate for the near- term, by modifying usage of current IP addresses. However, some conservative members of the community advise against relying entirely upon this one option and suggest that the "mid-term" option be developed with all due speed, in case CIDR proves inadequate. This would suggest that deployment needs to start during 1994! However, the IESG felt that none of the options being discussed this past Spring was sufficiently well specified to allow an adequate analysis of its capabilities. So, the IESG has called for a review at the November, 1992 IETF meeting, with analysis according to a published set of criteria developed by the IESG, using community input. [3] The contenders will make presentations at the IETF meeting and the IESG will later issue its recommendation. With luck, a considerable degree of community preference will have developed by that time. Otherwise, the IESG decision may suffer inadequate community support. Evaluation Criteria The IESG list divides into the following categories: Ą Changes Required: What is the basic technical work that must be done, to modify IP-related software to support a proposed alternative? This includes direct modification or replacement to the IP module, itself, in hosts and routers, but also includes concern for changes to directory, network management, and security services. In general, it is expected that support software will need to be modified, as will various operations and administration procedures. It should be noted that any change, no matter how small, requires that a new address format be supported. This well may be the most significant impact, and it is unavoidable. Note that protocol modules above IP, such as TCP and UDP, need to be able to pass the larger addresses to the IP module, as do user applications. In fact some applications, such as FTP and NFS, currently use IP addresses and will need changing. Ą Implementation Experience: Simply put, what is the empirical evidence that supports the viability and appropriateness of the alternative? Ą Large Internet Support: A goal of supporting 109 networks and 1010 hosts is cited. A proposed alternative needs to describe how its addressing structure will support these and what effect it will have upon routing architectures and tables. An essential concern is the way in which addresses will be administered. If IP addresses are to be sufficient for truly global communications, how will each user obtain their address? Ą Performance Impact: IP has demonstrated a remarkable robustness for application in increasingly high-speed networks. There is, then, concern about the performance cost of any changes in IP. A proposed alternative will need to explore this issue carefully. Ą Support for Unchanged IP Hosts: Changes cause incompat ibilities. Even if all systems eventually move to the new scheme, there will be a transition period. Related experience suggests that such a period will be extended, possibly on the order of 10 or more years. Hence, there is a concern for the interoperation between systems using the new scheme and systems continuing to use the current IP infrastructure. A proposed alternative needs to discuss its support for "late adopters". Ą Impact on Installed Base of Users: Amidst the wide-ranging concerns for technical factors, it is easy to miss the likely impact of this change upon the people who work with IP technology, ranging from developers and network admin istrators, to customer support and sales staff. They repre sent a massive investment in training and expertise. The extent to which an alternative affects that training needs to be discussed. Ą Deployment Plan: Since systems will not convert to a new scheme instantaneously a proposal needs to detail the methods by which the Internet and its constituents will convert to its use. Ą Future Evolution: To what extent does an alternative support continued evolution of the IP fabric, into the arena of long- term issues mentioned above? Design considerations Routing protocols, and the theories of route calculation, remain a specialized topic with relatively few contributors. There has been significant effort in this area, recently, with the develop ment of OSPF, IS-IS, IDPR, IDRP and BGP. Happily, the current crisis does not seem to require major changes to these new protocols. The key requirement to facilitate large-scale routing is that addresses of Internet neighbors be related in a manner which allows reducing the number of table entries, for distant routers. This is generally agreed to require some sort of addressing hierarchy, such that the hierarchy relates to the "dominant" topological hierarchy. A hierarchy is a tree- structured orientation, yet networks permit "mesh" attachments between any two nodes. Hence, networks usually are not organized as strict hierarchies. However political, economic, and management constraints do tend to cause network interconnections to follow a hierarchy. In the United States, for example, users tend to attach to their organization's backbone, which in turn tends to attach to inter-organization providers. These may also subdivide into regional and long-haul carriers. In addition to the routing-related constraint upon design of the new address space, global administration and end-system uniqueness are requirements. The new addresses must be globally unique, as are the current IP addresses. There also is some debate about the distinction between addressing an end-system machine (or process) versus the current style of addressing the network interface of an end-system. Global administration requires a distributed basis for dividing the space, so that different places can assign unique addresses. Experience with administration of the Domain Name System suggests that hierarchical delegation of assignment authority is best. Any of the schemes that require more than 32 bits for addressing forces a change to the header format. Proposals range from simply modifying the current IP header to accommodate the additional bits, up to a complete re-working of the header, according to more modern views of efficiency and modularity. Styles of addressing A number of approaches to large-scale addressing are being discussed: Ą Association-based: Curiously, one approach to solving this problem suggests that end systems continue to use 32-bit addresses, but that the network should consider them to refer to "associations" or end-system pairs, like a "connection id" in a virtual circuit protocol. At any given moment, it is unlikely that there will 4 billion IP-based "discussions" going on, so that the 32-bit address space may be large enough for uniqueness in identifying simultaneous conversations. This approach suggests retaining the current address field, but using it for such association ids, with routers performing address translation of an id into a series of routing decisions. This presumes some mechanism for establishing a temporary relationship between an id and a route. Ą ID-based: Somewhat similar to Association-based, this uses long-term, globally-unique IDs which specify individual end- systems. IDs for neighboring end-systems may be entirely unrelated. Hence, these IDs are a form of end-system name, rather than address. Addressing information, used to develop a routing sequence, would be derived dynamically. This is felt to be particularly friendly in supporting mobile hosts, but there also is a question about placing a "name" into every IP datagram along with still-necessary addressing information. Ą Provider-based: In the realm of classic global addressing, provider-based addresses would identify an end-system in terms of its attachment (or, more precisely, in terms of the end- system's network's attachment) to a given network service provider. For administrative ease, provider identification probably would be subdivided according to the country in which the provider is present. This leaves open the issue of referencing providers that are multi-national. The major criticism to provider-based addressing is that user networks would be required to change their addresses whenever they changed providers. There is concern that this might reduce competition. Ą Geography-based: Also offered as a classic approach, geographic addresses are gobally unique, but specify the end- system (network) strictly in terms of its geographic location, on the theory that a geographic hierarchy is a reasonable approximation of the global Internet's major topology. Originally proposed by Steve Deering, of Xerox PARC who called them city codes, the major concern about geographic addressing is determination of the final provider to which the target end- system (network) is attached. The current proposal calls for inter-connection facilities, called Metropolitan Internet eXchanges (MIX) for the "last hop" hand-off to the target provider. Ą Source-routed: Proposed by Dave Clark of MIT, a type of addressing which uses route fragments would specify a sequence of addresses. The concatenated sequence would be a kind of source route, but with large granularity, rather than specifying each hop along the way. This is spiritually related to the current Loose Source Route IP option. Proposals This article has discussed the nature of the impending and current problems, the process that will be used to evaluate proposals, and the types of technical choices that seem to be plausible. The remainder of this article briefly summarizes the major proposals that have been offered. It is remarkable that every single one of these alternatives appears to be entirely reasonable, according to a legitimate set of criteria. They have competent specifications and appear to be technically viable, on their own. For all that, the proposals are quite different, since they attend to different concerns. Hence, the Internet community is facing an extremely difficult decision. It cannot simply choose based upon the credibility of the people who are proponents or upon "political" concerns. The community needs to determine what factors are most significant to it. Once it does that, the technical choice is likely be straightforward. EIP Extended IP (EIP) was proposed by Zhen Wang, of University College London, and simply creates a new IP option which specifies the additional addressing bits that are needed.[7] There has been some debate about the performance impact of IP options, in routers, and in general, this proposal has not been the subject of sustained consideration within the IETF, though it represents the smallest imaginable change to the current IP format. IPAE IP Address Encapsulation (IPAE) has been proposed by Bob Hinden, of Sun Microsystems, and Dave Crocker, of The Branch Office. It retains the current IP format, but defines a mini-layer above IP and below the transport layer (TCP or UDP).[4] This mini-layer contains the new, larger global addresses for the source and the destination. Each commonwealth has its own 32-bit IP address space, within which the current 32-bit IP address behavior is retained. When the Internet runs out of 32-bit addresses, however, only those systems which have converted to full IPAE format will be able to communicate both within their commonwealth and to other commonwealths. IPAE addresses are likely to be proposed as a 12-octet hybrid of provider-based and geographic-based form with classic IP addresses in the last 4 octets. Discussion by the IPAE working group is on the mailing list: ip-encaps@sunroof.eng.sun.com SIP Simple IP (SIP) is a recent idea by Steve Deering. It retains IP, but make it simpler. It removes those IP fields that seem to be of little benefit, and retains those that clearly are needed. It proposes an 8-octet extended address, along geography/provider lines. Transition would fall into the category of "dual stack" in that an end-system and intervening routers would have to support old IP and new SIP, in order to talk with all hosts. CLNP Simple CLNP is the gist of the recommendation from the original ROAD group was to replace IP with ISO's Connectionless Network Layer Protocol, CLNP. This would have required use of other lower-layer OSI protocol's, but TCP, UDP, and the rest of the upper layers of the Internet protocols would be retained. The presumed strength of this proposal is its use of an international standard and the existence of some installed base. However, there is no operational TCP over CLNP and the conversion effort is not well understood. Classic 20-octet NSAP addresses were proposed. TUBA TCP/UDP over Bigger Addresses (TUBA) is being renamed to TCP/UDP on CLNP Addressed Networks (TUCAN) in order to place a reference to CLNP in its name. Written technical work has been provided by Ross Callon, of Digital Equipment Corp., and it proposes a conversion which is derived from CLNP but with changes appropriate for Internet usage.[1] Further details are provided by Dave Piscitello, of Bellcore.[5] The benefits of this approach are the same as for simple CLNP. One of its departures is use of ID-based addressing. Transition will require a "dual stack" operation, as discussed above. Discussion is on the mailing list: tuba@lanl.gov PIP The 'P' IP (PIP) has beeneveloped by Paul Tsuchiya, of Bellcore, and is a completely new internetworking protocol.[6] Header fields are divided into independent sections, such as for routing, versus handling. Transition requires dual stack operation. Very active discussion by the PIP working group is on the mailing list: pip@thumper.bellcore.com Nimrod New Routing and Addressing (Nimrod) is a routing architecture framework being developed by Noel Chiappa. It concerns itself less with header formats and more with basic issues in routing and derivative addressing structures. An addresses would be a hierarchical series of fields, derived from the "bottom" of the topology. Also supported would be end-point identifiers, along the lines of ID-based addressing. Summary This last section is decidedly not titled "Conclusions" because it is not possible to make any, yet. This section attempts a capsule description of the nature of the major proposals available to the Internet. It should be noted, however, that the specific addressing architecture proposals, with each header format proposal, could be replaced, so that some permutations and combinations may be worth considering. In any event, an undoubtedly biased summary of the options is: Ą PIP is the most radical and offers the greatest opportunity for long-term functional enhancements. By virtue of its great changes and lack of any operational experience, it also offers the greatest risk. Ą Nimrod is not tied to a specific format proposal, so that it may be possible to add a Nimrod routing and addressing scheme to PIP, TUBA or IPAE, if it can be developed and tested in time. Ą TUBA pursues use of an existing protocol which is functionally similar to IP but already contains larger addresses. It is, however, also pursuing changes to those addresses. The primary strength of TUBA is its relationship to a well- documented international standard; its greatest weakness is its differences in detail from IP, beyond the necessary addressing differences. Ą IPAE preserves current IP formats and software, imposing incremental changes only on those systems desiring full Internet connectivity. Its strength is the amount of software and training that will be retained. Its weakness is its apparent change to the Internet addressing model, by imposing a routing barrier between commonwealths. Ą SIP preserves the gist of the current IP formats which are known to be needed and otherwise only changes the size of addresses. The strength of SIP is its apparent simplicity and familiarity; its weakness is its newness and lack of operational experience. References IETF rules prohibit the citation of documents contained in the volatile Internet Drafts directory of the Internet Repository, which is replicated in various places around the Internet. Nonetheless, it may be the only location for some of the docu ments cited here (with inadequate information). On the other hand, the reader may find that RFCs have since been issued for such documents. [1] Callon, R., "TCP/UDP over Bigger Addresses (TUBA)", Request for Comments 1347, Network Information Center, SRI International, Menlo Park, CA, May 1992. [2] "Supernetting: an Address Assignment and Aggregation Strategy", Vince Fuller, Tony Li, Jessica Yu, and Kannan Varadhan (March 9, 1992). [3] Gross, P. and P. Almquist, "IESG Deliberations on Routing and Addressing". [4] Hinden, R. and D. Crocker, "A Proposal for IP Address Encapsulation (IPAE): A Compatible Version of IP with Large Addresses". To be re-issued as "IP Address Encapsulation (IPAE): A Compatible Version of IP with Large Addresses" and "IPAE Implementation & Transition". [5] Piscitello, D., "Use of ISO CLNP in TUBA Environments". [6] Tsuchiya, P., "The 'P' Internet Protocol". [7] Wang, Z., "EIP: The Extended Internet Protocol a long-term solution to Internet address exhaustion". _______________________________ * This is a draft of an article to appear in Connections: The Interoperability Report