The testbed initiative resulted in major advances in the use of SONET technology through its use in four of the five testbeds, ranging from the establishment of long-distance connections spanning over 2000 km to its use as a local interconnect technology. The major areas of activity centered around establishing and using carrier-provisioned SONET links, designing and experim
enting with customer-premises SONET access, exploring the use of striping over parallel SONET channels, and interworking SONET with other technologies (the last topic is discussed in a later section).
Carrier-provisioned SONET Links
To provide vendor-based SONET equipment for use in the testbeds, the participating testbed carriers worked together early in the project to develop specifications for the SONET equipment required in the two long-distance SONET-based testbeds, Aurora and Casa. At the time of this activity in 1990, a very limited range of equipment was available commercially from vendors, with minimal support for user-side interface rates above T3. The collective testbed needs for 622 Mbps user interfaces and unified procurement from the testbed carriers provided a major stimulant to SONET equipment vendors to accelerate delivery of higher speed equipment, with the result that prototype and in some cases operational versions of such equipment became available to the testbeds in the 1992-93 time frame.
The equipment which was subsequently deployed in the SONET-based testbeds operated at 2.5 Gbps on the trunks, and had either 155 or 622 Mbps user-side interfaces. The total bandwidth allocated between any two endpoints within the 2.5 Gbps trunks ranged from 622 Mbps to 1.2 Gbps, with the total endpoint rate achieved either through aggregating multiple 155 Mbps channels or through the use of synchronous 622 Mbps channels. Regenerative repeaters were used between termination points on the SONET links to make up optical fiber signal losses through digital signal recovery and amplification (Figure 4-2).
Figure 4-2. SONET Signal Regeneration
An area of major interest to the carriers was the potential problems which might be encountered when SONET equipment from different manufacturers were interconnected, in particular problems arising due to different interpretations of the SONET standard or of problems not foreseen by the standard developers. Both the Aurora and Casa testbeds provided an opportunity to deal with multi-vendor SONET equipment issues. In Aurora, the regional carriers NYNEX and Bell Atlantic both chose to install SONET equipment made by NEC, while MCI, the regional interconnect carrier, selected Northern Telecom equipment for its long-distance links. The Casa testbed included SONET equipment from three different vendors: Fujitsu, Northern Telecom and NEC.
In both testbeds, the multi-vendor interconnections were handled by collocating equipment from each vendor within the long-distance carrier (MCI) points of presence and using the user-side interfaces of the equipment for the connections between them (Figure 4-3). This arrangement allowed a single carrier to discover and resolve interconnection issues existing between different vendor equipment without also needing to coordinate among different carriers.
Figure 4-3. SONET Multivendor Interconnect
The use of different vendor equipment was highly beneficial to the goals of the testbed, for example some differences of interpretation of the SONET standard were discovered and resolved through coordination between the carriers and the vendors. More generally, important experience was gained with the installation and operation of this new equipment and of its performance over both short and long distances. Valuable experience was also gained in understanding how to deal with errors occurring on the SONET links [4, 6].
In addition to establishing SONET links, one of the testbed research groups developed and experimented with a SONET crossconnect switch. This switch allowed on-demand interconnection of full 622 Mbps SONET links, using synchronous OC-12c interfaces for connection to individual OC-12c SONET links provided by a carrier for that testbed. The design for this device was based on prototype lower-speed SONET chips just becoming available in the 1990-1991 timeframe, using advanced techniques to achieve synchronous 622 Mbps operation with chips operating individually at 155 Mbps. This activity clearly established the feasibility of building relatively low-cost equipment which operated with OC-12c links, and provided another valuable vehicle for exposing mis-interpretations of or inconsistencies in the SONET standards through its installation and use within the Vistanet testbed.
The carrier-provided equipment made OC-3c, OC-12, or OC-12c SONET interfaces available to testbed researchers through location of SONET termination equipment at the researcher's sites. To connect to the SONET links, a number of different research groups within the testbeds developed host, gateway or on-premises switch interfaces using prototype SONET chips to carry out the various functions required to deal with SONET data streams such as framing and synchronization (while such chips were just becoming available in prototype form, off-the-shelf equipment with SONET interfaces were rare at that point in time). Since four of the five testbeds used SONET for long-distance transmission, this resulted in a substantial body of experience in dealing with the multiple kinds of SONET interfaces made available to the testbeds.
For example, the Vistanet testbed used synchronous 622 Mbps SONET user interfaces, and so needed to deal with the issues of interfacing to a single synchronous bit stream at that speed in developing gateway equipment to connect HIPPI-based hosts within the testbed to their ATM/SONET wide-area equipment. In the Aurora testbed, the carrier equipment provided OC-12 interfaces at each site-in this case each SONET user port on the carrier equipment operates at a 622 Mbps rate but consists of four independent 155 Mbps logical channels. In the Casa testbed, the carrier equipment provided multiple OC-3c 155 Mbps physical user-side interfaces, while prototype equipment developed for the Nectar testbed multiplexed multiple STS-3c logical SONET channels into a SONET OC-48 2.5 Gbps trunk.
To further extend the experimentation opportunities offered by the testbeds, in many instances several different kinds of equipment were developed with SONET interfaces to explore differing approaches to dealing with particular networking problems, for example the interworking of other network technologies with SONET and in interfacing SONET links to different kinds of host architectures. These activities are discussed in more detail in their respective sections below on interworking and host interfacing.
A major anticipated use of SONET was to carry ATM cells, and as a result methods were defined for mapping ATM cells into SONET payloads as part of the international B-ISDN standards. More generally, the availability of ATM formatting standards circa 1990-91 allowed testbed equipment to be developed for ATM/SONET interfacing (with respect to an individual SONET channel) in a relatively straightforward manner within the testbeds. This was carried out in the Aurora, Nectar and Vistanet testbeds at SONET channel rates of both 155 and 622 Mbps through implementations associated with ATM switches, gateways, and host interfaces.
The mapping of variable-length packets into SONET channels, on the other hand, had not been addressed prior to the testbed work. The investigation of PTM (Packet Transfer Mode) technologies in the testbeds required that techniques be developed for accomplishing PTM/SONET interfacing at the high data rates used in the testbeds. Two of the testbeds, Aurora and Casa, addressed this problem.
In the Aurora testbed, IBM explored the use of PTM networking technology through their Planet PTM switch and Orbit LAN token ring technologies. Their solution to the PTM/SONET mapping problem, based on an optimization analysis, consisted of prefixing each packet with a 32-bit header containing the packet length in the first 16 bits and a CRC in the second 16 bits, with the CRC computed on the first 16 bits. The serial bit stream delivered by the SONET link is scanned for a correct CRC result, establishing the beginning of a packet; the packet length information is used to determine the end of the packet. This process is repeated after each packet is received, providing a link-level framing protocol residing above the SONET transmission layer.
The Casa testbed provided a second opportunity for investigating variable-length packet operation over SONET, in this case driven by the use of HIPPI-based technology at each user site and a direct mapping of HIPPI packets into SONET frames for long-distance transmission over dedicated SONET links. This solution, developed by LANL as part of their HIPPI-SONET gateway work, differs significantly from the Aurora solution in that it is tightly coupled to SONET framing. In particular, 72 bits are sent by the gateway at the beginning of each SONET Synchronous Payload Envelope (SPE), the part of the SONET frame which carries user data, to provide control information to the receiving gateway. The initial HIPPI packet sent on the link is aligned to begin immediately following these control words within the SPE; the end of the variable-length HIPPI packet is determined at the receiving gateway by detecting a special set of token bits appended to the packet.
Thus in the case of Casa a combination of SONET-level framing and serial data stream scanning is used to provide robustness in the presence of errors while taking advantage of control structures present for other purposes. While this solution is tied to word sizes used in HIPPI, both this and the Aurora solution have demonstrated the viability of carrying variable-length packets over SONET links and have provided significant directions for future standardization in this area.