« Network Blow-out | Main | The case for disbelief »
January 28, 2003
High-performance network, low-performance hosts
In the race to put in bigger, faster networks, application performance is usually overlooked or worse, assumed to scale with the network.
The two predominant IP protocols on the Internet are TCP and UDP. TCP is a well-behaved protocol that includes mechanisms ("slow-start" and "back-off") to enable lots of hosts to use the network without stomping on each other. TCP is a reliable transport protocol that ensures applications that everything transmitted will arrive intact at the other end. TCP accounts for 90+% of traffic in most networks. UDP has none of these features, and is used (most successfully) by applications that use a relatively small amount of bandwidth (typically < 10% overall ) and can handle unreliable connectivity.
The algorithms that make TCP behave well were designed for much slower and less-reliable networks than we have today. Similarly, Ethernet was designed for 10Mbps networks. Today's gigabit-per-second networks present some interesting challenges to high-performance applications.
The combination of TCP and Ethernet means that a typical host, in best-case-scenario, can achieve no more than 6.5Mb/s in a single transfer session. That isn't even possible in most cases due to poorly-written applications, operating systems, network drivers and CPU-intensive protocols.
Projects such as Web100, the Internet2 End-to-End Performance Initiative, TCP Tuning and Ethernet Jumbo Frames are focusing on this problem from various directions. Unfortunately, the changes they require take a long time to be implemented in today's networks. Guy Almes addressed this at Westnet, talking about some of the social issues around this issue.
This issue will become of increasing importance to UEN and our stakeholders this year. We are installing GigE and 100Mb circuits throughout our backbone and to many schools. It will be unfortunate if the applications are incapable of utilizing this bandwidth due to TCP, Ethernet and other limitations.
Posted by pete at January 28, 2003 10:48 PM
Trackback Pings
TrackBack URL for this entry: