?

Log in

No account? Create an account
fragglet [userpic]

It's the bandwidth, stupid: part 2

June 6th, 2007 (03:12 am)

Bill Dougherty has posted Part 2 of this "It's the latency, stupid" article. Sadly, this is filled with as many factual errors as the previous one.

Where do I start? First of all, HTTP: "HTTP 1.1 signals the web server to use gzip compression for file transfers". This is pure and simply wrong. Go and read the HTTP/1.1 specification. Although gzip is mentioned, there's no requirement that a HTTP/1.1 server should use gzip compression. I'd say that no browser has shipped for at least five years that uses HTTP/1.0, so this is a totally irrelevant suggestion to make. Even then, switching to HTTP/1.1 will not magically add gzip compression: it's up to the web server to optionally send you compressed data instead of the normal uncompressed data. 99+% will not do this.

Using HTTP/1.1 CAN provide an advantage, but for different reasons that are entirely unrelated to compression. The major difference between HTTP/1.0 and 1.1 is that HTTP/1.1 can reuse an existing connection to retrieve more files. HTTP/1.0 immediately closes the connection when a download has completed. This has advantages because of the way the congestion control algorithms work: they start off with a small TCP window size that is increased in order to determine the available bandwidth of the channel. With HTTP/1.0, this process is restarted when downloading each file. HTTP/1.1 allows you to reuse your existing connection that has already settled to a reasonable TCP window size. This is important for modern websites that have lots of images and other embedded content. As I mentioned before though, this is utterly irrelevant because all modern browsers already use HTTP/1.1 by default.

Then Bill comes up with this gem: "One effective method is to change the protocol. Latency is a problem because TCP waits for an acknowledgement". This is also wrong. He seems to be under the mistaken impression that TCP is a stop and wait protocol: that each packet is sent, an acknowledgement waited for, and then the next packet sent. What actually happens is that TCP sends a bunch of packets across the channel, and as the acknowledgement is received for each of packet, the next packet is sent. To use the trucks analogy again, imagine twenty trucks, equally spaced, driving in a circle between two depots, carrying goods from one depot to the other. Latency is not a problem, just like distance between the depots is not a problem: provided that you have enough trucks, the transfer rate is maintained. The TCP congestion control algorithms automatically determine "how many trucks to use".

TCP will restrict the rate at which you can send data. Suppose, for example, you're writing a sockets program and sending a file across a TCP connection: you cannot send the entire file at once. After you have written a certain amount of data into the pipe, you cannot write any more until the receiving end has read the data. This is a good thing! What is happening here is called flow control. You physically can't send data faster than the bandwidth of the channel you're using can support. Suppose that you're using 10KB/sec channel: you can't send 50KB/sec of data across that channel. All that TCP is doing is limiting you to sending data at the physical limit of the channel.

"If you control the code, and can deal with lost or mis-ordered packets, UDP may be the way to go". While this is true, it's misleading and potentially really bad advice, certainly to any programmers writing networked applications. If your application mainly involves transfer of files, the best thing to do is stick with TCP. The reason is that TCP already takes care of these problems: they've been thoroughly researched and there are many tweaks and optimisations that have been applied to the protocol over the years. One important feature is the congestion control algorithms, that automatically determine the available bandwidth. If you don't use these kind of algorithms, you can end up the kind of collapse that Jacobson describes in his original paper on network congestion. If you use UDP, you're forced to reinvent this and every other feature of TCP from scratch. As a general rule of thumb, it's best to stick with TCP unless there is some specific need to use UDP.

Finally, I'd like to examine his list of "tricks that network accelerators use":

"1. Local TCP acknowledgment. The accelerator sends an ack back to the sending host immediately. This ensures that the sender keeps putting packets on the wire, instead waiting for the ack from the actual recipient". This is nonsense. TCP keeps putting packets onto the wire in normal operation. It doesn't stop and wait for an acknowledgement. TCP acknowledgements should already be being transmitted correctly If you're interfering with the normal transmission of acknowledgements, all you're doing is breaking the fundamental nature of how the protocol and the sliding window algorithm work.

"2. UDP Conversion. The accelerators change the TCP stream to UDP to cross the WAN. When the packet reaches the accelerator on the far end, it is switched back to TCP. You can think of this a tunneling TCP inside of UDP, although unlike a VPN the UDP tunnel does not add any overhead to the stream." I fail to see what possible advantage this could bring.

"3. Caching. The accelerators notice data patterns and cache repeating information. When a sender transmits data that is already in the cache, the accelerators only push the cache ID across the WAN. An example of this would be several users accessing the same file from a CIFS share across your WAN. The accelerators would cache the file after the first user retrieves it, and use a token to transfer the subsequent requests." This is useful in the very specific case of CIFS, because SMB has known performance issues when running over high latency connections - it was designed for use on LANs, and the protocol suffers because of some assumptions that were made in its design. This doesn't apply, however, to the majority of other network protocols.

"4. Compression. In addition to caching, network accelerators are able to compress some of the data being transmitted. The accelerator on the other end of the WAN decompresses the data before sending it to its destination. Compressed data can be sent in fewer packets, thus reducing the apparent time to send." Amusingly, what this actually does is decrease the bandwidth used, and has nothing to do with latency.

Comments

Posted by: LionsPhil (lionsphil)
Posted at: June 6th, 2007 04:22 pm (UTC)
Re: Thanks again for the comments
ORLY

"Those Doom games"? I hope that's not intended to be as patronising as it sounds—because latency is such a massive factor in the perceived network performance of computer games, it's an area which has seen tremendous, even if ad-hoc, research. I would have thought that this was something you would have found interesting and relevant to your position.

  • UnrealEngine's network architecture; note the heavy emphasis on predictive techniques and use of UDP. This is the version of the networking code after its infamously bad start, and is—to the best of my knowledge—still basically correct for the latest versions of UnrealEngine.
  • Source engine networking; this one is heavy on the interpolation. Note that they emphasise the bandwidth/latency/packet loss relation in the case of attempting to push excessive state updates.
  • X-Wing vs Tie Fighter paper; mostly a historical curio (hence WayBack), but introduces Dead Reckoning for data prediction, and goes on to explain why TCP is unsuitable for games over old, rickety, dial-up Internet connections.
Hope you find those interesting reading.

2 Read Comments