Foundations of Python Network Programming 978-1-4302-3004-5

CHAPTER 3 ■ TCP
$ python tcp_deadlock.py client 1073741824
Sending 1073741824 bytes of data, in chunks of 16 bytes
734816 bytes sent
Why have both client and server been brought to a halt?
The answer is that the server’s output buffer and the client’s input buffer have both finally filled, and
TCP has used its window adjustment protocol to signal this fact and stop the socket from sending more
data that would have to be discarded and later re-sent.
Consider what happens as each block of data travels. The client sends it with sendall(). Then the
server accepts it with recv(), processes it, and then transmits its capitalized version back out with
another sendall() call. And then what? Well, nothing! The client is never running any recv() calls—not
while it still has data to send—so more and more capitalized data backs up, until the operating system is
not willing to accept any more.
During the run shown previously, about 600KB was buffered by the operating system in the client’s
incoming queue before the network stack decided that it was full. At that point, the server blocks in its
sendall() call, and is paused there by the operating system until the logjam clears and it can send more
data. With the server no longer processing data or running any more recv() calls, it is now the client’s
turn to have data start backing up. The operating system seems to have placed a limit of around 130KB
to the amount of data it would queue up in that direction, because the client got roughly another 130KB
into producing the stream before finally being brought to a halt as well.
On a different system, you will probably find that different limits are reached. So the foregoing
numbers are arbitrary and based on the mood of my laptop at the moment; they are not at all inherent in
the way TCP works.
And the point of this example is to teach you two things—besides, of course, showing that
recv(1024) indeed returns fewer bytes than 1,024 if a smaller number are immediately available!
First, this example should make much more concrete the idea that there are buffers sitting inside
the TCP stacks on each end of a network connection. These buffers can hold data temporarily so that
packets do not have to be dropped and eventually re-sent if they arrive at a moment that their reader
does not happen to be inside of a recv() call. But the buffers are not limitless; eventually, a TCP routine
trying to write data that is never being received or processed is going to find itself no longer able to write,
until some of the data is finally read and the buffer starts to empty.
Second, this example makes clear the dangers involved in protocols that do not alternate lock-step
between the client requesting and the server acknowledging. If a protocol is not strict about the server
reading a complete request until the client is done sending, and then sending a complete response in the
other direction, then a situation like that created here can cause both of them to freeze without any
recourse other than killing the program manually, and then rewriting it to improve its design!
But how, then, are network clients and servers supposed to process large amounts of data without
entering deadlock? There are, in fact, two possible answers: either they can use socket options to turn off
blocking, so that calls like send() and recv() return immediately if they find that they cannot send any
data yet. We will learn more about this option in Chapter 7, where we look in earnest at the possible
ways to architect network server programs.
Or, the programs can use one of several techniques to process data from several inputs at a time,
either by splitting into separate threads or processes—one tasked with sending data into a socket,
perhaps, and another tasked with reading data back out—or by running operating system calls like
select() or poll() that let them wait on busy outgoing and incoming sockets at the same time, and
respond to whichever is ready.
Finally, note carefully that the foregoing scenario cannot ever happen when you are using UDP! This
is because UDP does not implement flow control. If more datagrams are arriving up than can be
processed, then UDP can simply discard some of them, and leave it up to the application to discover that
they went missing.
47