socket programming

ConsistencyProblem statement:timeplayer1fireplayer2movefiremoveHow do you differentiate the two cases, at both player1 and player2?Sugih Jamin (jamin@eecs.umich.edu)

Lock-step Protocol• each player receives all otherplayers’ moves before renderingnext frameProblems:• long Internet latency• variable latencies• speed determined by the slowestplayertimePlayer1Player2synchronizemoves andrender sceneSugih Jamin (jamin@eecs.umich.edu)

Bucket Synchronization• buffer both local and remotemoves• play them in the future• each bucket is a turn, say forabout 200 ms• bucket size can be adapted tomeasured rttProblems:• game speed (bucket size) determinedby slowest player• what if a move is lost or late?timePlayer1XPlayer2synch movesand render scenea player can havemultiple movesper turnSugih Jamin (jamin@eecs.umich.edu)

Pessimistic ConsistencyEvery player must see the EXACT same worldAoE/AoK/AoM:• each player simulates its own copy of the world• all the worlds must be in sync.• uses bucket synchronization• each player sends moves to all other players• dropped packets retransmitted• a designated host collect measured rtts fromall players and set future bucket sizesProblems:• variable latencies• speed determined by the slowest playerSugih Jamin (jamin@eecs.umich.edu)

Dead Reckoning• a.k.a. client-side prediction• extrapolate next move basedon prior moves• compute the velocity and accelerationof objects to deadreckon• players can help by sending thisinfo along• obviously, only works if velocityand acceleration haven’tchangedvmove lost or late,dead reckonedSugih Jamin (jamin@eecs.umich.edu)

Roll-backIn case of inconsistency:• server always have authoritative view• when clients correct inconsistent views,players may experience “warping”• can players’ decisions be dead reckoned?See http://www.sim.inf.ethz.ch/projects/alpsim/Sugih Jamin (jamin@eecs.umich.edu)

Optimistic Consistency with Roll-backObservation: dead reckoning doesn’t have to be limited to lost packets!Half-Life:• each client plays back its own moves immediately and send themoves to server• each client also dead reckons the other players’ moves• server computes world and sends its authoritative version to all clients• clients reconcile dead reckoned world with server’s version• can result in some jerkiness and perception of “shooting aroundcorner”• only need to synchronize important events, but must be careful thatdead reckoning error doesn’t get compounded over timeSugih Jamin (jamin@eecs.umich.edu)

Consistency: CorrectnessFor consistency ALL user inputMUST pass through the synchronizationmoduleBe careful with random numbergenerators. Isolate the one used forgame-state updating from other uses(ambient noise etc.)Design for multiplayer from the start.Single-player becomes a specialcase of single-client multiplayergameInitializationReceive RemotePlayer(s) InputOverall GameControlSendLocal InputGame SessionControlRender sceneto bufferCopy buffer todisplayLocal PlayerInputMain Logic:- consistency- game AI- physics- collisionTime syncSugih Jamin (jamin@eecs.umich.edu)

Consistency: SmoothnessFor smoother playback, decouple bucket size from frame rate(even AoE does this)Immediately render local movesModify game design to allow for latency and loss, e.g.:• make players wait for elevator• teleportation takes time• require multiple hits per kill• let bullet/missile have flying time• build in inertia, don’t allow sudden change in facingSugih Jamin (jamin@eecs.umich.edu)

Reducing Consistency CheckDo area-of-interest management (a.k.a. relevance filtering):• aura: how far you can be sensed(cloaked ships have 0 aura)• nimbus: how far you can sense(use quantum-sensor to detect cloaked ships)Aura and nimbus are defined for a given set of “technology”(e.g., cloaking device, quantum sensor, etc.)Perform consistency check only when B is within A’s nimbus andA is within B’s auraSugih Jamin (jamin@eecs.umich.edu)

CheatingAoE doesn’t need cheat-proofing because each player simulates eachmove in lock stepAll moves are simulated, not just collisionsHalf-Life synchronizes only collisions, higher probability for cheatingCheats (more at megagames.com):• superhuman cheat: auto-aim, auto-position• game-state editing: boost player’s profile• rule bending: see/walk through walls• sixth-sense cheat• lookahead cheat: claim to be behind slow link• suppress-correct cheat: exploit dead-reckoning, claim moves werelost, then “reconstruct” advantageous moves based on others’ movesSugih Jamin (jamin@eecs.umich.edu)

Lookahead CheatPlayer CPlayer HPlayer CPlayer H0t=0t=000t=0050t=50t=505050t=5050time (ms)100150t=100t=100100150100150t=0t=50t=100100150200200200t=100200250250250250a) C is an honest player150ms from Hb) C is a cheater50ms from H claiming tobe 150ms from HSugih Jamin (jamin@eecs.umich.edu)

Suppress-correct CheatAt time 150, C sends out a move consistent with fake moves at time 0, 50,100 that were actually computed upon receiving packets from Player HPlayer CPlayer H050100150XXXt=0t=50t=100050100150200200250250Sugih Jamin (jamin@eecs.umich.edu)

Distributed Computing ModelInitializationOverall GameControlGame SessionControlMessage passing:• player inputs (either buttonpushes or higher-level movements)are sent to other players(or server)• all players update their owngame state• or server updates the globalgame state and send it out to allplayersPlayer inputs sent/received using socketsReceive RemotePlayer(s) InputSendLocal InputRender sceneto bufferCopy buffer todisplayLocal PlayerInputMain Logic:- update states- game AI- physics- collisionTime syncSugih Jamin (jamin@eecs.umich.edu)

Distributed Computing ModelInitializationOverall GameControlGame SessionControlDistributed objects:• characters and environmentmaintained as objects• player inputs are applied to objects(at server)• changes to objects propagatedto all players at end of game loop• object update usually implementedas one or more librarycallsObject update library implemented on socketsSend & ReceiveObject UpdatesRender sceneto bufferCopy buffer todisplayLocal PlayerInputMain Logic:- update objects- game AI- physics- collisionTime syncSugih Jamin (jamin@eecs.umich.edu)

Socket ProgrammingWhat is a socket?How to use socket for client-server computing?Sugih Jamin (jamin@eecs.umich.edu)

Socket Programmingsocket: a data structure containing connection informationConnection identifying information:• client IP (Internet Protocol) address• client port number• source IP address• source port numberClient-server connection:• server creates a socket and listens for connections on a well-knownport number• client creates a socket and connects to the server address at thewell-known port numberSugih Jamin (jamin@eecs.umich.edu)

TCP ConnectionWinSock API TCP ServerWSAStartup()WSAEnumProtocols()socket()bind()listen()accept()blocks untilclient connectsrecv()processrequestsend()closesocket()WSACleanup()connection establishmentdata requestdata replyWinSock API TCP ClientWSAStartup()WSAEnumProtocols()socket()connect()send()recv()closesocket()WSACleanup()Sugih Jamin (jamin@eecs.umich.edu)

server.cint visits;intmain(int argc, char *argv[]){struct sockaddr_in self, client;struct hostent *cp;int sd, td, len;char buf[BLEN];sd = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);memset((char *) &self, 0, sizeof(struct sockaddr_in));self.sin_family = AF_INET;self.sin_addr.s_addr = INADDR_ANY;self.sin_port = htons((u_short) PORT);bind(sd, (struct sockaddr *) &self, sizeof(struct sockaddr_in));listen(sd, QLEN);}while (1) {len = sizeof(struct sockaddr_in);td = accept(sd, (struct sockaddr *) &client, &len);cp = gethostbyaddr((char *) &client.sin_addr, sizeof(struct in_addr), AF_INET);printf("Connected from %s\n", cp->h_name);visits++;sprintf(buf, "This server has been contacted %d time(s).\n", visits);send(td, buf, strlen(buf), 0);close(td);}Sugih Jamin (jamin@eecs.umich.edu)

client.cintmain(int argc, char *argv[]){struct sockaddr_in server;struct hostent *sp;int sd;int n;char buf[BLEN];sd = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);memset((char *) &server, 0, sizeof(struct sockaddr_in));server.sin_family = AF_INET;server.sin_port = htons((u_short) PORT);sp = gethostbyname(SERVER);memcpy(&server.sin_addr, sp->h_addr, sp->h_length);connect(sd, (struct sockaddr *) &server, sizeof(struct sockaddr_in));n = recv(sd, buf, sizeof(buf), 0);while (n > 0) {write(1, buf, n);n = recv(sd, buf, sizeof(buf), 0);}}close(sd);exit(0);Sugih Jamin (jamin@eecs.umich.edu)

includes and definesTo be prepended to both server.c and client.c:#include #include #include #include #include #include #include #include #include #define SERVER "localhost"#define PORT 4897#define BLEN 256#define QLEN 200Sugih Jamin (jamin@eecs.umich.edu)

Socket APIs HighlightsWinSock APIs:http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/winsock functions.aspsocket: creates a socket data structureThen we need to populate the structure with the connection identifyinginformation:• client IP (Internet Protocol) address• client port number• source IP address• source port numberSugih Jamin (jamin@eecs.umich.edu)

TCP Socket AddressesIn the socket structure:IP addressPort#bind()connect()match incomingpkts’ destinationcopy to outgoingpkts’ destinationbind: used by server only, gives the server socket an IP address and/orport#connect:• TCP: initiates connection• udp: remembers remote addressSugih Jamin (jamin@eecs.umich.edu)

TCP Socket AddressesTCP Server:IP addressINADDR_ANYclient’s addressPort#TCP Client:IP addressclient’s addressserver’s addressPort#wellknownephemeralephemeralwellknownSugih Jamin (jamin@eecs.umich.edu)

NAT and FirewallsWhat are NAT (Network Address Translation) and firewalls?Sugih Jamin (jamin@eecs.umich.edu)

NAT and FirewallsNAT boxes remap port numbers (Why?)A1 3Ethernet LAN2 5BN1 2 3 4port#123mappingA.1A.3B.2The Internet2 5RFirewalls may filter out all unknown ports and all UDP packetsSugih Jamin (jamin@eecs.umich.edu)

TCP ConnectionWinSock API TCP ServerWSAStartup()WSAEnumProtocols()socket()bind()listen()accept()blocks untilclient connectsrecv()processrequestsend()closesocket()WSACleanup()connection establishmentdata requestdata replyWinSock API TCP ClientWSAStartup()WSAEnumProtocols()socket()connect()send()recv()closesocket()WSACleanup()Sugih Jamin (jamin@eecs.umich.edu)

Socket APIs Hightlights (cont)listen:• specifies max # of pending TCP connections• only useful for connection oriented services• TCP SYN denial of service attackaccept:• waits for client connection• returns a connected socket (different from the listening socket)Sugih Jamin (jamin@eecs.umich.edu)

Socket APIs Hightlights (cont)send:• returns how many bytes are actually sent• must loop to make sure that all is sent(except for blocking I/O, see UNP Section 6.2)What is blocking and non-blocking I/O?Why do you want to use non-blocking I/O?Sugih Jamin (jamin@eecs.umich.edu)

Different Types of I/OSynchronous: blocks (puts process to sleep) until I/O is readyBy default operations on sockets are blockingWaiting for I/O:1. wait for device availability2. wait for I/O completionSugih Jamin (jamin@eecs.umich.edu)

Non-blocking I/ONon-blocking I/O: keeps on checking (polling) until device is available• set socket non-blocking:int on = 1; ioctlsocket(socket, FIONBIO, &on);• call select on non-blocking socketSignal-driven I/O: process gets a signal when device is available• use WSAAsyncSelect() for signals tied to a window• or WSAEventSelect() for signals not tied to a windowAsynchronous I/O: process notified when I/O completed• Not widely supported yet(See UNP Section 6.2 for more info)Sugih Jamin (jamin@eecs.umich.edu)

TCP ConnectionWinSock API TCP ServerWSAStartup()WSAEnumProtocols()socket()bind()listen()accept()blocks untilclient connectsrecv()processrequestsend()closesocket()WSACleanup()connection establishmentdata requestdata replyWinSock API TCP ClientWSAStartup()WSAEnumProtocols()socket()connect()send()recv()closesocket()WSACleanup()Sugih Jamin (jamin@eecs.umich.edu)

Socket APIs Hightlights (cont)recv:• returns how many bytes are received• 0 if connection is closed, -1 on error• if non-blocking: -1 if no data with errno set to EWOULDBLOCK• must loop to make sure that all is received (in TCP case)• How do you know you have received everything sent?fixed size (part of protocol definition), prior handshakeSugih Jamin (jamin@eecs.umich.edu)

Selectselect(maxfd, readset, writeset, acceptset, timeout)• synchronous (blocking) I/O multiplexing• maxfd is the maximum file descriptor number + 1,so if you have only one descriptor, number 5, maxfd is 6.• descriptor sets provided as bit mask. UseFD ZERO, FD SET, FD ISSET, and FD CLRto work with the descriptor sets• (the fourth parameter is usually called the exceptset)Sugih Jamin (jamin@eecs.umich.edu)

Select (cont)select(maxfd, readset, writeset, acceptset, timeout)• returns as soon as one of the specified socket is ready for I/O• returns # of ready sockets, -1 on error, 0 if timed out and no device isready (what for?)Sugih Jamin (jamin@eecs.umich.edu)

ecv with select vs. PollingWhich of the following would you use? Why?loop {select(. . . , timeout);recv();} till done;or:loop {sleep(seconds)recv();} till done;Sugih Jamin (jamin@eecs.umich.edu)

Socket APIs Hightlights (cont)closesocket:• marks socket unusable• actual tear down depends on TCPif bind() fails, check WSAGetLastError() for WSEADDRINUSESugih Jamin (jamin@eecs.umich.edu)

Socket Options: getsockopt and setsockopt (UNP Ch. 7)• SO REUSEADDR: allows server to restart or multiple servers to bind tothe same port with different IP addresses• SO LINGER: whether close should return immediately or abortconnection or wait for termination• SO RCVBUF and SO SNDBUF: set buffers sizes• SO KEEPALIVE: server pings client periodicallySugih Jamin (jamin@eecs.umich.edu)

UDP Socket ProgrammingServer must always call bind(), but not listen() nor accept().Client doesn’t need to call connect().Use sendto() instead of send().However, connect() can still be used to tell the system to remember theremote address. Then send() instead of sendto() can be used.Call either recv() or recvfrom() to recv. recvfrom() also returnsthe address of the client.UDP packets have boundary, not a byte-stream as in TCP, so recv()retrieves one message at a time, i.e. no need to call recv() in a loop.Sugih Jamin (jamin@eecs.umich.edu)

UDP DatagramWinSock API UDP ServerWSAStartup()WSAEnumProtocols()socket()bind()recvfrom()blocks untildatagram receiveddata requestWinSock API UDP ClientWSAStartup()WSAEnumProtocols()socket()sendto()processrequestsendto()closesocket()WSACleanup()data replyrecvfrom()closesocket()WSACleanup()Sugih Jamin (jamin@eecs.umich.edu)

UDP Socket AddressesUDP Server:IP addressPort#bind()239.4.8.9 9489match incomingpkts’ destinationTo be filled inwith sender’s addr.by kernelUDP Client:IP addressPort#connect()239.4.8.9 9489To be filled in withhost’s IP addr. andephemeral port by kernelcopied to outgoingpkts’ destinationSugih Jamin (jamin@eecs.umich.edu)

UDP with connect()WinSock API UDP ServerWSAStartup()WSAEnumProtocols()socket()bind()recvfrom()blocks untildatagram receivedWinSock API UDP ClientWSAStartup()WSAEnumProtocols()socket()connect()processrequestsendto()data requestdata replysend()recv()closesocket()closesocket()WSACleanup()WSACleanup()Sugih Jamin (jamin@eecs.umich.edu)

Byte OrderingBig-endian: Most Significant Byte (MSB) in low address (sent/arrives first)(Sun Sparc, HP-PA)Little-endian: MSB in high address (sent/arrives later) (Intel x86, PS2)PowerPC and Alpha can be set to either modeMMORG servers and backend servers may live on big-endian machinesSugih Jamin (jamin@eecs.umich.edu)

Byte Ordering (cont)Actual Value 1:MSBLSB00000000 00000001A+1Asent withouthtons and ntohs00000000000000010000000000000001MSBLSBlittle endianA+1A0000000000000001LSBMSBbig endianvalue: 2^8Sugih Jamin (jamin@eecs.umich.edu)

Byte Ordering (cont)To ensure interoperability, ALWAYS translateshort, long, intto (from) “network byte order” before (after) transmissionby using these macros:htons(): host to network shorthtonl(): host to network longntohs(): network to host shortntohl(): network to host longSugih Jamin (jamin@eecs.umich.edu)

Protocol LayersApplicationDomain NameSystem (DNS)PresentationSessionsocket APITransportNetworkData LinkPhysicalSugih Jamin (jamin@eecs.umich.edu)

Naming and AddressingDNS (Domain Name System) name in ASCII string:irl.eecs.umich.eduIP address written out in dotted-decimal (dd) ASCII string:141.213.8.193IP address in 32-bit binary representation:10001101 11010101 00001000 11000001Use names instead of addresses:symbolic, easier to remember, variable length stringBut fixed-length address provides more efficient handling and fastercomparison, uses less memory and less bandwidth (bw)Sugih Jamin (jamin@eecs.umich.edu)

Name and Adress ManipulationSyscalls to map name to/from address:• dns to b: gethostbyname• b to dns: gethostbyaddressand to change representation:• dd to b: inet addr (inet aton)• b to dd: inet ntoadns to dd: gethostbyname plus inet ntoaSugih Jamin (jamin@eecs.umich.edu)

Other useful functions:• gethostname: returns DNS name of current host• getsockname: returns IP address bound to socket (in binary) Usedwhen addr and/or port is not specified (INADDR ANY), to find out theactual addr and/or port used• getpeername: returns IP address of peer (in binary)Debugging Tools: use tcpdump to look at packets on the networkhttp://windump.polito.it/install/Sugih Jamin (jamin@eecs.umich.edu)

tcpdump Output% sudo tcpdump -i fxp0 host tailtcpdump: listening on fxp008:52:07.913485 irl.eecs.umich.edu.3465 > tail.eecs.umich.edu.ssh: S 1334090569:1334090569(08:52:07.913766 tail.eecs.umich.edu.ssh > irl.eecs.umich.edu.3465: S 1738389661:1738389661(08:52:07.913820 irl.eecs.umich.edu.3465 > tail.eecs.umich.edu.ssh: . ack 1 win 57920 irl.eecs.umich.edu.3465: P 1:41(40) ack 1 win 57908:52:07.965857 irl.eecs.umich.edu.3465 > tail.eecs.umich.edu.ssh: P 1:40(39) ack 41 win 5708:52:07.966126 tail.eecs.umich.edu.ssh > irl.eecs.umich.edu.3465: . ack 40 win 57881 tail.eecs.umich.edu.ssh: P 40:584(544) ack 41 win08:52:07.966842 tail.eecs.umich.edu.ssh > irl.eecs.umich.edu.3465: . ack 584 win 57376 irl.eecs.umich.edu.3465: P 41:577(536) ack 584 wi08:52:07.995842 irl.eecs.umich.edu.3465 > tail.eecs.umich.edu.ssh: P 584:608(24) ack 577 wi08:52:07.996143 tail.eecs.umich.edu.ssh > irl.eecs.umich.edu.3465: . ack 608 win 57896 irl.eecs.umich.edu.3465: P 577:1001(424) ack 60808:52:08.146672 irl.eecs.umich.edu.3465 > tail.eecs.umich.edu.ssh: . ack 1001 win 57920 tail.eecs.umich.edu.ssh: P 608:1024(416) ack 100108:52:08.183112 tail.eecs.umich.edu.ssh > irl.eecs.umich.edu.3465: . ack 1024 win 57504 irl.eecs.umich.edu.3465: P 1001:1929(928) ack 10208:52:08.656695 irl.eecs.umich.edu.3465 > tail.eecs.umich.edu.ssh: . ack 1929 win 57920 tail.eecs.umich.edu.ssh: P 1024:1040(16) ack 192908:52:08.755369 tail.eecs.umich.edu.ssh > irl.eecs.umich.edu.3465: . ack 1040 win 57904 tail.eecs.umich.edu.ssh: P 1040:1088(48) ack 192908:52:08.755683 tail.eecs.umich.edu.ssh > irl.eecs.umich.edu.3465: . ack 1088 win 57872 irl.eecs.umich.edu.3465: P 1929:1977(48) ack 108808:52:08.756654 irl.eecs.umich.edu.3465 > tail.eecs.umich.edu.ssh: P 1088:1152(64) ack 1977Sugih Jamin (jamin@eecs.umich.edu)

SourcesStevens, R., Unix Network Programming, 2nd. or 3rd. ed., Prentice-Hall,2004. All you ever want to know about socket programming, even if you’reusing WinSock.Mulholland & Hakala, Programming Multiplayer Games, WordwarePublishing, 2004. Useful mainly for the WinSock coverage.Bettner, P. and Terrano, M., “1500 Archers on a 28.8: NetworkProgramming in Age of Empires and Beyond,” GDC 2001Bernier, Y.W., “Latency Compensating Methods in Client/Server In-gameProtocol Design and Optimization,” GDC 2001Rothschild, J., “Mpath: Technical Considerations,” Mpath Interactive, 1997Sugih Jamin (jamin@eecs.umich.edu)

Sources (cont)Spohn, D., “Internet Game Timeline,” 2003http://internetgames.about.com/library/weekly/aatimelinea.htmhttp://internetgames.about.com/cs/gamingnews/a/goty2003.htmBartle, R., “Early MUD History,” 1990http://www.mud.co.uk/richard/mudhist.htmNg, Y-S., “Internet Game Design,” Gamasutra, Aug. 1, 1997Ng, Y-S., “Designing Fast Action Games for the Internet,” Gamasutra,Sept. 5, 1997Rosedale and Ondrejka, “Enabling Player-Created Online Worlds withGrid Computing and Streaming,” Gamasutra, Sep. 18, 2003Sugih Jamin (jamin@eecs.umich.edu)

Sources (cont)Filstrup, B., Cronin, E., and Jamin, S., “An Evaluation of Cheat-ProofingMethods for Multiplayer Games,” NetGames 2002Brockington and Greig, “Neverwinter Nights Client/Server Postmortem,”GDC 2003, Gamasutra Mar. 6, 2003Isensee and Ganem, “Developing Online Console Games,” Gamasutra,Mar. 28, 2003Palm, “The Birth of the Mobile MMOG,” Gamasutra, Sep. 19, 2003Sugih Jamin (jamin@eecs.umich.edu)