CPS110: Landon Cox Page 1 of 15 Networks and distributed computing Hardware reality Lots of different manufacturers of NICs Network card has a fixed MAC address, e.g. 00:01:03:1C:8A:2E Send packet to MAC address (max size 1500 bytes) Packets may be reordered, corrupted, dropped, duplicated Anyone can sniff the packets from the network What abstractions does the OS provide for network communication? Distributed computing: making multiple computers look more like a single computer Distributed shared memory: make multiple memories look like one memory Remote procedure call, process migration, parallelizing compilers: make multiple CPUs look like one CPU Distributed file systems: make disks on multiple computers look like one file systemCPS110: Landon Cox Page 2 of 15 Abstractions and protocol layers Why build up abstractions in layers? Routing Hardware interface: deliver to neighbor computer on LAN Application interface: deliver to final destination through several hops Provided by the IP (Internet Protocol) layer Messages on LAN (e.g. Ethernet) are sent via the physical ID of the network interface card (e.g. 0:a0:c9:95:f5:58) NFS HTTP e-mail ssh RPC TCP UDP IP Ethernet ATM ppp Computer 1 Computer 2 Computer 3 Ethernet switchCPS110: Landon Cox Page 3 of 15 Internet is composed of lots of connected LANs How does computer on Ethernet D know how to get to Ethernet H? Should it send to C, F, or I? This is hard because the Internet has not centralized state. Routing is difficult for large systems, and for systems that change rapidly. The Internet is both large and dynamic. Basic idea: routers propagate information to each other and hope the global picture converges before it is out of date. Try running “traceroute www.cnn.com” Symbolic naming Low-level interface: destination specified by MAC address, e.g. 00:01:03:1C:8A:2E Middle-level interface: destination specified by IP address, e.g. 152.3.136.233 High-level interface: name destination by hostname, e.g. duke-cs-136-dhcp-233.cs.duke.edu Translation from IP address to MAC address is provided by ARP protocol (address resolution protocol) How does computer 1 know computer 2’s Ethernet (MAC) address? Start with the IP address for the computer (e.g. 152.3.136.233) Use ARP (Address Resolution Protocol) to get the mapping from IP address to MAC address To find out mapping from IP address to MAC addess (e.g. 152.3.136.233 maps to 00:01:03:1C:8A:2E), ARP broadcasts to all computers on the LAN Cache the mapping for next time Use broadcast when all else fails (but this doesn’t scale well) A B C D E I F G HCPS110: Landon Cox Page 4 of 15 Translation from hostname to IP address is provided by DNS (domain name service) Used to be done with one central server Central server has to learn about all changes Central server has to answer all lookups Split up the data into a hierarchical database (each DNS server stores part of the database). Hierarchy allows local management so everybody doesn’t notify one central server whenever their hostname changes), and spreads the lookup work across multiple servers. Example: translating www.cs.duke.edu Start with the (well-known) IP address of the root name server (A.ROOT-SERVERS.NET, 198.41.0.4) Ask root name server for IP address of the edu name server (also A.ROOT-SERVERS.NET, 198.41.0.4) Ask edu name server for the IP address of the duke.edu name server (dukedns1.netcom.duke.edu, 152.3.250.1) Ask duke.edu name server for the IP address of the cs.duke.edu name server (duke.cs.duke.edu, 152.3.140.1) Ask cs.duke.edu name server for the IP address of www.cs.duke.edu: 152.3.140.5 Message size Hardware interface: physical network type limits size of a message (e.g. Ethernet maximum packet size is 1500 bytes) Application interface: can send larger message (e.g. IP maximum packet size is 64KB) IP layer can fragment a packet when it’s larger than the next hop’s MTU (maximum transmission unit), then re-assemble it at the destinationCPS110: Landon Cox Page 5 of 15 Sockets and ports Hardware interface: machine-to-machine communication (one network endpoint per machine) Application interface: process-to-process communication (one or more network endpoints per process) A process can ask the OS to create a “socket” which will be one endpoint of a network connection Thread is like a virtual processor Address space is like a virtual memory An endpoint (socket) is like a virtual network interface card Each socket on a computer has a unique “port” number A process can associate a specific port number with a socket using the bind call When sending to a socket, the destination port number is included in each message. This allows the destination machine to know which process (and which socket in the process) should receive the message. The OS can multi-plex several network connections into a single physical card UDP (user datagram protocol) provides this process-to-process abstraction on top of IP TCP (transmission control protocol) is also built on IP Provides additional abstractions beyond UDP: ordered, reliable, byte streams Process A Socket 1 Socket 2 Process B Socket 3 Operating system Network interface cardCPS110: Landon Cox Page 6 of 15 Ordered messages Hardware interface: networks can re-order messages that IP layer sends out E.g. Send: A, B. Arrive: B, A Application interface: all messages are received in the order in which they were sent How to provide ordered messages? To have a notion of order, we must first have the notion of a network “connection” (so we know that multiple messages are related) With TCP, process open a connection (using connect), then send a sequence of messages, then closes the connection Sequence # is specific to socket-to-socket connection Reliable messages Hardware interface: networks can drop, duplicate, or corrupt messages Application interface: each message is delivered exactly once, without corruption How to fix a dropped message? How does sender know a message has been dropped?CPS110: Landon Cox Page 7 of 15 Duplicate messages are easy to detect (look at the sequence #) and fix (just drop the duplicate) To detect corrupted