i o multiplexing On adding a poll method to our character mode device driver for an 82573L network controller An application idea We want to create an application program that would allow two users to conduct an on line chat session while working from two separate nodes on our anchor cluster Whatever either user decides to type will be displayed on the both users screens To keep their two input streams visually separated we want to use two windows netchat cpp Whatever a user types will be displayed in the lower window on that user s screen and in the upper window on the other user s screen Hello Susan Hi Paul Hi Paul Hello Susan anchor01 anchor02 Multiplexed input streams In order to implement the foregoing idea it s necessary for the netchat program to accept input from TWO hardware devices the keyboard i e standard input device the ethernet controller i e dev nic Such a situation is a fairly common one in UNIX Linux application programming and is referred to as i o multiplexing Device driver s role Special software support is needed within each of the device drivers in order for any application program to do i o multiplexing in a way that avoids wasting of CPU time For an excellent discussion of the various approaches that can be taken to deal with this basic programming issue see Richard Stevens Advanced Programming in the UNIX Environment Chapter 12 The basic problem Normally when an application reads from a device file that process will sleep until some data is available from that device So if data becomes available on another device it will not get processed because the application is blocked from being given any CPU time by the OS scheduler This would spoil our netchat application read causes blocking read terminal write netchat application write read Ethernet controller Whichever device the application attempts to read from it will get blocked until that device has some data to deliver Do multiprocessing One idea for getting around this blocking problem would be to just use the fork system call to create a separate process for reading from the different device files Each process can sleep and whichever process receives any new data will be awakened and scheduled for execution No changes needed to device driver code Different processes do read read netchat parent process write Ethernet controller terminal write netchat child process read Using multiple processes can overcome the blocking read problem but complicates the code for program termination Non blocking read It is possible for the application to request non blocking read operations i e any read calls will immediately return with 0 as return value in case no data is available The standard input device driver already has support for this non blocking option and it can be easily added to the read function in network controller s driver Code modification ssize t my read struct file file char buf size t len loff t pos static int rxhead 0 in case no new data has been received then either return immediately if non blocking mode is in effect or else sleep until some new data arrives or until the user hits CONTROL C to cancel execution if rxhead ioread32 io E1000 RDH if file f flags O NONBLOCK return 0 if wait event interruptible wq rx rxring ioread32 io E1000 RDH return EINTR Uses busy waiting loop read terminal write netchat application write read Ethernet controller Using the nonblocking read option overcomes the problem of a sleeping task but it wastefully consumes the CPU time The elegant solution The select system call provides a very general scheme for doing i o multiplexing in a manner that avoids wasting CPU time or making the program code complicated But it does require adding an extra driver method the so called poll function The select arguments Using select requires an application to setup an fd set object which defines the set of file descriptors whose activity needs to be monitored by the Linux kernel in our netchat application this would be just the two device files the console keyboard and the gigabit ethernet network controller This fd set object becomes an argument Using select in netchat int int kbd STDIN FILENO aux open dev nic O RDWR fd set permset FD ZERO permset FD SET kbd permset FD SET aux permset keyboard ID device file ID create an fd set object initialize it to empty add keyboard to set and add the nic to set while 1 fd set readset permset if select 1 aux readset NULL NULL NULL 0 break if FD ISSET kbd readset process keyboard input if FD ISSET aux readset process network input How it works The readset argument to the select system call lets the kernel know which device drivers should have their poll method invoked Then each device driver s poll method will perform a test to determine if any new data is ready to be read from that device So the application calls read only when a device is ready with data immediately struct file operations We need to include the function pointer to our implementation for the poll method struct file operations my fops owner read write ioctl poll THIS MODULE my read my write my ioctl my poll Our driver s poll method Linux provides helper functions to do most of the supporting work for use of select include linux poll h for the poll wait helper function unsigned int my poll struct file file struct poll table struct wait unsigned int mask 0 poll wait file wq recv wait if ioread32 io E1000 RDH rxhead mask POLLIN POLLRDNORM return mask The ncurses library Our netchat application uses the Linux implementation for the UNIX curses API This function library includes routines that not only can manage the console s cursor but also implements raw keyboard input for instantaneous keystroke processing and allows drawing of window borders Compiling netchat The g compiler can compile and link in a single command g netchat cpp l ncurses o netchat The C source file Name of the library Lowercase letter L for library to link with Name for the compiler s output file Try it out We can try running the netchat program on any pair of our anchor cluster stations Of course we will first need to install our nicpoll ko device driver module on each of those anchor machines Then a user at either machine can type in any messages and every character typed will be visible immediately on both screens But There is a slight problem with using the present version of our nicpoll c module All of the network packets are broadcast to every station on the anchor cluster So any third party who is