M a c h i n e E t h i c s The Nature Importance and Difficulty of Machine Ethics James H Moor Dartmouth College T he question of whether machine ethics exists or might exist in the future is difficult to answer if we can t agree on what counts as machine ethics Some might argue that machine ethics obviously exists because humans are machines and humans have ethics Others could argue that machine ethics obviously doesn t exist because Implementations of machine ethics might be possible in situations ranging from maintaining hospital records to overseeing disaster relief But what is machine ethics and ethics is simply emotional expression and machines can t have emotions A wide range of positions on machine ethics are possible and a discussion of the issue could rapidly propel us into deep and unsettled philosophical issues Perhaps understandably few in the scientific arena pursue the issue of machine ethics You re unlikely to find easily testable hypotheses in the murky waters of philosophy But we can t and shouldn t avoid consideration of machine ethics in today s technological world As we expand computers decision making roles in practical matters such as computers driving cars ethical considerations are inevitable Computer scientists and engineers must examine the possibilities for machine ethics because knowingly or not they ve already engaged or will soon engage in some form of it Before we can discuss possible implementations of machine ethics however we need to be clear about what we re asserting or denying how good can it be Varieties of machine ethics When people speak of technology and values they re often thinking of ethical values But not all values are ethical For example practical economic and aesthetic values don t necessarily draw on ethical considerations A product of technology such as a new sailboat might be practically durable economically expensive and aesthetically pleasing absent consideration of any ethical values We routinely evaluate technology from these nonethical normative viewpoints Tool makers and users regularly evaluate how well tools accomplish the purposes for 18 1541 1672 06 20 00 2006 IEEE Published by the IEEE Computer Society which they were designed With technology all of us ethicists and engineers included are involved in evaluation processes requiring the selection and application of standards In none of our professional activities can we retreat to a world of pure facts devoid of subjective normative assessment By its nature computing technology is normative We expect programs when executed to proceed toward some objective for example to correctly compute our income taxes or keep an airplane on course Their intended purpose serves as a norm for evaluation that is we assess how well the computer program calculates the tax or guides the airplane Viewing computers as technological agents is reasonable because they do jobs on our behalf They re normative agents in the limited sense that we can assess their performance in terms of how well they do their assigned jobs After we ve worked with a technology for a while the norms become second nature But even after they ve become widely accepted as the way of doing the activity properly we can have moments of realization and see a need to establish different kinds of norms For instance in the early days of computing using double digits to designate years was the standard and worked well But when the year 2000 approached programmers realized that this norm needed reassessment Or consider a distinction involving AI In a November 1999 correspondence between Herbert Simon and Jacques Berleur 1 Berleur was asking Simon for his reflections on the 1956 Dartmouth Summer Research Project on Artificial Intelligence which Simon attended Simon expressed IEEE INTELLIGENT SYSTEMS some puzzlement as to why Trenchard More a conference attendee had so strongly emphasized modal logics in his thesis Simon thought about it and then wrote back to Berleur My reply to you last evening left my mind nagged by the question of why Trench Moore sic in his thesis placed so much emphasis on modal logics The answer which I thought might interest you came to me when I awoke this morning Viewed from a computing standpoint that is discovery of proofs rather than verification a standard logic is an indeterminate algorithm it tells you what you MAY legally do but not what you OUGHT to do to find a proof Moore sic viewed his task as building a modal logic of oughts a strategy for search on top of the standard logic of verification Simon was articulating what he already knew as one of the designers of the Logic Theorist an early AI program A theorem prover must not only generate a list of well formed formulas but must also find a sequence of well formed formulas constituting a proof So we need a procedure for doing this Modal logic distinguishes between what s permitted and what s required Of course both are norms for the subject matter But norms can have different levels of obligation as Simon stresses through capitalization Moreover the norms he s suggesting aren t ethical norms A typical theorem prover is a normative agent but not an ethical one Ethical impact agents You can evaluate computing technology in terms of not only design norms that is whether it s doing its job appropriately but also ethical norms For example Wired magazine reported an interesting example of applied computer technology 2 Qatar is an oil rich country in the Persian Gulf that s friendly to and influenced by the West while remaining steeped in Islamic tradition In Qatar these cultural traditions sometimes mix without incident for example women may wear Western clothing or a full veil And sometimes the cultures conflict as illustrated by camel racing a pastime of the region s rich for centuries Camel jockeys must be light the lighter the jockey the faster the camel Camel owners enslave very young boys from poorer countries to ride the camels Owners have historically mistreated the young slaves including limiting their food to keep them lightweight The United Nations and the US State Department have objected to this human traffickJULY AUGUST 2006 ing leaving Qatar vulnerable to economic sanctions The machine solution has been to develop robotic camel jockeys The camel jockeys are about two feet high and weigh 35 pounds The robotic jockey s right hand handles the whip and its left handles the reins It runs Linux communicates at 2 4 GHz and has a GPS enabled