Chapter 24. Meeting 24, Discussion: Aesthetics and Evaluations 24.1. Announcements • Sonic system reports due and presentations begin: 11 May 24.2. Quiz Review • ? 24.3. The (Real) Turing Test • Turing, A. M. 1950. “Computing Machinery and Intelligence.” Mind 59: 433-460. 267© Oxford University Press. All rights reserved. This content is excluded from our Creative Commons license. For more information, see http://ocw.mit.edu/fairuse. 268• A test of human and computer indistinguishability • Based on a party game in which an interrogator attempts to distinguish the gender of two human agents • Through removing biases (sound, visual presence), and focusing on language alone, can a machine be indistinguishable from a human? • Multiple tests can be averaged; after 5 minutes of conversation correct identification must be less than 70 percent • Claim only of achieving thinking, not intelligence • Functional rather than structural indistinguishability (2000, p. 429) 269 Image by MIT OpenCourseWare.?Computer96-G2KComputer• Deception is permitted: mathematical questions can take longer, or fake mistakes • Is human-like conversation the sole determinate of thinking? 24.4. The Eliza Effect • Humans too easily associate humanity with machines • Eliza in emacs: shift + escape; enter “xdoctor” and return 24.5. Other Tests: The John Henry Test • The John Henry Test (JHT): a test of verifiable distinguishability between human and machine • Other examples? 24.6. Other Tests: The Turing Hierarchy • Steven Harnad • Total Turing Test: full physical and sense based interaction • T4: internal microfunctional indistinguishability • T5: microphysical indistinguishability, real biological molecules • t1: toy tests: subtotal fragments of our functional capacity (Harnad 2000, p. 429) • The TT is predicated on total functional indistinguishability; anything less is a toy 24.7. A Little Turing Test • Hofstadter, D. R. 1979. Gödel, Escher, Bach: an eternal golden braid . New York: Vintage. • The little turing test (1979, p. 621) 270© Vintage Books. All rights reserved. This content is excluded from our Creative Commons license. For more information, see http://ocw.mit.edu/fairuse. 271• Is this a Turing Test? 24.8. A (Kind of) Turing Test • Kurzweil, R. 1990. The Age of Intelligent Machines. Cambridge: MIT Press. • “The essence of the Turing Test is that the computer attempts to act like a human within the context of an interview over terminal lines. A narrower concept of a Turing test is for a computer to successfully imitate a human within a particular domain of human intelligence. We might call these domain-specific Turing tests. One such domain-specific Turing test, based on a computer’s ability to write poetry, is presented here.” (1990, p. 374) • 28 question “poetic Turing test” administered to 16 human judges; 48 percent correct overall • Cybernetic Poet http://www.kurzweilcyberart.com/poetry/rkcp_akindofturingtest.php • “Music composed by computer is becoming increasingly successful in passing the Turing test of believability. The era of computer success in a wide range of domain-specific Turing tests is arriving.” (1990, p. 378) • Kurzweil and Kapor Long Bet: 20,000 that a machine will pass the Turing Test by 2029 • Is there a narrower concept of a Turing Test? 24.9. A Musical Turing Test • Compare chants created by computer and by humans 272© source unknown. All rights reserved. This content is excluded from our Creative Commons license. For more information, see http://ocw.mit.edu/fairuse. • Is this a Turing Test? • How would this test be different if the music was performed by humans? 24.10. Musical Turing Test Archetypes • Musical Directive Toy Test (MDtT) • Musical Output Toy Test (MOtT) • The problem of musical judgements • Music is not natural language • We have aesthetic expectations for human and computer music • All executed tests report a win for the computer • Does success of a MDtT or a MOtT offer a sign of system design success? • Does aesthetic success suggest system design success? 27324.11. Discrimination Tests • Blind comparison of musical outputs • Often material used to create the music is used as part of the test • All listening test are bound by musical judgements 24.12. Cope’s MOtTs • Cope does not associate these test directly with the TT • Compares EMI generated Mozart with Mozart • 1992 AAAI conference conducted a test with 2000 visitors, claiming “absolutely no scientific value” but claims that “machine-composed music has some stylistic validity” • Compares virtual music to real music in The Game • Many have used Cope’s music or related tests as examples of musical TTs where the machine wins 24.13. Machine Authorship in Generative Music Systems • Is the machine responsible for the musical output? • Is the test testing the machine at all? 24.14. Aesthetic Intention in Generative Music Systems • The intentional fallacy: the idea that understanding the artist’s intention is necessary for evaluating a work (Beardsley 1946) • Is intention required to make music? • Can authorship be given to things that do not have intention? 24.15. Listening • Listening: David Soldier, “The Birth of Ganesha,” Elephonic Rhapsodies, 2004 274• Elephants trained and directed in improvisation with instruments 24.16. Naughtmusik • Soldier, D. 2002. “Eine Kleine Naughtmusik: How Nefarious Nonartists Cleverly Imitate Music.” Leonardo Music Journal 12: 53-58. • Genuine music requires composers with intent • Naughtmusik: nonart sounds, composers without intent • An Adapted Turing Test: can human judges detect naughtmusik? • The Tangerine Awkestra: children 2 to 9, produce sounds using instruments they do not know how to play, recorded in a studio; listened to free jazz of Ornette Coleman and others • 5 sophisticated adults: 5 of 8 trials led to correct identification: not iron-clad • Thai elephant orchestra • “There is something out there that looks, sounds, feels, smells like music, but isn't” (2002, p. 58) 24.17. Listening • The People’s Choice Music: with Vitaly Komar and Alex Melamid • Survey given to 500 Americans • Survey responders had no intent; the works were created without individual intent, and thus no creative decision making was