Performance of Scripting LanguagesNate NystromUTASaturday, May 8, 2010Static vs. dynamic typing• Static typing:• great for ensuring interfaces used correctly, enabling optimizations, enforcing security policies, ...• tiresome to write, difficult to modify, extend• Dynamic typing:• great for rapid development, extension• but, brittle• My current research: Thorn• an OO scripting language for distributed systems•supports gradual typing:• add types gradually as the program grows• best of both worldsSaturday, May 8, 2010Thorn• My current research: Thorn• an OO scripting language for distributed systems•supports gradual typing:• add types gradually as the program grows• best of both worlds• Joint work with colleagues at IBM and Purdue:• Bard Bloom, John Field, Jan Vitek, Tobias Wrigstad, and othersSaturday, May 8, 2010Dynamic language implementation• Decided early on to implement Thorn on the JVM• How should dynamic languages be implemented on the JVM?• How well do current dynamic languages perform?Saturday, May 8, 2010Taxonomy• Terms often used interchangeably (I will do this)• No hard and fast definitions, but I’ll try anyway:• Dynamic languages• support run-time code or type extension• e.g., eval, dynamic inheritance• Dynamically typed languages• type-check at run-time• Scripting languages• languages used for “scripting” in some domaindynamicdynamically typedscriptingSaturday, May 8, 2010Characteristics• Scripting languages are usually...• dynamically typed (or untyped)• dynamic (e.g,. they provide a read-eval-print loop)• interpreted• high level• and are often...• domain-specificSaturday, May 8, 2010Scripting languagesLanguageDomainAbstractionssh, csh, ...UNIXpipes, redirectionAWKtext filesstrings, regexesApplescriptMac applicationsapplication dictionariesJavascriptclient-side webDOMUnrealScript3D gamesactors, lightingActionScriptFlashimages, movies, soundPHPserver-side webHTMLGroovyJavaJava objects, lists, mapsPerl, Python, Rubygeneral purposeobjects, lists, mapsSaturday, May 8, 2010Examplesputs "hello world"RubyPerlPython GroovyScalaPHPprint "hello world"print "hello world\n";println "hello world"<?phpprint "hello world\n";?>object hello extends Application { Console.println("hello world")}class hello { public static void main(String[] args) { System.out.println(“hello world”); }}JavaThornprintln(“hello world”);Saturday, May 8, 2010Examplesm = {}m{“one”} = 1RubyPerlPythonScala%m = ();$m{“one”} = 1;def m = [:]m[“one”] = 1val m = new HashMap[String,Int]();m += “one” -> 1;Map<String,Integer> m = new HashMap<String,Integer>();m.put(“one”, 1);Javam = {}m{“one”} = 1GroovyPHP$m = array();$m[“one”] = 1;Thornval m = Map();m(“one”) = 1;Saturday, May 8, 2010Performance• 9 language implementations• Ruby, Python, Perl, PHP (C interpreters)• JRuby, Jython, Groovy (Java interpreters)• Java, Scala (Java compilers)• 42 programs from the Programming Language Shootout site*• All programs small, short running (< 10 min)• Caveats:• Not all programs ported to all languages• Sometimes different implementation strategies used• Setup:• Macbook Pro, 2.4GHz Intel Core Duo, 2GB RAM• JVM: HotSpot JVM 1.5.0, 512MB heap*http://shootout.alioth.debian.orgSaturday, May 8, 2010Reading the graphs05101520ruby python perl php java scala jruby jython groovy16.008.004.002.001.000.500.250.130.06C interpreters Java compilersJava interpretersless time orless memoryJava = 1.0Saturday, May 8, 2010Run times / Java012.525.037.550.0ruby python perl php java scala jruby jython groovy42.4616.4728.911.931.004.272.273.993.60All benchmarks / Java (geo mean)0255075100ruby python perl php java scala jruby jython groovy96.1715.5782.0112.981.0014.819.329.4517.90Fannkuch time / Java0100200300400ruby python perl php java scala jruby jython groovy00375.631.271.0069.2922.1221.5064.46Fasta time / Java037575011251500ruby python perl php java scala jruby jython groovy419.5237.121187.6401.0020.2426.8319.42105.08Mandelbrot time / Java• Dynamic languages oftenmuch slower than Java• C interpreters: ~2-5x• can be 12x faster, 145x slower• Java interpreters: ~16-43x• up to 1200x slowerSaturday, May 8, 2010Run times / Java0255075100ruby python perl php java scala jruby jython groovy96.1715.5782.0112.981.0014.819.329.4517.90Fannkuch time / Java0100200300400ruby python perl php java scala jruby jython groovy00375.631.271.0069.2922.1221.5064.46Fasta time / Java037575011251500ruby python perl php java scala jruby jython groovy419.5237.121187.6401.0020.2426.8319.42105.08Or, to scale:Saturday, May 8, 2010C interpreter performance• Overall 2-5x slower than Java• Implementation:• Ruby, Perl, PHP: AST walking interpreter• Python: bytecode (Pycode) interpreter• Java/Scala: bytecode interpreter + run-time compilation• Could improve by adopting same techniques as JVMs(and Self before that)• difficult, time-consuming to engineer, maintain• not very portable• Better (perhaps): dynamically compile to bytecode, run on JVMSaturday, May 8, 2010Scala• Scala used as a proxy for “best possible” performance of typical scripting language• Has many of the same features (e.g., closures, iterators) as Python, Ruby, etc• Statically compiled to Java bytecode• ~2x slower than JavaSaturday, May 8, 2010Java interpreter performance• Jython, Groovy implementation:• dynamic compilation to Java bytecode• JRuby:• AST interpreter in Java• JRuby, Jython ~4-8x slower than Ruby, Python• Overall 16-43x slower than Java• Mandelbrot: JRuby 1200x, Groovy 420x slower• Should be able to approach Scala performance with better implementationsSaturday, May 8, 2010Does it matter?• Often, no• Many scripts short running• Many scripts are I/O bound• database• network• other processes• But, when performance does matter:• Often rewrite applications in Java or C• Lose benefits of programming in high-level language• For server-side web applications: scalability matters• Want fast startup, low memory usageSaturday, May 8, 2010Why so slow on the JVM?• Startup costs• Object model mismatch• Duck typing• High-level language features: iterators, closuresSaturday, May 8, 2010Startup time01.53.04.56.0ruby python perl php java scala jruby jython groovy5.015.975.571.971.000.100.050.270.08Hello time / JavaHello, WorldC interpreters• 4-20x faster than JavaJava interpreters• 5-6x