An ARM Backend for PyPyls Tracing JIT - STUPS Group

More documents

Recommendations

Info

38 8 EVALUATION8 EvaluationFigure 23: BeagleBoard-xM development boardIn order to evaluate the performance of the ARM backend and to analyze how it behavescompared to PyPy on x86 in this section we want to present the results of different benchmarks.We gathered results from benchmarks executed on PyPy’s Python interpreter andon Pyrolog [BLS10] a Prolog interpreter written in RPython.8.1 HardwareAll tests and benchmarks on ARM were run on an otherwise idle BeagleBoard-xM 7 runningUbuntu 10.10 for ARM 8 .The BeagleBoard-xM is an ARM based development board with an ARM Cortex A8 CPUat 1 GHz and with 512MB of memory with Linux 2.6.35. See Figure 23 for a picture of theboard.The benchmarks on x86 were performed on an otherwise idle Intel Core2 Duo P8400processor with 2.26 GHz and 3072 KB of cache on a machine with 3GB RAM runningLinux 2.6.35.8.2 BenchmarksTo measure the performance of the ARM backend we performed two different sets ofbenchmarks, one running the PyPy benchmarks and one running the Prolog benchmarksdescribed in [BLS10]. All interpreters were translated using a fixed revision(370c23f085d7) of the ARM branch of PyPy’s mercurial repository 9 .7 http://beagleboard.org/hardware-xM8 https://wiki.ubuntu.com/ARM9 https://bitbucket.org/pypy/pypy/src/arm-backend-2
8.2 Benchmarks 398.2.1 Python BenchmarksTo run the PyPy Python benchmarks we created a set of different versions of PyPy’sPython interpreter. We created a version of PyPy with and without the JIT for ARMboth using the Boehm-Demers-Weiser garbage collector [BW88] (wich we will refer to asBoehm GC) to compare them against CPython on ARM. Additionally we created fourversions of PyPy’s Python interpreter for x86. We created interpreter versions with allcombinations of JIT, no JIT, Boehm GC and PyPy’s own GC. The CPython version onARM and x86 is version 2.6.6 revision r266:84292. The benchmarks are a mix of smalland medium sized python programs composed of synthetic benchmarks and real applications.We had to remove the spambayes, spitfire, pyflate-fast and rietvield benchmarksfrom the full set, because they were hitting an issue in the ARM backend that is stillopen. The full set of benchmarks as well as the current results on x86 can be seen onthe PyPy Speed Center website 10 , the benchmark’s sourcecode can be found at the PyPybenchmarks repository 11 . All benchmarks were run 50 times in the same instance of theVM, taking the average of all runs so that they contain interpretation, warm-up, codegenerationand execution of the generated code. The errors were calculated using a confidenceinterval with a 95% confidence level [GBE07].The results for the benchmarks run on the BeagleBoard are shown below. In Table 1 wepresent the absolute runtimes for the different benchmarks. Table 2 shows the relativeruntimes which as well as the graphed data in Figure 24 are normalized to CPython.To put the results gathered on ARM into context we also performed the same benchmarkson x86 with all four PyPy versions mentioned above. The graph of the relative speedcompared to CPython on x86 can be seen in Figure 25. The tables with the absolute (seeTable 6) and the relative results (see Table 7) can be found in Section 11. The benchmarksshow that running PyPy with the Boehm GC is always, with one exception, slower thanthe corresponding version of PyPy with its own GC. Also on x86 the versions using theBoehm GC and the JIT are almost always faster than the version with the Boehm GC andno JIT.Looking at the graph in Figure 24 it can be seen that on ARM the JIT almost alwaysimproves over the non-jitted version of PyPy. At the same time the results comparedto CPython are not that clear. While PyPy without a JIT is always slower that CPythonthe jitted version only outperforms CPython in about half of the benchmarks. In somebenchmarks the jitted version is significantly slower than CPython, even being slowerthan PyPy without a JIT on the twisted_pb benchmark. The mixed results on ARM andthe results on x86, especially those for the versions of PyPy using the Boehm GC, showthe importance of a fast GC that is well integrated with the JIT to get good results.An important result can be seen when comparing the relative results calculated for bothplatforms. When looking at the relative results comparing the PyPy versions to CPythonon both platforms, as shown in Table 3, we see that on ARM we get relative results thatare comparable to those achieved on x86 for the interpreters using the Boehm GC withand without the JIT. The cases where the relative results differ might be due to severalmicro-optimizations that have not yet been done in the ARM backend as well as some10 http://speed.pypy.org11 http://codespeak.net/svn/pypy/benchmarks/
Page 1: INSTITUT FÜR INFORMATIKSoftwaretec
Page 5: AbstractA large part of the computi
Page 9 and 10: 31 IntroductionARM cores are presen
Page 11 and 12: 52 ARMARM is at the same time the n
Page 13 and 14: 2.2 The ARM Architecture 7The curre
Page 15 and 16: 2.2 The ARM Architecture 9and from
Page 17 and 18: 113 Just-in-Time CompilationJust-in
Page 19 and 20: 3.2 Trace Based Just-In-Time Compil
Page 21 and 22: 154 PyPyThe PyPy project was starte
Page 23 and 24: 175 PyPy’s Approach to Tracing JI
Page 25 and 26: 5.1 The Shape of a Trace 19program
Page 27 and 28: 5.2 Optimizations 21loop_start(a0,
Page 29 and 30: 236 A Backend for PyPy’s JITIn th
Page 31 and 32: 6.1 Low level Code Generation Inter
Page 33 and 34: 6.2 Register Allocation 27def gen_l
Page 35 and 36: 6.3 Setup to Execute a Compiled Loo
Page 37 and 38: 6.4 Compiling Trace Operations 31Ac
Page 39 and 40: 6.4 Compiling Trace Operations 33of
Page 41 and 42: 6.5 Guards and Leaving the Loop 35o
Page 43: 37ARM using a cross compiler, such
Page 47 and 48: 8.2 Benchmarks 41Benchmark cpython
Page 49 and 50: 8.2 Benchmarks 43Ratio6x5x4x3x2x1x0
Page 51 and 52: 8.2 Benchmarks 458.2.2 Prolog Bench
Page 53 and 54: 8.2 Benchmarks 47SWInojit, boehmjit
Page 55 and 56: 499 Related WorkThere are many virt
Page 57 and 58: 5111 AnnexBenchmark cpython [ms] no
Page 59 and 60: REFERENCES 53References[AACM07] ANC
Page 61 and 62: REFERENCES 55annual ACM SIGPLAN con
Page 63 and 64: LIST OF FIGURES 57List of Figures1

An ARM Backend for PyPyls Tracing JIT - STUPS Group

Create successful ePaper yourself

Delete template?

Save as template?