Giter Site home page Giter Site logo

dacapobench / dacapobench Goto Github PK

View Code? Open in Web Editor NEW
151.0 151.0 59.0 237.37 MB

The DaCapo benchmark suite

Home Page: https://www.dacapobench.org/

License: Apache License 2.0

Python 4.85% Java 78.79% HTML 0.62% Perl 2.69% CSS 0.13% Shell 2.59% Batchfile 1.21% XSLT 9.05% C 0.06%

dacapobench's Introduction

The DaCapo Benchmark Suite

Last updated 2023-11-08

This benchmark suite is intend as a tool for the research community. It consists of a set of open source, real world applications with non-trivial memory loads.

Guidelines for use

When quoting results in publications, the authors of this suite strongly request that:

  • The exact version of the suite be given (number & name, eg 'dacapo-23.11-chopin')

  • The suite be cited in accordance with the usual standards of acknowledging credit in academic research.

  • Please cite the 2006 OOPSLA paper or a more recent paper by the DaCapo authors providing an up-to-date description of the suite if and when such a paper becomes available.

  • All command line options used be reported. For example, if you explicitly override the number of threads or set the number of iterations, you must report this when you publish results. Likewise you should report exactly which JVM version you use, and all commandline options provided to the JVM.

For more information see the DaCapo Benchmark web page.

Building

The easiest way to obtain the benchmark suite is to download the pre-built suite file from the DaCapo Benchmark web site above.

If, however, you want to build from source read on...

Run ant:

ant -p [prints out description, including configuration and environment variable settings]

ant [builds all benchmarks, creates a zip file]

ant dist [builds all benchmarks, this is the default]

ant source [builds a source distribution including benchmarks and tools]

ant bm [builds a specific benchmark, bm]

NOTE

A log of each directory is created under this benchmark directory for benchmark build status and build success or failure files to be stored. The directory log directory is normally of the form ${basedir}/log/${build.time} and contains status.txt where each benchmark build status is recorded, and either pass.txt if all benchmarks build, or fail.txt if one or more benchmarks fail to build. Note: that either fail.txt or pass.txt is created when a full build is performed.

IMPORTANT: before trying to build the suite:

  1. Set your JAVA_HOME environment variable appropriately (it must be set and be consistent with the VM that will be used to build the suite).

  2. Create the local.properties file (using default.properties as a templcate)

  3. Set jdk.11.home, in the local.properties, to point to a Java 11 installation.

For more information, run ant -p in the benchmarks directory.

Customization

It is possible to use callbacks to run code before a benchmark starts, when it stops, and when the run has completed. To do so, extend the class Callback (see the file harness/src/ExampleCallback.java for an example).

To run a benchmark with your callback, run:

java -jar dacapo-23.11-chopin.jar -c <callback> <benchmark>

Source Code Structure

harness (The benchmark harness)

This directory includes all of the source code for the DaCapo harness, which is used to invoke the benchmarks, validate output, etc.

bms (The benchmarks)

  • bms/<bm>/src Source written by the DaCapo group to drive the benchmark <bm>
  • bms/<bm>/downloads MD5 sums for each of the requisite downloads. These are used to cache the downloads (avoiding re-downloading on each build)
  • bms/<bm>/data Directory containing any data used to drive the benchmark
  • bms/<bm>/<bm>.cnf Configuration file for <bm>
  • bms/<bm>/<bm>.patch Patches against the orginal sources (if any)
  • bms/<bm>/stats-*.yml Workload statistics for <bm>
  • bms/<bm>/build.xml Local build file for
  • bms/<bm>/build Directory where building occurs. This is only created at build time.
  • bms/<bm>/dist Directory where the result of the build goes. This is only created at build time.

libs (Common code used by one or more benchmarks.)

Each of these directories more or less mirror the bm directories.

License

The DaCapo Benchmark Suite conmprises several open source or public domain programs, plus a test harness, some patches to enable the benchmarks to run under the test harness, and a packaging process. The benchmarks are distributed under their own licenses and the remaining component is distributed under the Apache License, version 2.0.

Copyright 2023 The DaCapo Project, Schoool of Computer Sciences Australian National University Acton, ACT 2601 Australia

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

dacapobench's People

Contributors

aoli-al avatar caizixian avatar eirbjo avatar farquet avatar john5f35 avatar noricc avatar sewe avatar steveblackburn avatar yangxi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

dacapobench's Issues

fop build error on x86_64 Jaunty

FOP junit test failing on Ubuntu Jaunty.

Attached is a log generated on an Ubuntu Jaunty x86_64 install, using the Sun Java 1.5 VM (64bit), from the FOP unit tests after having built bms/fop
- ant junit (in bms/fop/build/fop-0.95)

Reported by: jzigman

Provide more info on warmups

We should indicate the iteration number for every iteration, whether or not it is a warmup. Could be a count up or a count down.

Reported by: steveblackburn

failure to start server under JRocket 1.6

The geronimo part of daytrader fails to start under JRocket 1.6 with the stack trace below. Note that this failure does not occur on SUN JDKs.

Caught exception invoking server:java.lang.reflect.InvocationTargetExceptionjava.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.dacapo.daytrader.Launcher.startServer(Launcher.java:97)
at org.dacapo.daytrader.ServerThread.run(ServerThread.java:12)
Caused by: java.lang.NoClassDefFoundError: org/apache/geronimo/kernel/rmi/RMIClassLoaderSpiImpl$ClassLoaderServerAware
at org.apache.geronimo.kernel.rmi.RMIClassLoaderSpiImpl.getClassAnnotation(RMIClassLoaderSpiImpl.java:78)
at java.rmi.server.RMIClassLoader.getClassAnnotation(RMIClassLoader.java:364)
at sun.rmi.server.MarshalOutputStream.annotateClass(MarshalOutputStream.java:75)
at java.io.ObjectOutputStream.writeNonProxyDesc(ObjectOutputStream.java:1250)
at java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1203)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1387)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
at sun.rmi.registry.RegistryImpl_Stub.bind(Unknown Source)
at com.sun.jndi.rmi.registry.RegistryContext.bind(RegistryContext.java:120)
at com.sun.jndi.toolkit.url.GenericURLContext.bind(GenericURLContext.java:208)
at javax.naming.InitialContext.bind(InitialContext.java:401)
at javax.management.remote.rmi.RMIConnectorServer.bind(RMIConnectorServer.java:625)
at javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:412)
at org.apache.geronimo.jmxremoting.JMXConnector.doStart(JMXConnector.java:197)
at org.apache.geronimo.gbean.runtime.GBeanInstance.createInstance(GBeanInstance.java:998)
at org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:268)
at org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:103)
at org.apache.geronimo.gbean.runtime.GBeanInstanceState.startRecursive(GBeanInstanceState.java:127)
at org.apache.geronimo.gbean.runtime.GBeanInstance.startRecursive(GBeanInstance.java:556)
at org.apache.geronimo.kernel.basic.BasicKernel.startRecursiveGBean(BasicKernel.java:380)
at org.apache.geronimo.kernel.config.ConfigurationUtil.startConfigurationGBeans(ConfigurationUtil.java:456)
at org.apache.geronimo.kernel.config.KernelConfigurationManager.start(KernelConfigurationManager.java:188)
at org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:563)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:124)
at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:832)
at org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57)
at org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35)
at org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96)
at org.apache.geronimo.gbean.GBeanLifecycle$$EnhancerByCGLIB$$6ef217f9.startConfiguration(<generated>)
at org.apache.geronimo.system.main.EmbeddedDaemon.doStartup(EmbeddedDaemon.java:162)
at org.apache.geronimo.system.main.EmbeddedDaemon.execute(EmbeddedDaemon.java:79)
at org.apache.geronimo.kernel.util.MainConfigurationBootstrapper.main(MainConfigurationBootstrapper.java:45)
at org.apache.geronimo.cli.AbstractCLI.executeMain(AbstractCLI.java:67)
at org.apache.geronimo.cli.daemon.DaemonCLI.main(DaemonCLI.java:31)
... 6 more

Reported by: *anonymous

J9 (and only J9) fails to find the tradebeans.log file

running tradebeans from the dacapo-head.jar (of 2009-09-09) under the

java version "1.6.0"
Java(TM) SE Runtime Environment (build pxi3260sr5-20090529_04(SR5))
IBM J9 VM (build 2.4, J2RE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr5-20090519_35743 (JIT enabled, AOT enabled)
J9VM - 20090519_035743_lHdSMr
JIT - r9_20090518_2017
GC - 20090417_AA)
JCL - 20090529_01

VM fails to find the log file to use. NOTE: This does not happen with JRocket, or Sun JDKs.

java.io.FileNotFoundException: ./scratch/tradebeans.log (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:112)
at java.io.FileInputStream.<init>(FileInputStream.java:72)
at java.util.logging.LogManager.readConfiguration(LogManager.java:459)
at java.util.logging.LogManager$3.run(LogManager.java:221)
at java.security.AccessController.doPrivileged(AccessController.java:202)
at java.util.logging.LogManager.<clinit>(LogManager.java:207)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:200)
at java.util.logging.Logger.getLoggerWithRes(Logger.java:337)
at java.util.logging.Logger.getLogger(Logger.java:367)
at sun.net.www.protocol.http.HttpURLConnection.<clinit>(HttpURLConnection.java:58)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:200)
at sun.net.www.protocol.http.Handler.openConnection(Handler.java:56)
at sun.net.www.protocol.http.Handler.openConnection(Handler.java:51)
at java.net.URL.openConnection(URL.java:945)
at org.apache.geronimo.common.GeronimoEnvironment.init(GeronimoEnvironment.java:41)
at org.apache.geronimo.system.main.CommandLine.<clinit>(CommandLine.java:58)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:200)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:167)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:167)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:41)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:39)
at java.lang.reflect.Constructor.newInstance(Constructor.java:515)
at org.apache.geronimo.gbean.runtime.GBeanInstance.createInstance(GBeanInstance.java:948)
at org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:268)
at org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:102)
at org.apache.geronimo.gbean.runtime.GBeanInstanceState.startRecursive(GBeanInstanceState.java:124)
at org.apache.geronimo.gbean.runtime.GBeanInstance.startRecursive(GBeanInstance.java:555)
at org.apache.geronimo.kernel.basic.BasicKernel.startRecursiveGBean(BasicKernel.java:379)
at org.apache.geronimo.kernel.config.ConfigurationUtil.startConfigurationGBeans(ConfigurationUtil.java:456)
at org.apache.geronimo.kernel.config.ConfigurationUtil.loadBootstrapConfiguration(ConfigurationUtil.java:208)
at org.apache.geronimo.kernel.config.ConfigurationUtil.loadBootstrapConfiguration(ConfigurationUtil.java:167)
at org.apache.geronimo.kernel.util.MainConfigurationBootstrapper.loadBootConfiguration(MainConfigurationBootstrapper.java:84)
at org.apache.geronimo.kernel.util.MainConfigurationBootstrapper.getMain(MainConfigurationBootstrapper.java:57)
at org.apache.geronimo.kernel.util.MainConfigurationBootstrapper.main(MainConfigurationBootstrapper.java:38)
at org.apache.geronimo.cli.AbstractCLI.executeMain(AbstractCLI.java:67)
at org.dacapo.daytrader.DaCapoCLI.main(DaCapoCLI.java:30)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:599)
at org.dacapo.daytrader.Launcher.invokeGeronimoClientCLI(Launcher.java:85)
at org.dacapo.daytrader.Launcher.initialize(Launcher.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:599)
at org.dacapo.harness.Tradebeans.prepare(Tradebeans.java:38)
at org.dacapo.harness.Benchmark.run(Benchmark.java:132)
at org.dacapo.harness.TestHarness.runBenchmark(TestHarness.java:139)
at org.dacapo.harness.TestHarness.main(TestHarness.java:107)
at Harness.main(Harness.java:5)

Reported by: jzigman

Digest of tradebeans output

The output of geronimo for tradebeans etc includes the JVM ID string, this means that the output on each different JVM produces a different MD5 value and causes the testharness to fail. either the test harness must use digests for different JVMs (with a default/fallback digest) or geronimo must be convinced not to report the JVM ID string.

Reported by: jzigman

Remove java14 lib references from default.properties

Remove the following references from the default.properties and ensure the build works for the release.

java14.lib=/usr/lib/j2se/1.4/jre/lib
java14.compile.classpath=${java14.lib}/rt.jar;${java14.lib}/jce.jar;${java14.lib}/jsse.jar

Reported by: jzigman

Xalan build failure under Java 6

The benchmark version of Xalan does not build under Java 6, either update the documentation to reflect this or update xalan to build under Java 6.

Reported by: jzigman

Comment block

A standard comment block should be included at the beginning of each sources file written for DaCapo Bench.

Reported by: jzigman

Allow harness bypass

Ideally we'd have a functional (non-stub) static entry point to each benchmark to facilitate debugging and testing without the harness.

Reported by: steveblackburn

call back classes and class loading

Currently the dacapo.jar does not seem to use external classpath as would be expected. This means that using the -c <mycallbackclass> requires that class (and any classes it uses) to be in the dacapo.jar

Reported by: jzigman

Trade benchmarks hardwire scratch into path

The trade benchmarks currently depend on some paths being included in the classpath defined within the dacapo jar's MANIFEST.MF. These paths include "scratch", which is usually over-riidable by other command line options. If the directory for scratch is over-ridden, then these paths will become invalid and trade will not function.

Specifically, bin/META-INF/MANIFEST.MF includes:
Class-Path: scratch/geronimo-jetty6-minimal-2.1.4/lib/geronimo-kernel-2.1.4.jar scratch/geronimo-jetty6-javaee5-2.1.4/lib/geronimo-kernel-2.1.4.jar

Reported by: steveblackburn

Output Reporting

The output of a benchmark should be directed to stdout/stderr in addition to going to logs/being digested. Currently, the output is digested and silently processed.

Reported by: jzigman

Low CPU utilization on linux

Why? When comparing the same instance of the Sun 1.6 & 1.5 JDKs running on MacOS X and linux, the linux instances showed woeful CPU utilization. So this appears to be an OS-dependent performance issue.

Reported by: steveblackburn

Add sanity pages

Obtain sanity information from the logs for each VM X Benchmark X Size and links to logs for each failure.

Reported by: jzigman

Clean Formatting

All DaCapo Bench source files should have a standard format that does not use tab characters.

Reported by: jzigman

update eclipse

update eclipse to the most recent release

Reported by: jzigman

META-INF is duplicated

The META-INF directory for the dacapo jar appears to be duplicated in the source tree, at bin/META-INF and harness/src/META-INF. We need to delete one.

Reported by: steveblackburn

Deterministic Transaction Set

For any given number of threads each run performs the same set of transactions, however, varying the number of threads varies the total set of transactions, i.e.
i one terminal (thread) is used the set of transactions A is performed and if two terminals (threads) are used then the transactions sets B and C are performed. While one thread will perform A each time, and two threads would performn B and C each time, A is not equal to B plus C

Reported by: jzigman

Print version

We should print out the version number. We could do this with a command line flag. However, given the importance of version numbers, I think we should print it out at start up, every time.

The version number should be exactly one of two things:

a) a version string specified at build time
b) the SCM revision number (svn for now)

Reported by: steveblackburn

Tradebeans and soap fail

D:\downloads>ver

Microsoft Windows XP [Version 5.1.2600]

D:\downloads>java -jar dacapo-9.10-beta1.jar tradebeans
Booting Geronimo Kernel (in Java 1.6.0_17)...
Geronimo startup complete
Successfully created tables
===== DaCapo 9.10-beta1 tradebeans starting =====
Resetting database and populating with 1098 stocks...
Populating database with 384 users...
Finished repopulating database
Running 256 trade sessions directly on server
Completed 256 trade sessions comprising 4098 trader actions
Home .................... 769 (18,8%)
Portfolio ............... 341 ( 8,3%)
Quote ................... 1706 (41,6%)
Buy ..................... 334 ( 8,2%)
Sell .................... 326 ( 8,0%)
Update .................. 62 ( 1,5%)
Register ................ 24 ( 0,6%)
Login ................... 256 ( 6,2%)
Logout .................. 280 ( 6,8%)
Digest validation failed for stdout.log, expecting 0xfa3e4a1b471247726fb6c3a6f42e5853f2522d22 found
0xeeb1369c09846ddf07b20a23607b6b1b799bbb29
===== DaCapo 9.10-beta1 tradebeans FAILED =====
Shutting down Geronimo...
Validation FAILED for tradebeans default

D:\downloads>java -jar dacapo-9.10-beta1.jar tradesoap
Booting Geronimo Kernel (in Java 1.6.0_17)...
Geronimo startup complete
Successfully created tables
===== DaCapo 9.10-beta1 tradesoap starting =====
Resetting database and populating with 1098 stocks...
Populating database with 48 users...
Finished repopulating database
Running 32 trade sessions from client via soap
Completed 32 trade sessions comprising 639 trader actions
Home .................... 128 (20,0%)
Portfolio ............... 40 ( 6,3%)
Quote ................... 285 (44,6%)
Buy ..................... 56 ( 8,8%)
Sell .................... 46 ( 7,2%)
Update .................. 12 ( 1,9%)
Register ................ 4 ( 0,6%)
Login ................... 32 ( 5,0%)
Logout .................. 36 ( 5,6%)
Digest validation failed for stdout.log, expecting 0x7a214488a9a0f009e8266dc03cbc6742fe1e91cf found
0xb16e3bbe86a95d7539549ebe72f8eb95de3ee872
===== DaCapo 9.10-beta1 tradesoap FAILED =====
Shutting down Geronimo...
Validation FAILED for tradesoap default

Reported by: weberjn

eclipse doesn't run on x86-84

eclipse appears to be platform specific, that is it will only run correctly on x86 (32bit). This needs to be verified and corrected when the new version of eclipse is introduced into dacapo.

Reported by: jzigman

Warehouse Size

The TPC-C like benchmark for H2 specifies the number of warehouses (and scale of the same value). The volume of data specified in the TPC-C means that only 2 Warehouses will fit in a 512M heap---we need to reduce this in order to increase the number of warehouses while maintaining a sensible heap size. Ideally we are looking at 8 warehouses in that space.

Reported by: jzigman

Trade build failure

daytrader build is failing due to the jstl jar havinging been moved, see build logs.

Reported by: jzigman

update xalan

update xalan to the most recent release.

Reported by: jzigman

memory leak in tradebeans

Each iteration of the tradebeans benchmark fails to release some resources and eventually it will run out of heap space.

Reported by: jzigman

eclipse fails on a number of JVMs

Need to understand why eclipse fails on so many JVMs. If the problem is theirs, that's fine. Otherwise we need to fix.

Reported by: steveblackburn

vm-specific search results

The jdt search tests are returning different results for different JVMs. We need to fix this at its source because the workload must be vm-oblivious.

Reported by: steveblackburn

Concurrency audit

We need to audit the level of concurrency within every benchmark, and ensure the following:
1. That the mode of concurrency is reasonable (e.g. not grossly artificial or needlessly single-threaded)
2. That where applicable the benchmark respects the different concurrency models available via the harness (setting the number of threads etc)
3. That the level and nature of concurrency is documented in the cnf file

Reported by: steveblackburn

tradebeans failure

tradebeans fails semi-intermittently under jdk1.5 (sun) and others, often the first failure with more than one iteration (-n k for k>1) is a deadlock and subsequent failures are oome, occasionally the first error is oome and occasionally something else.

Reported by: jzigman

Build on Java 6

All benchmarks should be updated so that they can be built under 1.6.

Reported by: jzigman

tomcat fails under windows

tomcat fails to boot the server thread under windows, this may be a platform issue related to path separators.

Reported by: jzigman

new system for facilitating static analyses

After discussions with Eric Boden, it seems clear that we should move to a new mechanism for accommodating those who want to perform static analysis over the framework. Eric has a mechanism which allows us to identify all classes (methods?) that are executed by the benchmark. The proposal is that we generate such a list for every benchmark and include it in the shipped jar. Those wanting to do static analysis can then consult the list to identify which classes/methods are part of the dynamic call graph.

This would allow us to remove the stubs at the bottom of each benchmark's harness which previously served such a purpose.

Reported by: steveblackburn

command line help inconsistent with options

The help offered at the command line is incomplete and possibly inconcisstent.

The short term fix is just to make it correct.

The better fix is to recode the command line parser so that options, their short forms, and their descriptions are kept in a table, and the command line parser and the help system both operate via this table, so they're always in synch.

Reported by: steveblackburn

derby should use tpcc

Derby currently uses pseudo jdbc, as a carry-over from hsqldb, which derby replaced. Robin made an inital attempt to get it working with tpcc. We should push further with this as tpcc is a better base workload.

Reported by: steveblackburn

Be explicit about threading model

The adaptive threading model (N per h/w thread) may appear mysterious. We should probably output an explicit message at the start of each execution of each benchmark stating how many threads are running (and in the case of N per h/w thread, state the reason why (ie that there are N h/w threads detected)).

Reported by: steveblackburn

Tomcat build failure under Java 6

The version of Tomcat does not build using Java 6 (either OpenJVM or Sun) so either the Documentation must reflect this limitation or a version of Tomcat must be obtained that does build using Java 6.

Reported by: jzigman

Report Benchmark Configuration

The configuration of each benchmark should be reported prior to running each benchmark. This should include the thread model used and the number of threads used for all iterations of that benchmark.

Reported by: jzigman

Terse summary of what bm is doing

Users often want to know what the difference is between diffefrent "sizes" of each benchmark. This can be established via the .cnf file, but even then is often obtuse.

We should document the affect of size choice in the cnf file as intelligible prose, and allow this to be printed (via the -i command line option probably)

Reported by: steveblackburn

flush stdout

Some benchmarks (eg eclipse) output "."s to indicate progress. This is a useful property of the benchmarks, particularly when they are long running. However these appear to be coming out only at teh EOL. Need to flush them (perhaps just need to call flush(), but possibly may need to work with the code that does the tee to the digest).

Reported by: steveblackburn

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.