Upon the request of some potential and existing customers, Software Tree has developed a benchmark framework that tests the performance of object-relational mapping (OR-Mapping) engines. The benchmark is called STORM™, which stands for Software Tree’s Object Relational Mapping benchmark. Although there are many different performance aspects of an OR-Mapping engine that can be measured in different ways, we set the following major goals for the STORM benchmark:
This benchmark does not prescribe any particular hardware or software setup. You can use any combination of hardware, JVM, RDBMS and JDBC driver. The idea is to easily and quickly run the benchmark programs and get meaningful results. We believe that by running the benchmark programs in your particular environment and settings, you can get both absolute and relative performance numbers, which will make sense for your particular situation.
We hope that our efforts on STORM benchmark will help the community in evaluating the two most basic and important performance characteristics (inserts and queries) of different OR-Mapping technologies and help us improve our offerings.
The STORM benchmark uses the instances of an “Account” class, which has ten attributes of different types and maps into a database table with a record size of 100 bytes. The benchmark program does the operations in the following order:
This high-level overview and the actual reference implementation of the benchmark for JDX constitute the current specification of the STORM benchmark.
Software Tree has developed the STORM benchmark program for its JDX OR-Mapping engine. Software Tree has also ported the benchmark for Hibernate. These implementations are available with the JDX evaluation kit in their own subdirectories under the examples\STORMBenchmark directory. Each subdirectory is self-contained and comes with extensive documentation including ANT scripts to configure and run the benchmark tests. Implementations for other OR-Mapping engines may be added later on.
Your performance numbers would depend upon your setup (including CPU, Memory, JDK version, RDBMS, JDBC driver, etc.) but the results from running the STORM benchmark programs using different OR-Mapping engines under similar setup should give a reasonable indication of the relative efficiency and scalability of those engines.
STORM benchmark package is shipped with the JDX evaluation kit, which can be downloaded from Software Tree’s web site. The package is available under the examples\STORMBenchmark directory,
Your suggestions and contributions for improvements to STORM are welcome. Please send them to email@example.com. We will try to incorporate them in STORM as and when possible.
Software Tree, LLC
2953 Bunker Hill Lane, Suite 400
Santa Clara, CA 95054 (USA)