Benchmark Factory® for Databases
Version 5.7.1
Release Notes
October 2008
Quest Software is pleased to make available Benchmark Factory® for Databases 5.7.1 to meet your database workload testing needs. Benchmark Factory is a database performance and code scalability testing tool that simulates users executing transactions on the database and replays production workloads in non-production environments. This enables developers, DBAs, and QA teams to validate database scalability as their user load increases, application changes are made, and platform changes are implemented. Benchmark Factory is available for Oracle, SQL Server, DB2, Sybase, MySQL, and other databases via ODBC connectivity. Benchmark Factory:
Simulates thousands of concurrent users with a minimal amount of hardware.
Simulates database production workloads.
Reports tests results with built-in reporting features.
Upon completion of a test execution, all test results are collected and stored in the repository for data analysis and reporting. Benchmark Factory collects a variety of informative and detailed statistics that include:
Overall server throughput (measured in transactions per second/minute or bytes transferred).
Detailed transaction executed statistics by individual agent workstations producing a load.
Quest Software has been committed to constant improvement of Benchmark Factory® for Databases for over ten years and will maintain this commitment for the foreseeable future. Benchmark Factory® for Databases will continue to be an industry leading load testing and scalability tool for the DBA and database developers. We will maintain our heterogeneous support for database platforms, including native support for Oracle and MS SQL Server, and support for DB2, MySQL, and Sybase via ODBC. We are committed to continued integration with our flagship Toad for Oracle product, hugely popular amongst Oracle developers and DBAs alike over the last ten years.
Because our focus is on the DBA and Developer user community and not necessarily on the large scale performance testing market, Quest software will be providing a more focused approach to supporting this customer base by limiting the amount of concurrent virtual users that will be supported on a single test. Moving forward, we will support only those instances where users are testing concurrent user loads no greater than 2,500 virtual users in a single test. This support limitation will also apply to the use of Benchmark Factory Agents. Quest Software will support no more than five agents in a single test scenario. While the product currently allows user loads and agent distribution greater than the afore mentioned amounts, support will only be provided in those scenarios where the user is working within the designated parameters.
This change is effective on Sep 17, 2008 for all supported versions of Benchmark Factory for Databases.
In future releases of Benchmark Factory® for Databases, we will be building these limits into the tool but until then, this will serve as notification that the support offering for Benchmark Factory has changed with the strategy of the product.
Benchmark Factory Run Reports now allows you to copy general run and userload results to the Windows Clipboard and paste those results into the desired program.
The TPC-D Benchmark Scale can now be a value from 1 to 200,000,000 by 1. The scale factor of 0.1 is no longer valid.
Benchmark Factory is now integrated with Quest SQL Optimizer for Oracle, allowing you to test SQL statements for scalability under a specific user load to determine the best performing SQL alternatives.
Benchmark Factory now supports the Oracle 11g database.
The Oracle Trace File import process has been enhanced to allow for datetime bind parameters, NULL bind values, and better dependency filtering, as well as support Oracle 11g trace file import.
The SQL Server 7 trace file and SQL Server 2000/2005 workloads trace tables have been combined into single workload.
Note: Benchmark Factory no longer supports trace file import. To import a trace table, you must first save it as a trace table using SQL Optimizer, then import the table.
Benchmark Factory Run Reports now provides improved disclosure reporting of computed maximum qualified throughput at maximum user load for TPC-C tests and the power calculation and Timing Intervals for TPC-H tests.
An Agent Management tab has been added in Settings, allowing users to automatically start and shutdown agents.
The submit job process has been simplified to quickly run the desired test.
Benchmark Factory has some new advanced mapping options added for SQL Server and Sybase databases, allowing customization of the creation of benchmark objects.
A new setting has been added to limit the number of rows to be retrieved when executing custom SQL transactions.
Benchmark Factory can now reside on an Oracle database.
The following is a list of issues addressed in this release of Benchmark Factory for Databases.
|
Resolved Issue |
Change Request |
| Large SQL statements are cutoff for SQL Server trace imports. | 44,475 |
| When running a test with a large number of transactions and multiple agents, the agents do not start at the same time. | 50,789 |
| Can't import SQL Server trace tables with large number of rows. | 54,533 |
The following is a list of issues known to exist at the time of Benchmark Factory for Databases release.
|
Feature |
Known Issue |
Change Request |
| Benchmark Factory Console | Benchmark Factory does not support data types that are created by the user. | N/A |
If you have installed the Benchmark Factory for Databases Beta, uninstall first, before installing the latest version.
Older versions of Benchmark Factory may not work using a repository created with newer version of Benchmark Factory.
Client libraries for database types used during the load testing process must be installed on all testing machines (Benchmark Factory console and Agents).
The Benchmark Factory 3.3 files and below can be converted to newer versions of Benchmark Factory files. A project conversion program is available, and can be accessed by logging on to Quest Software, then select Downloads and Upgrades|Benchmark Factory for Databases|Special Patches| Benchmark Factory Project Converter.zip.
There is no upgrade path for the Benchmark Factory Repository version 3.3 or below.
Before installing Benchmark Factory for Databases, ensure your system meets the following minimum hardware and software requirements:
| Repository | Benchmark Factory cannot run without a repository, and by default
a MySQL repository is installed during the install process. After
installation, you can remove the MySQL service and delete the MySQL
repository, then select a repository of your choice. Benchmark Factory
supports the following databases:
|
| Platform | 1.0 GHz x86 compatible CPU |
| Memory | 512 MB of RAM recommended
Note: The memory required may vary based on the following:
|
| Hard Disk Space | 120 MB of disk space for installation
Note: The disk space required may vary based on options selected for installation. |
| Operating System | Microsoft® Windows 2000, 2003, 2008, XP Professional, and Vista |
| Database Client/Sever Versions | Oracle: 8, 8i, 9i, 9.2, 10g, and 11g (Note:
Requires Oracle 8.1.7 client libraries or higher.) SQL Server: 7, 2000, 2005, and 2008 Sybase: 12.5 and 15 DB2 LUW: 8.1.5 through 9 MySQL: 4.5 and 5.x |
This section contains information about installing and operating this product in non-English configurations, such as those needed by customers outside of North America. This section does not replace the materials about supported platforms and configurations found elsewhere in the product documentation.
The platforms and configurations supported in this release may use any time zone, character encoding, language, or locale configuration to operate this product. All components of this product must/should be configured to use the same or compatible character encodings and must/should be installed to use the same locale and/or regional options.
This release has been tested with the following language and platform combinations in addition to U.S. English:
The Benchmark Factory for Databases release package contains the following products:
| info@quest.com | |
| Quest Software, Inc. World Headquarters 5 Polaris Way Aliso Viejo, CA 92656 USA | |
| Web |
Refer to our Web site for regional and international office information.
Quest Support is available to customers who have a trial version of a Quest
product or who have purchased a commercial version and have a valid maintenance
contract.
Quest Support provides around the clock coverage with SupportLink,
our web self-service. Visit SupportLink at http://support.quest.com/.
From SupportLink, you can do the following:
View the Global Support Guide for a detailed explanation of support programs,
online services, contact information, and policy and procedures.
The guide
is available at: http://support.quest.com/pdfs/Global%20Support%20Guide.pdf.
This document contains proprietary information protected by copyright. The software described in this guide is furnished under a software license or nondisclosure agreement. This software may be used or copied only in accordance with the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose other than the purchaser's personal use without the written permission of Quest Software, Inc.
© 2008 Quest Software, Inc. ALL RIGHTS RESERVED.
Quest, Quest Software, the Quest Software logo, Aelita, AppAssure, Benchmark Factory, Big Brother, DataFactory, DeployDirector, ERDisk, Fastlane, Final, Foglight, Funnel Web, I/Watch, Imceda, InLook, IntelliProfile, Internet Weather Report, InTrust, IT Dad, JClass, Jint, JProbe, Knowledge Xpert, LeccoTech, LiteSpeed, LiveReorg, Matrix Insight, Matrix.Net, MIQ, NBSpool, NetBase, Npulse, PerformaSure, PL/Vision, Quest Central, RAPS, SharePlex, Sitraka, SmartAlarm, Speed Change Manager, Speed Coefficient, Spotlight, SQL Firewall, SQL Impact, SQL LiteSpeed, SQL Navigator, SQLab, SQLGuardian, SQLProtector, SQL Watch, Stat, Stat!, StealthCollect, Tag and Follow, Toad, T.O.A.D., Vintela, Virtual DBA, Xaffire, and XRT are trademarks and registered trademarks of Quest Software, Inc in the United States of America and other countries. Other trademarks and registered trademarks used in this guide are property of their respective owners.
If you have any questions regarding your potential use of this material, contact:
| Quest Software World Headquarters LEGAL Dept 5 Polaris Way Aliso Viejo, CA 92656 Email: legal@quest.com |
The information in this document is provided in connection with Quest products. No license, express or implied, by estoppel or otherwise, to any intellectual property right is granted by this document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect to the accuracy or completeness of the contents of this document and reserves the right to make changes to specifications and product descriptions at any time without notice. Quest does not make any commitment to update the information contained in this document.