A method for evaluating the impact of software configuration parameters on e-commerce sites

E-commerce systems are composed of many components with several configurable software parameters which, if properly configured, can optimize system performance. The cost and burden of implementing upgrades can be deferred by judicial tuning of these parameters. Tuning is a time consuming effort and energy has to be spent on the most relevant parameters. This paper provides a method to evaluate the statistical significance of configurable parameters in e-commerce systems, the interaction effects between each parameter and e-commerce function types, and a method to rank key configurable e-commerce system parameters that significantly impact overall system performance, and the performance of the most significant Web function types. Both on-line and off-line parameters at each of the e-commerce system layers---Web server, application server, and database server---are considered. The paper discusses the design of a practical and ad-hoc approach that involves conducting experiments on an e-commerce site compliant with the TPC-W benchmark. The performance metrics of interest include server's response time, system throughput, and probability of rejecting a customer's request. The paper also provides a set of useful guidelines for tuning the performance of e-commerce sites.