| Swen Conrad
Director of Platform Marketing
for IT Optimization and
Best Practices for Improving Your Application Performance Management – in Spite of Shrinking IT Resources
Think back to the last business application project you were part of: Do you remember how much time and attention was given to the functional analysis and design of the new application? Now, compare this to the attention – or lack thereof – applied to testing, including performance testing, of the new solution.
In the worst cases without rigorous performance testing, the new system goes live with the users immediately complaining about poor system response times or worse, customers abandoning transactions. And, despite immediate efforts from a cross-functional “SWAT” team of UI, network, application, and database engineers, you have a group of experts wasting valuable time reconciling the performance metrics of their respective tools, rather than solving the problem.
Some think these issues are a thing of the past, but just consider the increasing complexity of IT combined with tightened IT budgets. Ca
n IT still ensure that it can bridge the application performance gap between what users expect and what it can deliver, despite reduced resources? This article outlines five best practices to help companies do just that.
1. Leverage IT Best Practices
Whenever you are dealing with a suboptimal IT process that poses a risk to application performance, you’ll need to take the steps needed to fully analyze, optimize, and possibly automate that process (see the sidebar on the next page). While you should encourage your team to make ad-hoc process improvements, any larger IT optimization activities should seek to follow existing IT best practices, like those provided by ITIL, Cobit, or CMMI. Leveraging existing knowledge allows you to compare and benchmark your own IT processes and activities to those of other companies or to those of an outsourcing provider you are considering contracting with, for example.
2. Establish a Release Management Process
The release management process – covering multiple application management phases like design, development, testing, and release – is a good candidate for process improvement. Too often, application projects are managed in various ways across a company, with little standardized processes, deliverables, or coordinated go-live dates, resulting in confusion and frustration. A more structured and methodical release management approach will have a positive effect on overall IT contribution, and improve the business’s perception of IT.
3. Design Your Application for Quality and Per?formance
Most IT professionals agree that an application’s functional performance – and its response time – can best be influenced during the application design, development, and testing phases. Application performance should be part of any initial requirements design document; the number of users and the usage patterns for the new application or process are key inputs for performance testing. Also, it’s important to document possible corner cases – worst-case scenarios that could occur in a business process – and outline how they should be treated from a development point of view and how the tester or test lead should ensure that new software performs to specif?ications.
4. Build a Solid Testing Practice
Customers know how important testing is and that they need to improve testing processes and tools, but the process can be highly complex. You should utilize testing tools – most of which will help automate test planning, test case execution, and test reporting. In addition, your overall test plan should include a portfolio of automated testing and manual user testing. Automated testing is best for well-understood and lower-risk processes – for performance testing and regression testing, for example. Manual testing is best for testing highly critical and risky deployments since automated testing would not provide the variability that an actual user would.
If you’ve set up a structured release management process, you will ideally have set go-live dates for your projects, allowing you to see when testing for different projects might overlap. This parallel project testing carries some risk, but also gives you the perfect opportunity to perform regression testing of parallel solutions, enabling you to control production risk and reduce testing effort and cost.
5. Measure and Monitor Performance
Even if you build application performance into the business solution, issues will still likely occur. But this is okay, because there is a point when problem avoidance becomes more expensive than problem mitigation. In other words, you can do twice as much pre-production testing without f?inding twice as many software defects.
So, if an application performs poorly in production, you will need to analyze the root-cause of the issue across all layers of the technology stack and not just on the application layer. While IT departments have tools to analyze each layer individually, integrated tooling across different layers is not yet common. Besides a single analysis trace through all layers, you’ll also need a single set of information that can be quickly shared across teams for immediate issue resolution.
While there is no silver bullet for solving application performance issues in the foreseeable future, solutions to address key pain points within the application management life cycle do exist. My recommendation is to pick any of the processes that historically cause the most issues in your company and start automating them, measuring results, and driving ongoing improvement. But don’t just react by deploying isolated point solutions for any IT challenge. Instead, make sure that any quick-win IT solution you deploy is also part of a larger, more comprehensive vendor strategy.
In My Experience: Adhering to Best Practices
I’ve had my own experience with confronting subpar testing and with trying to follow those best practices – analyze, optimize, and automate – toward rationalized and improved performance. When I became test manager for a worldwide SAP implementation project, I was asked by senior business and IT teams about testing the batch driving interfaces. While members of my team assured me that this testing was taken care of, no one could detail who was responsible or how it was done.
In the end, I asked one of my test leads for an analysis of existing interfaces. After we understood the scope of a comprehensive testing project, we prioritized each interface based on the risk of business disruption if it were not tested properly. We used this prioritization to define a clear test plan, with an understanding between business and IT that we would not test every interface process. Based on the new clarity around our testing effort, the senior business and IT managers were satisfied, and we never experienced any performance problems.
While we completed analysis and optimization, we never reached automation. Managing batch-driven interfaces in a single, cockpit-type environment would have solved the remaining issue of coordinating the batch interfaces all over the company.
To learn more, visit service.sap.com/alm; www.sap.com/solutions/executiveview/include/run-and-optimize-it/index.epx; or http://itsmfusa.brighttalk.com/node/601 for an online conference on integrated IT service management.
back to top
and Partner Group
Manage and Monitor Your Business Applications’ Performance
Key Solutions to Help You Get Started
Today’s complex IT environments, along with a dramatic rise in network traff ic, have made it increasingly diff?icult for organizations to manage their software applications. Even in the face of this challenge, however, IT must work to enable solid application performance. This means that IT needs to:
- Ensure a comprehensive day-to-day performance management strategy that covers SAP and nonSAP applications, including application process execution, business process, systems, and user application performance.
- Test application performance in preproduction stages to identify problems before they affect the organization. IT must also design for application performance at the beginning of new implementations, migrations, and updates to mitigate performance risks and bottlenecks at go-live.
- Improve transaction cycle time to enable timely visibility into key business information and allow business users to operate at peak efficiency.
To help meet these requirements, SAP has identif?ied select third-party offerings, called solution extensions, that enable IT to perform proactive and reactive application monitoring and measuring, as well as problem resolution to ensure enhanced application performance for your business processes.
Solution extensions can also help you address key application performance pain points, resulting in ongoing cost savings and a positive ROI. And because SAP tests these solutions to ensure that they meet certain standards and that they integrate with SAP’s technology roadmap, they are ideal for incorporating into SAP landscapes.
There are many different solution extensions, but SAP has pinpointed six solutions in particular that improve automation, management, and control across an enterprise’s systems, applications, databases, and processes, enabling SAP customers to better manage and monitor the performance of their business applications.
1. Proactively Resolve Application Performance Issues with SAP Extended Diagnostics by CA Wily
Companies can lose a lot in the long run if they don’t properly test application performance in preproduction stages, identify problems before they affect users, or properly evaluate application service-level agreements (SLAs). That’s why it’s important for IT to manage its infrastructure for real-time application performance monitoring; and the SAP Extended Diagnostics application by CA Wily does just that. The solution enables companies to constantly monitor complex IT environments and detect and resolve problems quickly. Such real-time intelligence can help you ensure the success of Web-based applications and services that are mission-critical to meeting your business and performance goals (see f?igure).
Since strong expertise across the full range of Web application dependencies is next to impossible to find, SAP has leveraged CA Wily’s extended diagnostic technology in the SAP Solution Manager application management solution. SAP Extended Diagnostics is a performance-monitoring solution for SAP customers, which supports both SAP and non-SAP software environments. With this, SAP customers can use the same tools for root cause analysis as the SAP Active Global Support organization.
SAP Extended Diagnostics integrates seamlessly with existing solutions, making performance management in heterogeneous environments more effective. Since it is platform independent, it works on any Web application environment. SAP Extended Diagnostics provides real-time performance metrics for Java and Microsoft .NET applications, from development to production, to help your IT staff quickly conduct root cause analysis of performance problems. In addition, the application can be clustered for scalability in large and complex environments, supports automatic discovery of systems and components without additional coding, and provides preconfigured, intuitive real-time views to all transactions.
SAP Extended Diagnostics also allows lines of business to better manage their SLAs, plan for future technology needs, analyze performance trends, and comply with corporate governance initiatives. It also enables companies to create user-friendly custom dashboards, reports, and graphic displays that provide detailed, intuitive views of the entire application infrastructure. Even users who are less technically skilled can easily pinpoint the root cause of subpar performance through auto-generated views of the application environment. And every stakeholder can use this performance data collectively for application health and availability monitoring, incident triage, compliance with quality initiatives, trend analysis, and capacity planning.
2. Maximize System Performance with SAP LoadRunner by HP
To ensure productive application behavior and performance, IT teams must optimize their end-to-end business processes – which run on both SAP and non-SAP software – over the entire solution life cycle. SAP LoadRunner application by HP has performance and load testing capabilities that can emulate hundreds, even thousands, of concurrent users while collecting information from key infrastructure components (including Web servers and database servers).
Consider a Web site that, during peak times, hosts a much higher number of concurrent users than the typical load – such as at the start of the workweek. During the testing phase, it’s simply not practical to have a testbed of thousands and thousands of users, nor can you afford to wait for a real-time spike of activity to diagnose any bottlenecks. Instead, SAP LoadRunner can simulate simultaneous server access by a large number of users. With the help of thorough load testing, you can better deliver high-performing applications. It also supports companies managing performance during application functionality tests, performance tests, and capacity planning and sizing.
Automated to test the full spectrum of applications supporting business scenarios, SAP LoadRunner predicts solution behavior and performance under any of these conditions. Using SAP LoadRunner, you can better deliver high-performing business processes – and execute, upgrade, and modify existing processes – on time and within budget.
3. Improve Process Performance with SAP Central Process Scheduling by Redwood
Real-time business processes demand increasingly complex technical requirements to enable optimized application performance. With an automated process scheduling solution like the SAP Central Process Scheduling application by Redwood, companies can:
- Increase the efficiency of end-to-end, business-process workload execution
- Integrate SAP and non-SAP business application processes
- Reduce application workload completion time
- Improve adherence to SLAs
- Optimize the use of existing IT resources
With SAP Central Process Scheduling by Redwood, you can create a dynamic scheduling environment that provides a core infrastructure component, which enables an approach to service-oriented architecture that’s compatible with enterprise-wide automation.1 This guarantees transparent and proper application performance at all times (see figure).
4. Optimize Process Performance with SAP Process Performance Management by IDS Scheer
The SAP Process Performance Management application by IDS Scheer analyzes processes running in SAP solutions and automatically generates graphic documentation. At the same time, the application uses previously def?ined process performance indicators to help you evaluate your current process performance.
With intuitive cockpit displays of the current process performance indicators, SAP Process Performance Management serves as an integrated, early warning system that automatically monitors all ongoing processes and warns if these deviate from the plan (see f?igure). This supports decisive efforts toward continuous improvement to sustain eff?icient and innovative SAP solution-based business processes.
SAP Process Performance Management uses def?ined key performance indicators to constantly track the performance of real-world business processes. This allows users to monitor core applications to guarantee transparent and proper performance at all times.
Typically, companies deploy several different IT systems to support a core process from end to end. By combining the process-relevant data from multiple IT systems, SAP Process Performance Management can reconstruct the execution of each process from start to f?inish. This makes it possible to display individual transactions within that process in a graphical format, and also calculates throughput times, deadline compliance, supply capacity, response times, process costs, and change frequency.
Because SAP Process Performance Management provides multiple perspectives simultaneously, users gain a multifaceted view of end-to-end processes. The application also allows you to aggregate multiple process instances to better understand how something like a sales order is handled within your company. SAP Process Performance Management lays out the variable paths of process execution associated with a single process chain, enabling you to spot inherent weaknesses or deviations from expected behavior.
5. Test Your Applications Before Go-Live with SAP Quality Center by HP
During any technical or process change, such as an application implementation or upgrade, an organization’s core business performance shouldn’t suffer. The highly secure and mission-critical nature of enterprise application data means that you need to regularly test these applications and related business processes.
If you’re looking for a fast, reliable way to test new applications before go-live, consider the SAP Quality Center application by HP. Its comprehensive, automated, and accurate procedures reduce testing costs via a Web-based application that provides activities essential to quality management, including gathering and managing requirements, scheduling and running tests, managing defects, and running consolidated reports.
SAP Quality Center offers these functions either as a standalone solution or integrated within a global quality center of excellence (COE) environment. And when deployed in conjunction with SAP Solution Manager, SAP Quality Center can be scaled up to meet the heavy demands of an enterprise COE.
SAP Quality Center lowers the risk inherent in all software implementations and upgrades, and delivers high performance throughout the application life cycle. With SAP Quality Center, you can implement, upgrade, and prepare for go-live with confidence.
6. Optimize User Performance with SAP User Experience Management by Knoa
When you deploy SAP software, you want to realize benefits as quickly as possible, while minimizing the business disruption, costs, and risk. But companies often fail to realize expected ROI because of end-user issues that hinder the efficient execution of processes. Yet, how can you tell how your SAP applications perform for individual users? How can you identify which people have diff?iculty executing which transactions? What does it mean when people say the system is “slow?” How can you use data to separate real issues from opinions and take corrective action? In short, how can you discover – beyond hearsay and anecdotal reporting – the user experience?
The SAP User Experience Management application by Knoa can help ensure you realize expected ROI. It identifies how each person in your organization uses your SAP solution. The offering provides detailed reports that you can use to improve application response times, pinpoint training needs, adjust business processes, and ensure that employees are delivering the value you expect from your SAP solution.
As an added advantage, SAP User Experience Management measures and reports critical information about how people experience the actual performance of your SAP solution – right at their own desktops. SAP User Experience Management also:
- Provides insight to executives about application usage, adoption, and compliance
- Maximizes training investment by linking it to actual user performance
- Reduces application and end-user errors
- Lowers end-user support costs and reduces calls to the help desk
- Measures actual response time of your SAP software
With SAP User Experience Management, you gain comprehensive insight into application usage and performance to make informed decisions about where to deploy resources to improve solution performance and effectiveness.
In the past, when businesses ran applications on client servers, it was much easier to identify and diagnose performance problems, which were usually easily linked to the network or to a given technology silo. Today, however, IT must stay on top of a complicated array of interconnected technologies, business processes, and workflows that can make it difficult to identify the source of a performance problem. To deal with this challenge and ensure that applications perform at their best, companies need access to a consistent, comprehensive view of their business processes.
That’s why IT needs solutions that provide automatic alerts that immediately let them know about software problems, allowing operations or application support personnel to rapidly triage an incident, gather essential data about the nature of an issue, and assign the right person to fix the underlying problem. They also need root cause diagnostics to further facilitate rapid problem resolution and minimize the drain on your IT resources. With solution extensions from SAP, companies can access this needed functionality and be assured that it will integrate with their existing SAP landscape.
|SAP Extended Diagnostics by CA Wily
|SAP Central Process Scheduling by Redwood
|SAP Process Performance Management by IDS Scheer
1 For more information on SAP Central Process Scheduling by Redwood, see “Expect More from Your Job Scheduling Solution,” an SAP NetWeaver Unleashed column by Dr. Astrid Tschense-Oesterle in the July-September 2008 issue of SAP Insider (sapinsider.wispubs.com). [back]
back to top
| Andreas Mueller
Solution Marketing Director
HP Software and Solutions
5 Golden Rules of an Effective, Proactive Performance Management Strategy
Best Practices for Improving Application Performance
Agility, speed, and effectiveness are the key ingredients to succeed in the new economic reality. Today, almost all mission-critical business processes rely on business applications, like those from SAP. Given the increasing amount of business process automation, application performance is having a growing impact on a company’s overall top-line performance.
But many organizations still take a more reactive approach to performance management and try to fix problems after they occur. They conduct resource-intensive, manual stress tests or super-size their IT infrastructure under the belief that doing so will avoid performance problems.
These approaches are very costly and risky. Consider the impact of missing fiscal reporting deadlines or having a few hundred people with significant productivity loss due to slow system response time. And while it may be tempting to simply increase your hardware power, this approach does not necessarily mean an increase in application performance. The key factor for successful application performance management is about proactively identifying and fixing performance problems before users are affected.
Design an Effective, Proactive Performance Management Strategy
Application performance management is much more than running a couple of stress tests and monitoring your system infrastructure. It requires a building-block approach that involves several phases and a tight alignment between IT or a service provider and the line of business or a client. Based on our many years of experience working with customers to optimize the performance of their SAP systems and to improve the overall return on their investment, we recommend that you consider the five golden rules of performance management described here.
Rule 1: Gather Proper Requirements Upfront
Gather proper requirements prior to your testing project so that everyone has a clear understanding of the objectives. It’s also important to set realistic business requirements and to tie the project’s costs to incremental performance improvement. You’ll also need to set your service-level objectives (SLOs) for the performance tests based on these requirements.
Rule 2: Plan for Multiple Types of Performance Test Cycles
There are many types of load tests, each providing a different angle on application performance and additional information to measure the business risk associated with releasing a given application into production. Smoke, baseline, incremental load, and duration testing are common types of performance testing.
For each type of test, you should set an individual timeline, resource requirements, entrance and exit criteria, and reporting requirements. Rather than thinking of test execution as a one-time run, it is better to think of it as a continuous process that will involve many types of tests, iterations, and re-runs to get a more complete picture of projected application performance.
Rule 3: Allocate Time to Properly Diagnose and Fix Performance Issues
Work with your database administrators, system administrators, project managers, and business analysts to help diagnose and fix application and infrastructure issues that are impairing system performance. You may need to run tests multiple times to get to the bottom of the issue, and then run them again to validate the fix. Tools for analyzing and diagnosing problems should be integrated into the performance testing suite and allowed to directly correlate performance issues to their root causes.
Rule 4: Improve Application Performance Under Realistic Workload Conditions
Proactively monitor your production system by realistically emulating the actual user experience. Try to create an environment that closely mimics the production system – ensure that the right business processes and transactions have been emulated in your performance tests. This way, when the application goes live, there will be no surprises from unexpected performance problems involving infrastructure components that weren’t included in the test environment. Once the system is live in production, monitor the system performance to ensure the application continues to perform as expected. You should also capture real user data and use it to conduct regression testing with improved usage data.
Rule 5: Create Open Communication and Foster Teamwork with Cross-Functional Groups
Most SAP implementations span multiple sites in different countries and time zones. For performance tests, you should set up a central point of control for all involved stakeholders and project team members. The importance of collaboration and communication – not only within the performance testing team, but also among all the business and technical stakeholders – cannot be emphasized enough.
Share project goals and consider input from all groups early on, and regularly communicate progress and results to all team members including test leads, development managers, project managers, and quality assurance (QA) managers. Implement consistent checkpoints, key performance indicators, and scorecards throughout the performance testing project to support regular progress reports. Through consistent reporting and input from various stakeholders, it is easier to keep the project on track and make sure that testing goals align with business requirements. Ideally, test scripts can be re-used in production environments and for functional testing purposes, further fostering collaboration among cross-functional teams.
HP and SAP Provide Tools to Optimize Your Performance Testing
HP has been providing the performance testing “engine” under the SAP hood for many customers since the early SAP R/3 releases. HP, which runs one of the largest SAP installations in its own company, also works closely with SAP to provide customers with the tools they need to optimize their performance testing.
HP LoadRunner and HP Performance Center help customers prevent costly performance problems in production by detecting bottlenecks before they deploy a new system or move the system into production. These solutions also help optimize investments in hardware and infrastructure. Thousands of SAP customers are utilizing our technology to ensure on-time SAP implementations, achieve high levels of end-user satisfaction, and minimize the risk of running highly complex environments.
With HP LoadRunner and HP Performance Center, your enterprise can measure its end-to-end performance and diagnose application and system bottlenecks – all from a single point of control. What’s more, they support a wide range of enterprise environments beyond SAP technology.
Visit www.hp.com/go/QMforSAPsoftware for more information about HP’s solutions for SAP applications.
Summary: 5 Golden Rules of Performance Testing Management
- Gather proper requirements upfront.
- Plan for multiple types of performance test cycles (including a smoke test, stress test, bandwidth test, soak test, and failover test).
- Budget for time to fix the system after uncovering performance issues and to re-test to ensure that the issues have been fixed properly.
- Emulate real user environments and workloads to predict system behavior and proactively improve application performance.
- Create an open communication and sense of teamwork with all parties involved in the testing process, and implement consistent reporting, checkpoints, key performance indicators, and scorecards throughout.
back to top
| Shoeb Javed
Vice President of Engineering
Overcome the Challenges of Performance Testing with Worksoft Certify Performance Test
If your job includes managing the performance and availability of SAP applications, ask yourself:
- When you have a problem with your application’s performance, can you quickly pinpoint the source?
- Does your application perform and scale to meet normal and peak production usage?
- Is your infrastructure optimized to provide adequate capacity to meet your service levels?
If you answered “no” to any of these questions, it’s time to re-evaluate how you test your SAP applications.
Why Is Performance Testing So Critical?
Your business applications are in a constant state of change – from launching new products and services, complying with regulations and industry standards, or acquiring or merging with another business. In addition, SAP solutions evolve with new service packs, enhancement packages, upgrades, and new modules.
And each change to your system requires performance testing to ensure that your business processes continue uninterrupted and that system performance remains unaffected. Without this testing, your system could encounter, for example, a delay in user response time that wastes time and creates user frustration, or a reduced transaction throughput that impairs overall operating efficiency, even to the point of compromising system availability and stability.
Traditional Solutions Aren’t Enough
Performance testing is not new. But with traditional solutions, this testing requires a significant amount of time and resources – often requiring cryptic scripting language skills, intricate manipulations for test data, and a disjointed set of graphs and reports – to understand the root cause of the performance issue. These solutions are also disconnected from existing functional validation test processes and tools, adding another layer of required tools, time, and effort.
Worksoft’s New Approach to Performance Testing
Worksoft Certify Performance Test overcomes the traditional challenges of performance testing SAP applications. The solution is built within the SAP landscape so it is familiar to users, allowing a performance test that simulates hundreds or thousands of users without requiring users to write a single line of code.
The main objective of performance testing is to ensure that business applications run optimally and can scale to support all concurrent end users and their daily business functions. Certify Performance Test is an efficient performance testing solution for SAP as it requires no scripting or coding, includes an intuitive user interface within the SAP landscape, integrates with Worksoft Certify for a complete functional and performance testing solution, and requires 75% less time and resources than traditional solutions. To learn more, visit www.worksoft.com.
|Certify Performance Test stores test definitions, organized by business process, within the SAP landscape
back to top