Q&A with Shoeb Javed
October 29, 2013
Sponsored by Managing Your SAP Projects
How many devices will your next mobile app run on? With every device and platform, the testing time, resources and risk grow exponentially.
Shoeb Javed, speaker at Managing Your SAP Projects, joined us for a Q&A to share his tips, best practices, and tools to close out the Testing phase of a mobility project faster and more efficiently. Topics covered included testing with highly iterative projects, testing for iOS vs. Android, test-driven development practices, crowdsourcing testing, load and performance testing, Testing Centers of Excellence, testing with cloud and HANA, key metrics and KPIs when testing, security and authorizations, and more.
Here is the edited transcript of the chat, which was moderated by Amy Thistle, conference producer for Managing Your SAP Projects 2013.
Amy Thistle: Thanks for joining in today’s chat with Shoeb Javed. Shoeb will be taking questions today about the topic of his upcoming session: “A Comprehensive Guide to Managing the Testing Phase of Your Mobility Project.” Shoeb is CTO of Worksoft, where he’s responsible for technology strategy, software development, and customer support.
We’ll be chatting with Shoeb about his experiences working with SAP customers on testing for mobile vs. desktop, and all the testing challenges that come with a shift to mobile projects.
Thanks, Shoeb, for taking the time to take some questions today!
Shoeb Javed: Thank you Amy! It's great to be here! I am looking forward to the Managing Your SAP Projects conference in Orlando and meeting everyone there!
Amy: There are a number of questions waiting for you, but to kick off the Q&A, I want to ask about TechEd last week. Can you tell us what the big takeaways were from that event?
Shoeb Javed: TechEd this year was great. SAP has transformed significantly over the last three years. All of the new technologies that they introduced like SAP HANA, mobile applications, hybrid cloud technologies, etc., have become more mainstream now.
It is interesting to see this transformation and see people's concerns shift from just curiosity to how do we practically implement all these new technologies in our SAP landscape
Comment from Malini Rao: What kind of areas in Security, Authorizations needs to be taken care when it comes to Mobility project?
Shoeb Javed: There are two kinds of security/authorization concerns: Infrastructure security and application security.
The first one is typically handled by your overall mobile/network infrastructure, including mobile device management solutions like Afaria and infrastructure platforms like Sybase Unwired Platform (SUP) as well as SAP NetWeaver Gateway.
For application security I would strongly recommend testing-role based security just like you do for your core SAP applications. It is important to ensure that mobile applications respect role-based authorizations, also.
Comment from Ken: Hello Shoeb, regarding testing different form factors: Is there any different approach to testing if we're using responsive design? Thanks!
Shoeb Javed: This is an interesting one. Assuming that you are using HTML5 and some kind of responsive framework like Zurb foundation for example. In this case you can do a great deal of testing just from the browser by simply resizing it to the device form factors that you support.
Testing on real devices then is limited to actual device hardware functions like gestures, swipes, scrolling, etc. You may also have to verify that the pages load correctly in different mobile browsers on the device itself.
Automating some of these tests would help also to cut down on the time it takes. There are solutions available that let you access the DOM from the browser and then automate most user/browser actions on your application.
Comment from Anne: We'd like to try crowd-sourced testing, but how does this process work? And how do companies normally get feedback and track and measure testing results?
Shoeb Javed: There are some very sophisticated platforms available for crowdsourcing. I don't typically recommend any particular solution in a session like this but there are a number of them that are available.
They allow you to post your request and then have people download and test your application 'in the wild,' so to speak. The testers log results in an online application; they can put in screenshots, etc., to show what they did and the application responses. You can get pretty comprehensive feedback in a structured way.
Comment from Guest: So you treat testing differently if you're rolling out for iOS only? We have a BYOD policy - but it's really a BYO(iOS)D policy. :) Any tips?
Shoeb Javed: No, I don't think so. You still have to consider native applications versus HTML5-based applications, still need to address functional, performance, load and security testing.
Your task is a little easier though since you can skip a number of other devices in your actual device based testing and focus only on iPhones and iPads.
Making sure end-to-end business processes work in the context of the tasks that users perform is still important.
Comment from Malini Rao: If sales representatives are using mobile solutions on iPhone & Android and accessing CRM & BW reports, what kind of integration testing is necessary for mobility projects?
Shoeb Javed: Interesting question. Testing end to end is important if the mobile applications perform operations that are not simply reading data from these systems.
For example, if you want to create sales orders in your CRM system from the mobile device, you need to make sure that the order that was generated in the CRM system is correct and is processed further in ERP, etc., and gets fulfilled, invoiced, has right tax calculations, and so on. It depends on the business process being implemented.
For BW/Analytics reporting, the need for doing back-end testing for mobile-specific projects is less so since you are simply pulling up reports and if those reports are accurate then you just make sure that the data is formatted correctly for the device and is accessible on the device without worrying too much about the back-end systems.
Comment from Guest: We are planning to see versions of the app every 4-6 weeks - so it seems like we'll be testing all the time! How will these iterations change our approach final release testing?
Shoeb Javed: This is again a very interesting question, because you are not alone. All mobile applications projects are short and highly iterative. I would suggest that you set up a good automation framework so that you can run regression tests quickly and frequently. There is no way to handle this through manual testing. You want to automate tests as early in the cycle as possible and then modify them as your application changes. For each new iteration, you can build the automation for that iteration in the current cycle but run automation to make sure everything else from all other cycles is not broken
Comment from Guest: Hi Shoeb. Performance issues for our mobile apps are one of our big anxiety points. What are the key metrics you look for in load/performance testing?
Shoeb Javed: This is once again a common concern. We typically look for three or four main things. What is the end user response time under typical load conditions? How is this response time affected by geography? How is this response time affected by the carrier networks being used by the user? How is my back-end infrastructure keeping up? (app servers, load balancers, database systems, application gateways, etc.) Most times load-based problems cannot be solved by simply throwing more hardware at it.
You have to take an intelligent approach and actually simulate the load to isolate all these issues. Not an easy task.
Comment from Matt: Hi Shoeb, what kinds of tests should be performed at the hardware level specific that are unique to mobility?
Shoeb Javed: Well if you are testing on actual mobile devices than I suggest things like removing the battery, which simulates a power down condition.
I would also suggest hardware-level accelerometer tests, swapping SIM cards to change networks, etc. There are solutions that give you full hardware level control of the device in the cloud that would be great for this purpose.
Amy Thistle: In your session you discuss Testing Centers of Excellence. Can you talk about organizations that have started building a TCOE? What are the reasons for centralized testing teams?
Shoeb Javed: Good question Amy. We are seeing more and more of our Fortune 500 clients going to Test Centers of Excellence. Given the diversity of enterprise applications that they have to support and the volume of change that gets driven through these systems, doing testing ad hoc is no longer a viable option for a lot of companies.
A centralized competency allows much more efficient utilization of resources, infrastructure, best practices, and money.
Amy Thistle: It also sounds like having a tool is not the end of the story – successful testing still depends on the right processes. You mention “discipline” in your session presentation. Can you explain what those might be?
Shoeb Javed: A lot of companies want this TCOE to be in-house for the know-how and then augment through outsourcing for scale.
I think you have to have an institutionalized, repeatable approach to brining in new technology innovation. Discipline means following through consistently on a chosen methodology for all projects without exception.
Believe it or not, being disciplined makes projects actually go faster :)
Comment from Guest: How will technologies like cloud or HANA change our processes for testing performance or security?
Shoeb Javed: There will not be a fundamental change in those processes. For performance testing, you can still use standard HTML performance testing solutions for doing cloud and HANA applications. What you have to consider though would be more automated ways of doing it because you have to do it more often since things change a lot more frequently for HANA and cloud based applications.
Comment from Matt: Hi Shoeb, your presentation mentions measuring testing performance. What are you measuring? And which metrics and KPIs do you recommend across all mobile projects?
Shoeb Javed: I think the most important thing to measure would be end user performance. The response time of your application under various network and load conditions is key. Users are very impatient these days and any delay or perceived inefficiencies in your application will spell a death knell for adoption.
Amy Thistle: Thanks to everyone who joined the chat today. For more advice and tips on projects and mobility topics, be sure to join Shoeb and all our speakers at our upcoming Managing Your SAP Projects 2013 conference, which is co-located with our Enterprise Mobility event, November 20-22. Shoeb will be presenting on this topic at the event. We hope to see you in Orlando in a few weeks!
Shoeb Javed: I want to thank SAPinsider for the opportunity to moderate today’s Q&A forum. I hope to meet many of you at my session at next month’s Managing SAP Projects Conference. For more info. about automated business process validation, visit Worksoft at www.worksoft.com or contact email@example.com.
You can reach me directly at firstname.lastname@example.org.
Amy Thistle: And Shoeb, thank you again for taking these questions today. Looking forward to seeing you again in Orlando!