A couple of months ago, I had the opportunity to experiment with a mobile device testing tool called DeviceAnywhere.   The testing suite allows mobile application developers and organizations the ability to test their applications on a variety of mobile devices without having to procure the physical hardware.

Based on available online sources, some of the benefits that can be realized using DeviceAnywhere are as follows:

·      Real handsets – on live mobile networks – remotely accessible over the Internet

·      Distributed teams can access real handsets

·      Network and OS agnostic

·      Automate tests across hundreds of handsets

·      Script tests that use multiple devices (eg. messaging)

·      Record and view results

·      Schedule tests to run at your convenience

·      Streamline customer care through virtual phones and step-by-step tutorials

·      Benchmark and trend performance and quality

·      Schedule repeatable tests, and view results in an easy-to-use reporting interface

·      Identify glitches through alerts before end-users do

During a product deep dive presentation, I also had the opportunity to ask some questions regarding the test suite.  These questions are summarized below:

Q. Will SIM applications residing on a SIM that is in a SIM switcher have access to all the information within the mobile device? In many security models, the SIM application leverages the mobile device’s IMEI code as well as the MDN for security authentication.  With the SIM card not physically inserted into the mobile device, will application security logic become an issue? 


A. The SIM switcher is hard wired to the mobile device so the SIM will look as though it is physically seated in the mobile device.   Because of this, the SIM switcher will have access to all of the mobile device’s firmware information.

Q. DeviceAnywhere mobile devices reside in many data centers globally.  However, if you decide to test a SIM application for an operator located in a region that DeviceAnywhere does not have a data center in, will the performance test results be influenced by network latency?  For instance, if you are testing a Bharti AirTel SIM application in London, how do we extrapolate carrier network hops and partner roaming handoffs out of the test results so that we have representative test results for the Bharti AirTel application?

A.  At the current time, removing the network latency due to network hops from test results is not available. 


Q. Can DeviceAnywhere provide granular trace reports for troubleshooting errors?  For instance errors at the mobile device layer versus errors at a network communication layer versus errors at a GPRS roaming partner network, etc. 


A. DeviceAnywhere can provide reports on performance, errors, logs, test scenario and settings of the mobile device itself.  Information on carrier network and roaming partner performance cannot yet be reported through DeviceAnywhere.

Q. Is DeviceAnywhere capable of leveraging dynamic data for automated testing? 


A. The automated test suite provided by DeviceAnywhere allows the end user to develop test scripts that can repeatedly execute one or many mobile devices at once.  The test data can be dynamic at each execution, and on each mobile device.

Q. What is the format of the exported test results? Can the format easily configured for reporting engines such as CrystalReports, Cognos or Actuate? 


A. The reports can be customized to the specific needs of the organization.

Q. How quickly are new mobile devices added to the DeviceAnywhere roster for testing?  I.e. upon new product release, the mobile device is added within a week? Prior to market release? Etc. 


A. Usually, the mobile devices are added within 2 days of market release.  However, if the testing organization has access prototype handsets, DeviceAnywhere can install them within any data center of your choice.

Q. For test scripting, and test automation, what is included and supported in the monthly fees?  

A. Instructional documentation is included with monthly memberships.  In addition, customized training curriculums are available upon request.

After playing around with the functions for a few days, my overall impression of the test tool was favorable.  Although testing SIM based applications was possible, deploying SIM cards was inefficient since most SIMTK applications had to be loaded onto SIM cards, sent to the data center, and physically swapped onto the SIM switch.  The process was as fast as the SIM cards can be burned and delivered.

Despite this, I found it useful to have the ability to quickly test one application on many devices very convenient.  Especially when you don’t have to deal with procuring the hardware, dealing with chargers and USB connection cables, setting up the development environment with drivers and synchronization software, and compatibility issues with drivers of various mobile devices.  DeviceAnywhere provides a consistent user interface that allows the tester to focus on…well, testing.  Deploying an application to the mobile device can take various form such as over-the-air, SMS download, Browser download, or in the case where direct connectivity is available, it is as simple as selecting the application version from your desktop, and selecting which phone you want to deploy the application to.

Another strength I found was the ability to validate and test mobile applications in roaming partner networks without having the need to deploy testers internationally.

The bottom line is if you have the need to test your mobile applications on a variety of handsets, on a global basis, DeviceAnywhere can save you cost on testing.

 

---

Name: Long Van Title: Mobile Financial Services Solution Architect Company: DonRiver Inc. View Long Van's Blog