Report: The State of Mobile App Development and Testing 2014

Mobile is in everywhere. If you want to be successful in your domain you need to put your business to mobile world too. Because the trend of mobile device usage is still and is going to be increased, people want to continue to use web application with their mobile devices while they are mobile. According to Flurry, leading domain in this trend is social media like facebook, twitter, foursquare and many others; and  the other popular domains are respectively utilities, entertainment,  e-commerce, games.

By SmartBear, a report about the mobile application development and testing is published. Let's look at what SmartBear does and how to benefit from this research the we can analyze the situation. SmartBear is software company, mainly focus on software development, software quality and system management tools. They have more than 10 famous products on the market. For software quality and testing, SoapUI, LoadUI and TestComplete are well-known tools. Based on my own experinece, these tools are very effective and stable. For mobile business, SoapUI is usable testing APIs, automating regression and functional test cases. The licence of SoapUI is cheaper than $400 for a year. They also have many open-source and free tools but normally free version of the tools have limited features, you can reach complete list of tools from this link.  As a brief, SmartBear has tools for mobile application testing and the increase in mobile development and increasing trends towards mobile application quality may also increase the SmartBear revenue. Therefore this research may show the reality but, be careful, it may direct us to see what they want us to see from the mobile market.

From the report, the important points can be listed as following:

    %30 of the application development firms develop mobile application
    more than half of mobile application development firm started mobile application in last two years
        it means there is trend for mobile application development
        %84 percent of non-mobile application development firms plan to enter mobile
    Challenge in mobile development
        first one is the quality of products with %20
        profit is 5th challenges with %11
            not a real problem they may think being in mobile world is key factor, but then?
        competition has %7
            who is mobile world is the winner? almost there no rival but then?
    defect is really a problem for user
        %19 of user delete application immediately
        %30 wait and delete if it is not fixed
    testing type
        manual VS automated = %28 VS 18
        API testing is %18
            if the API doesn't work, then what reason to test application?
    who test the application
        developers and testers = %64
        testers = %22
        developers = %8
            testers can not test everything in mobile application?
            developers also have testing responsibility such as unit test

 You can get the report from

http://www2.smartbear.com/inbound-testcomplete-state-of-mobile-testing-webinar-website.html?&_ga=1.1394231.742788738.1391667943.

Report: The State of Mobile App Development and Testing 2014

Mobile is in everywhere. If you want to be successful in your domain you need to put your business to mobile world too. Because the trend of mobile device usage is still and is going to be increased, people want to continue to use web application with their mobile devices while they are mobile. According to Flurry, leading domain in this trend is social media like facebook, twitter, foursquare and many others; and  the other popular domains are respectively utilities, entertainment,  e-commerce, games. 

By SmartBear, a report about the mobile application development and testing is published. Let's look at what SmartBear does and how to benefit from this research the we can analyze the situation. SmartBear is software company, mainly focus on software development, software quality and system management tools. They have more than 10 famous products on the market. For software quality and testing, SoapUI, LoadUI and TestComplete are well-known tools. Based on my own experinece, these tools are very effective and stable. For mobile business, SoapUI is usable testing APIs, automating regression and functional test cases. The licence of SoapUI is cheaper than $400 for a year. They also have many open-source and free tools but normally free version of the tools have limited features, you can reach complete list of tools from this link.  As a brief, SmartBear has tools for mobile application testing and the increase in mobile development and increasing trends towards mobile application quality may also increase the SmartBear revenue. Therefore this research may show the reality but, be careful, it may direct us to see what they want us to see from the mobile market.

From the report, the important points can be listed as following:

• %30 of the application development firms develop mobile application
• more than half of mobile application development firm started mobile application in last two years
• it means there is trend for mobile application development
• %84 percent of non-mobile application development firms plan to enter mobile
• Challenge in mobile development
• first one is the quality of products with %20
• profit is 5th challenges with %11
• not a real problem they may think being in mobile world is key factor, but then?
• competition has %7
• who is mobile world is the winner? almost there no rival but then? 
• defect is really a problem for user
• %19 of user delete application immediately
• %30 wait and delete if it is not fixed
• testing type
• manual VS automated = %28 VS 18
• API testing is %18
• if the API doesn't work, then what reason to test application?
• who test the application
• developers and testers = %64
• testers = %22
• developers = %8 
• testers can not test everything in mobile application?
• developers also have testing responsibility such as unit test

 You can get the report from this link.

Read more at http://www.testrisk.com/2014/02/report-state-of-mobile-app-development.html#Qcbvi6kdQc1BAfRp.99

Mobile is in everywhere. If you want to be successful in your domain you need to put your business to mobile world too. Because the trend of mobile device usage is still and is going to be increased, people want to continue to use web application with their mobile devices while they are mobile. According to Flurry, leading domain in this trend is social media like facebook, twitter, foursquare and many others; and  the other popular domains are respectively utilities, entertainment,  e-commerce, games. 

By SmartBear, a report about the mobile application development and testing is published. Let's look at what SmartBear does and how to benefit from this research the we can analyze the situation. SmartBear is software company, mainly focus on software development, software quality and system management tools. They have more than 10 famous products on the market. For software quality and testing, SoapUI, LoadUI and TestComplete are well-known tools. Based on my own experinece, these tools are very effective and stable. For mobile business, SoapUI is usable testing APIs, automating regression and functional test cases. The licence of SoapUI is cheaper than $400 for a year. They also have many open-source and free tools but normally free version of the tools have limited features, you can reach complete list of tools from this link.  As a brief, SmartBear has tools for mobile application testing and the increase in mobile development and increasing trends towards mobile application quality may also increase the SmartBear revenue. Therefore this research may show the reality but, be careful, it may direct us to see what they want us to see from the mobile market.

From the report, the important points can be listed as following:

• %30 of the application development firms develop mobile application
• more than half of mobile application development firm started mobile application in last two years
• it means there is trend for mobile application development
• %84 percent of non-mobile application development firms plan to enter mobile
• Challenge in mobile development
• first one is the quality of products with %20
• profit is 5th challenges with %11
• not a real problem they may think being in mobile world is key factor, but then?
• competition has %7
• who is mobile world is the winner? almost there no rival but then? 
• defect is really a problem for user
• %19 of user delete application immediately
• %30 wait and delete if it is not fixed
• testing type
• manual VS automated = %28 VS 18
• API testing is %18
• if the API doesn't work, then what reason to test application?
• who test the application
• developers and testers = %64
• testers = %22
• developers = %8 
• testers can not test everything in mobile application?
• developers also have testing responsibility such as unit test

 You can get the report from
Read more at http://www.testrisk.com/2014/02/report-state-of-mobile-app-development.html#Qcbvi6kdQc1BAfRp.99

The Bugs That Deceived Me

When I started my software development career, I was introduced to the big QA database. “The bug store” was where the testers stored all the bugs they found as well as those found by customers. I never thought there’s another way to work, until I moved to Typemock.
As a startup, we could choose whatever tools we wanted, and in the beginning, we used a wiki. Later on as the product grew in features, and thankfully with customers, we started looking for other tools. When I became a product manager, I decided the best way to deal with them is with an Excel file.
As much as I’d like to dismiss the big bad bug database (it’s not an “agile” tool), I can see a lot of resemblance between the two. It’s not about how the tool manages the information, it’s how we perceive it. Every time we look at the data, we perform an analysis that helps us make decisions, hopefully the right ones.
It is possible to make wrong decisions. Along the way, I’ve picked up a few traps where bug data mislead me to make bad decisions. These traps are in the data itself, not the tools, and can lead us in the wrong direction.

The Age of Bugs
When we start testing a new project, all bugs are comparable. That means that we can apply our analysis at the same moment in time. We can differentiate between high-priority and low-priority bugs and decide to fix the former, because at the time of our analysis, the former looked like a must-fix.
Of the products I have tested, the worst versions always seem to be the initial versions. I’ve a few of those projects, and the bug databases quickly filled with high-priority bugs; we didn’t get to fix all of them, either. We “managed scope,” cut some corners, and released a product. We also didn’t have the nerve to remove the high-priority (and sometimes the low-priority) ones from our database. A year later, we still had high-priority bugs in our system.
Yes, the database was cluttered. The more open bugs we had, the longer that triage and re-analysis (or “grooming” in agile-speak) took. The real problem was the list of high-priority bugs contained both old and new high-priority bugs. The truth is, of course, that the old ones were not really high priority, but we still compared them as if they were.
The logical way to deal with these bugs, and the one I’ve adapted through the years, is to go back and re-prioritize. Recently though, I’m more inclined to deleting as many bugs that I don’t see us handling in the very near future. Some don’t even go into the database, because the team closes the loop quickly and decides together that these bugs can wait. It’s still a struggle, both internal and within the team (“We need to keep this, so we don’t forget how important it is”).
Big databases seem bigger every time you go back to them. Make them as small as possible by removing the less important stuff. It may take some cold decisions, but it will focus the team on what’s important.
Bug data doesn’t just risk our decision-making process about what to handle next. It can also point us away from where we can really improve our process quality.

The Bug and Code Disconnect
I’ve managed bugs in different ways over the years. In all projects, they were never connected directly to the source code. This disconnect makes it hard to spot problems in specific parts of the code. The closest I got was the component level; I knew which component was more bug ridden than others. However, the code base was large and the information was not helpful in pinpointing problems. This was never a quantitative measure as bugs were usually tagged as belonging to components during analysis, but the real code changes were not logged. We could not rely on the tagging as a problem locator
Some application lifecycle management (ALM) tools do the connection: Once you have a work item for the bug, the code changes for the bug fix are kept under it. Yet, I found that extracting information from these tools is still hard and the information you get is partial.
Finding errors in the process around coding can save us loads of problems. We can avoid more bugs by diverting attention to the problem areas in coding, reviewing, and testing. I haven’t found a good tool for that yet, so I guess the solution is in the process; whatever tool you use, try to keep the bugs and related code connected and tagged correctly. If you can do that, you can do some very interesting analysis.
But that’s not all the data that gets lost.

The Lost Data
Here’s a shocker: all the bugs in our database were found during testing.
We officially call them bugs after we found them. But there are others that appear along the way that don’t get to that special occasion. These are the bugs that either the developer caught on the way, as part of coding, or the ones that were caught by the suite of automated tests.
“That’s what test suites are for, genius!”

Yes they are. And still, these unmentioned bugs can contribute to the same analysis. These bugs have been a blind spot for me. As I test the whole application, I don’t see them, and I’m definitely not aware of what happened before the code entered source control.
Because this data is lost, we‘re left with the bugs in the database.
To tell the truth, I’ve decidedly let this one go. Collecting all this information requires more attention, more data collection.
Instead, we discuss the big picture in a qualitative manner. Luckily, I work with a small team, and we do ongoing analysis of bugs as part of retrospectives. Although not accurate, these discussions help us to identify and handle the risky parts of the code.

More Data, Better Analysis

As a developer, I never thought about grouping bugs. When I found them in my code before anyone else, I didn’t even call it a bug. When I “grew up” and adopted a more encompassing point of view, I look at them differently.
Bug information doesn’t live in a vacuum. In agile we talk about context and how it’s part of information.
With a bug, we’re interested not just with the bug description, but also where and when it was found, how and who did the analysis, etc. We can then group bugs together to point us at quality problems.
Every once in a while, it helps to take a look at the big picture, not just look at the bugs individually. Bugs are usually symptoms of ineffective processes.

How Do I Start?
Start with simple questions about bugs like “Where do they come in droves?” and “Where do they not frequently appear?” After doing so, make a decision about what to track and follow up.
Then continue to ask questions. It’s not like these bugs are going away, are they?

sTICKYminds

10 best practices for an effective testing & QA implementation

After years of polishing and fine-tuning a division-wide testing & QA effort, we thought it would be a good idea to share the top ten lessons learned for what we now consider as the key to a successful outcome. We hope you find this short list useful as a source of validation or ideas.

1) Process: It is critical that the organization defines a process that is robust and certified by experts in order to initiate the software assurance quality culture. The process will serve as a guideline that may evolve over time. Most importantly, it should be made official and should be followed through. Improvements will be made until a mature process is established.

2) Managerial Commitment: Managerial commitment should stem from the CIO to ensure alignment from each of the development managers, as well as from the development areas of each country. Everyone must be aware of the value that is added by testing & QA to the business. The process, therefore, must account for the value of the solutions that it offers to the organization.

3) Personal Experience: Hiring someone as a tester that lacks necessary experience is a common mistake. It is vital to acknowledge that the position requires experience in both the business and in software development in general.

4) Deliverables: As part of the software development and testing processes, it is necessary to define deliverables, such as requirements, a testing plan, and testing cases. These will guarantee that testers can effectively follow-up throughout the project from the software quality perspective.

5) Tool Usage: Both the use of tools for tracking and managing defects, as well as the creation of test cases and execution, are essential for increasing the maturity of the testing & QA process. The process may begin without tools, but they are a requisite for increasing execution maturity.

6) Metrics: Developing and creating metrics to track the software quality in its current state, as well as to compare the improvement with previous versions, will help increase the value and maturity of the testing process (e.g. the number of components with errors in the software/the total number of components in the software; or the number of errors detected in the testing phase/total number of errors detected).

7) Testing Environment: Implementation of appropriate testing environments that allow developers to reproduce the system execution in production environments is crucial to the creation and execution of the corresponding test cases.

8) Test Data: The testing environment required for day-to-day operation should provide or ensure availability of the necessary data to enable the corresponding test execution. Even if you have developed the appropriate testing environments, developers need to access specific data required to execute the associated test cases.

9) Change Management: Like any other production environment, the testing environment should properly track changes in configuration, ensuring not only controlled results, but that the tests are run in environments that closely resemble those of the real production environment.

10) Developer Awareness: It is critical to have an awareness process that includes management commitment at each and every business unit and for associated developers. The goal is to demonstrate that testing activities add value to their daily work.