Jan 20, 2013

Designing Test Automation Framework with Selenium

Do we need to go for Test Automation? If yes, then which is the tool?
If we straight away start automating test cases what are all the technical challenges we might face? Do we need a framework for Automation? How we go about designing a framework?
These are all the questions popup when we want to start Test Automation. Okay, let us start exploring answers for the above questions.

If there are hundreds of test scenarios which needs to be executed on each build then it will be a tedious task and error prone.  Because a Test Engineer might make mistakes while entering the test data and output might be wrong. We can avoid such situations by automating the test cases.
Before selecting a tool for Test Automation, we need to analyze the technologies used for different UI components. For example, Java Swing components, Telerik controls, ExtJS tables, Ajax, Silverlight, Flex / Flash controls, etc. If we want to use Selenium as Test Automation tool then the first and foremost thing needs to be checked is whether the application is a Web Application. If so, then check whether most of the UI controls are identified by the Selenium. If more than 70% of the test cases can be automated then definitely we can go ahead and automate the test cases.
In order to handle large number of test cases we need to have a clear cut strategy for designing the framework. Some of the framework functionalities can be mentioned as follows:
  1. If there is any unhandled exception, then the execution should not stop.  Particular test scenario should be stopped and continue with the next scenario.
  2. Screen shots must be taken for each error so that analyzing the failures should be easy.
  3. Proper exception handling should be done in the framework.
  4. Proper naming convention should be used
  5. Avoid hard coding test data
  6. Test Data should be parameterized using external means such as XMLs, Database, Spreadsheet, etc.
  7. Logs should be created that makes life easier while analyzing the Test Execution
  8. Proper folder structure should be created.
 If we need to create a Test Automation Framework with all the above mentioned features (or anything beyond this) for Selenium, then we need to use the programming language as JAVA and Eclipse as the IDE (Integrated Development Environment).
Test Automation Engineers should have knowledge of Java then it makes their life easier for automating the test cases

Jan 19, 2013

Testing for Agile Software Development


Testing for Agile Software Development

In this tutorial you will learn about Testing for Agile Software Development - Background, Understanding Agile Software Development, How is Testing approach different in an Agile Development Scenario? What to test? Typical bugs found when doing agile testing? Steps Taken to Effectively Test in Agile Development Methodology, Ensuring software test coverage and Summary

Background:

To understand the Testing Process in an Agile Development Methodology, it is important to understand the Agile Development paradigm.
Agile Development paradigm is not very new. Although the Agile Software Development Manifesto came into existence in February 2001, the concepts existed long before that and were expressed in different ways. Spiral Development Methodology is one such example.

Understanding Agile Software Development:

The Agile Software Development primarily focuses on an iterative method of development and delivery.
The developers and end users communicate closely and the software is built. A working piece of software is delivered in a short span of time and based on the feedback more features and capabilities are added.
The focus is on satisfying the customer by delivering working software quickly with minimum features and then improvising on it based on the feedback. The customer is thus closely involved in the Software Design and Development Process.
The delivery timelines are short and the new code is built on the previous one.
Despite this, high quality of the product cannot be comprised.

This creates a different set of challenges for Software Testing.

How is Testing approach different in an Agile Development Scenario?

The Testing Strategy and Approach in Agile Development could be very different from traditional bureaucratic methods. In fact it could vary with project needs and the project team.
In many scenarios, it may make sense to not have a separate testing team.
The above statement should be understood carefully. By not having a testing team we do not consider testing to be any less important. In fact testing can done more effectively in short turn around times, by people who know the system and its objectives very well.

For example in certain teams Business Analysts could be doing a few rounds of testing each time the software version is released. Business Analysts understand the Business Requirements of the Software and test it to verify if it meets the requirements.

Developers may test the software. They tend to understand the system better and can verify the test results in a better way. Testing for AGILE Software Development requires innovative thinking and the right mix of people should be chosen for doing the testing.

What to test?

Given the relatively short turn around times in this methodology it is important that the team is clear on what needs to be tested. Even though close interaction and innovation is advocated rather than processes, sufficient emphasis is given to the testing effort.

While each team may have its own group dynamics based on the context, each code has to be unit tested. The developers do the unit testing to ensure that the software unit is functioning correctly.
Since the development itself is iterative it is possible that the next release of code was built by modifying the previous one. Hence Regression Testing gains significant importance in these situations.
The team tests if the newly added functionality works correctly and that the previously released functionality still works as expected.

Test Automation also gains importance due to short delivery timelines. Test Automation may prove effective in ensuring that everything that needs to be tested was covered.
It is not necessary that costly tools be purchased to automate testing. Test Automation can be achieved in a relatively cost effective way by utilizing the various open source tools or by creating in-house scripts. These scripts can run one or more test cases to exercise a unit of code and verify the results or to test several modules.
This would vary with the complexity of the Project and the experience of the Team

Typical bugs found when doing agile testing?

Although nothing is typical about any Agile Development Project and each project may have its own set of complexities, by the very nature of the paradigm bugs may be introduced in the system when a piece of code is modified/enhanced/changed by one or more Developers.
Whenever a piece of code is changed it is possible that bugs have been introduced to it or previously working code is now broken. New bugs/defects can be introduced at every change or old bugs/defects may be reopened.

Steps Taken to Effectively Test in Agile Development Methodology:

As a wise person once said there is no substitute to hard work.
The only way one can effectively test is by ensuring Sufficient Test Coverage and Testing Effort Automated or otherwise.
The challenge could be lack of documentation, but the advantage could be close communication between team members thereby resulting in greater clarity of thought and understanding of the system.

Each Time Code is changed Regression Testing is done. Test Coverage is ensured by having Automated Scripts and the right mix/combination of People executing the Test Cases.
Exploratory Testing may also be encouraged. Exploratory Tests are not pre-designed or pre-defined. The Tests are designed and executed immediately. Similarly ad hoc testing may also be encouraged. Ad-hoc testing is done based on the tester’s experience and skills.

While automated Test Cases will ensure that the Test Cases scripted are executed as defined, the team may not have enough time to design and script all the test cases.

Ensuring software test coverage

To ensure that delivered product meets the end user’s requirements it is important that sufficient testing is done and all scenarios are tested.

Sufficient Test Coverage in an Agile Development Scenario may be tricky but with close cooperation and the right team dynamics it may not be impossible.

The objectives of the Project should be clear to the entire team. Many teams advocate Test Driven Development. At every stage the Software is tested if it meets the Requirements. Every Requirement is translated to a Test Case and Software is validated/ verified. While the Processes and Documentation are not stressed upon sufficient steps are taken to ensure that the software is delivered as per the User Expectations. This implies that each Software delivery should be tested thoroughly before it is released.

The timelines being short requires that the person testing the software has sufficient knowledge about the system and its objectives

Summary:

In this article we studied Software Testing Methods and its challenges for Agile Software Development.

Software Cost Estimation

Software Cost Estimation

This article aims to study the process of Software Cost Estimation and its impact on the Software Development Process. We also highlight the various challenges involved in Software Cost Estimation and common solutions to navigate through these challenges.
Background:
Software Cost Estimation is widely considered to be a weak link in software project management. It requires a significant amount of effort to perform it correctly. Errors in Software Cost Estimation can be attributed to a variety of factors. Various studies in the last decade indicated that 3 out of 4 Software projects are not finished on time or within budget or both
Who is responsible for Software Cost Estimation?
The group of people responsible for creating a software cost estimate can vary with each organization. However the following is possible in most scenarios -
- People who are directly involved with the implementation are involved in the
..estimate.
- Project Manager is responsible for producing realistic cost estimates.
- Project Managers may perform this task on their own or consult with
..programmers responsible.
- Various studies indicate that if the programmers responsible for development
..are involved in the estimation it was more accurate. The programmers have
..more motivation to meet the targets if they were involved in the estimation
..process.
Following scenarios are also possible
- An independent cost estimation team creates an Estimate
- Independent Experts are given the Software specification and they create a
..Software Cost estimate. The Estimation team reviews this and group
..consensus arrives at a final figure.
Factors contributing to inaccurate estimation
· Scope Creeps, imprecise and drifting requirements
· New software projects pose new challenges, which may be very different from
..the past projects.
· Many teams fail to document metrics and lessons learned from past projects
· Many a times the estimates are forced to match the available time and
..resources by aggressive leaders
· Unrealistic estimates may be created by various ‘political under currents’
Impact of Under-estimating:
Under-Estimating a project can be vary damaging
- It leads to improper Project Planning
- It can also result in under-staffing and may result in an over worked and
..burnt out team
- Above all the quality of deliverables may be directly affected due insufficient
..testing and QA
- Missed Dead lines cause loss of Credibility and goodwill
The Estimation Process:
Generally the Software Cost estimation process comprises of 4 main steps:
1) Estimate the size of the development product.
This comprises of various sub-steps or sub tasks. These tasks may have been done already during Requirement Analysis phase. If not then they should be done as a part of the estimation Process. Important thing is that they should be done to ensure the success of the Estimation Process and the Software Project as a whole
a) Create a detailed Work Break Down Structure. This directly impacts the accuracy of the estimate. This is one of the most important steps. The Work Break down structure should include any and all tasks that are within the scope of the Project, which is being estimated. The most serious handicap is the inability to clearly visualize the steps involved in the Project. Executing a Software Project is not just coding.
b) The work Break down structure will include the size and complexity of each software module that can be expressed as number of Lines of Code, Function Points, or any other unit of measure
c) The Work Break down structure should include tasks other than coding such as Software Configuration Management, various levels and types of Testing, Documentation, Communication, User Interaction, Implementation, Knowledge Transition, Support tasks(if any) and so on
d) Clearly indicate or eliminate any gray areas (vague/unclear specifications etc.)
e) Also take into account the various Risk Factors and down times. There are many different Risk Factors involved – Technical aspects such as availability of the Environment, Server/Machine uptime, 3rd party Software Hardware failures or Human aspects – Employee Attrition, Sick time, etc. Some of them may seem to be ‘overkill’ but real world experience shows that these factors affect the time lines of a project. If ignored they may adversely impact the Project timelines and estimates.
2) Estimate the effort in person-hours.
The Result of various tasks involved in step 1 is an effort estimate in person hours. The effort of various Project tasks expressed in person-hours is also influenced by various factors such as:
a) Experience/Capability of the Team members
b) Technical resources
c) Familiarity with the Development Tools and Technology Platform
3) Estimate the schedule in calendar months
The Project Planners work closely with the Technical Leads, Project Manager and other stakeholders and create a Project schedule. Tight Schedules may impact the Cost needed to develop the Application.
4) Estimate the project cost in dollars (or other currency)
Based on the above information the project effort is expressed in dollars or any other currency.

Measuring the Size/Complexity of the Software Program:
This is one of the most elusive aspects in the Software Cost Estimation Process.
There are different methodologies for arriving at and expressing the size/complexity of the Software Program. Some of the popular ones are
1) Function Points
2) Lines of Code
3) Feature Points
4) Mk II function points
5) 3D Function Points
6) Benchmarking

We briefly explain each of the above methods in the next few pages
Function Points
The Function Point methodology was developed by Allan Albrecht at IBM. This methodology is based on the belief that the size of a software project can be estimated during the requirements analysis. It takes into account the inputs and outputs of the system. Five classes of items are counted:
1. External Inputs
2. External Outputs
3. Logical Internal Files
4. External Interface Files
5. External Inquiries
The Total Function Point count is calculated based on the
a) Counts for each of these items
b) The weighting factors and adjustment factors in this methodology
What are function points and why count them?
“Function points are a measure of the size of Software applications and the projects that build them. The size is measured from a functional, or user, point of view. It is independent of the computer language, development methodology, technology or capability of the project team used to develop the application.”
Function points are not a perfect measure of effort to develop an application or of its business value, although the size in function points is typically an important factor in measuring each. Since the function point count for an application is independent of the technology used to develop the application it can be used for almost all types of applications such as GUI, OOP, Client Server, etc.
Since function points are based on screens, reports and other external objects, this measure takes the users' view. In these days of outsourcing and other confusion regarding the role of IT in an organization, understanding the users' view is of critical importance!
Lines of code:
Counting lines of code measures software from the developers' point of view The number of lines of code is the traditional way of measuring the application size. Many people consider this method as irrelevant now. There are technical problems with the lines of code measure. It is difficult to compare lines of code when a mix of technologies is used. There is no standard definition of what a line of code is. A Program may have blank lines, comments, data declarations, and multi-line statements.
Feature points Methodology:
It was developed by Software Productivity Research (SPR) in 1986. This technique takes into account the number of algorithms used in the application. It is compatible with the Function Points Methodology. The size calculated by the two methods for an ordinary transactional program would be the same. Feature Points Methodology is generally more useful for estimation in real-time process control, mathematical optimization and various embedded systems. The estimates are higher and considered more accurate in these cases.
Mk II function points Methodology:
This was developed Charles Symons in 1984 at Nolan, Norton & Co., part of KPMG Management Consulting. The Original Function Point approach suffers from the following weaknesses:
· It is often difficult to identify the components of an application.
· The original Function Point Methodology assigned weights to function point
..components based on "debate and trial."
· The original Function Point Methodology did not provide a means of accounting
..for internal complexity. ‘Feature points’ technique addresses these issues.
· When small systems are combined into larger applications. Function
..Points Methodology makes the total function point count less than the sum
..of the components.
MKII decomposes the application being counted into a collection of logical transactions. Each transaction consists of an input, a process and an output. For each transaction, Unadjusted Function Points (UFP) become a function of the number of input data element-types, entity-types referenced and output data element-types. The UFPs for the entire system are then summed. Mk II is widely used in the UK, India, Singapore, Hong Kong and Europe. Users include governmental organizations, finance, insurance, retail and manufacturing.
3D function points:
This methodology was developed by Boeing Company and published in 1992. The new technique was designed to address two classic problems associated with the Albrecht approach( the original Functional Point Methodology)
a) The original Functional Point Methodology is not user friendly
b) It is inaccurate when measuring complex scientific and real-time systems.
The 3D function points takes into account the following Dimensions - data, function and control. The data dimension is similar to the original Function Point Methodology. The function dimension accounts for the transformations or algorithms. The control dimension accounts for transitions or changes in application state.

Benchmarking:
Over the years many Organizations with significant development experience and mature processes have collected metrics on the various software development projects. These include the time, effort required to develop applications on various platforms and in various Business Domains. Based on this data benchmarks are created.
Each new software module to be developed can be categorized using the
a) Number of inputs
b) Number of outputs
c) Number of transactions
d) Algorithms
e) Features of the module
Based on the above factors the module can be categorized for example as Simple, Medium or Complex. If it is too Complex you could express it in multiples of the above three categories. The baseline effort in terms of person-hours it takes for each category is predefined based on historical data/metrics for a similar platform. This figure can be improvised/refined over a period of time This can be correlated to an algorithm for calculating Car Insurance Premium. This is used to estimate the size and the effort needed for Software Development.

Summary:
In this article we discussed the various steps in Software Cost Estimation We also discussed some of the popular techniques and industry best practices for Software Cost Estimation.