Cloud has been one of the buzz words in the IT industry over the last
few years now: and it is, for the right reasons. As we all know the
world of cloud can manifest itself in one of the following forms to be
leveraged by various disciplines in the IT industry:
1. Software as a Service (SaaS)
2. Platform as a Service (PaaS)
3. Infrastructure as a Service (IaaS)
From a software testing perspective, SaaS and IaaS become more important than PaaS itself. In this article, I want to discuss with you the elements that are important to take note of and act on, from a Software Testing angle, in the arena of SaaS and IaaS.
Software as a Service (SaaS) is increasing in its popularity by the day and this can be attributed to several reasons such as: evolving cloud technology, need for optimization of resources and on-demand usage, global expansion of business that necessitates a platform to bring its people together, to name a few.
As an independent testing company, that has been testing SaaS applications over the last few years now, I will be talking about how testing for SaaS applications is different from testing a traditional on-premise implementation or rather what more to additionally focus on, in SaaS applications. To do this, let us look at a simple definition of SaaS and derive testable elements from it.
SaaS is a software delivery model, where the software is made available to the end customers over the Internet. The software vendor hosts the software for the customer, without the customer having to build up hardware or software environments, at their end. The customer herein, just provides software customization requirements to the vendor and follows a "pay per use model" to enable optimum use of resources. SaaS offers the customer the flexibility to scale up or down, the resource usage on an "on-demand" basis. Depending on the SaaS implementation model, there are varying levels of SaaS model maturity ranging all the way from:
• Single instance - single tenant (customer)
• Customizable multi tenancy on the base source code
• Multi tenancy on the base source code without any customizations
Let us now draw the testable elements from this definition:
Take away 1:
Software made available to end customers over the Internet: Software has evolved to enable us carry on a zillion different activities all the way from day to day shopping, banking, networking to possibly onetime events like home purchase, finding a match for yourself etc. If you are going to do all of this over the Internet, what is the first concern that comes to your mind? Privacy, confidentiality, security. Are all my transactions going to be secure especially when multiple other tenants use the same instance of the software? Security testing, especially, web application security testing is very important for SaaS applications. Take the initiative of doing the threat model analysis, or if your development team owns it, actively participate to ensure you understand the potential threats so you can test them thoroughly.
Take away 2:
The software vendor hosts the software for the customer: Given the scale of operations involved in the SaaS model, performance is something to definitely plan for in advance, and test. This is something that the end customers have come to expect, however, this can become challenging for the vendor when multiple instances are being planned for. Discuss the performance requirements with the business or product planning team upfront and start this testing early on in the product life cycle to conduct incremental tests. Besides just performance testing, scalability and capacity planning testing is also important in SaaS given that it works on an "on demand" model. Scalability and capacity planning does not always mean throwing in more hardware - doing so, may potentially result in wasted resources. Work with the development team to understand how scalable the product architecture is, to see if any optimizations can be done. Start all these discussions early on, since architectural changes may be very expensive, if not impossible, to implement at later stages.
Take away 3:
The customer herein just provides software customization requirements to the vendor and follows a "pay per use model" to enable optimum use of resources: Besides the core product functionality, two other important aspects to test in SaaS implementations are: right metering and billing and the right customer customizations.
One of the inherent benefits of the SaaS model is the pay-per-use feature. Unless this has been implemented correctly, there is no way to keep track of the customers accurate usage and this may affect the success of the entire application. Test for all the payment and usage algorithms, including boundary cases to ensure the implementation is correct.
Since often times the same application instance is customized and offered to meet the customer requirements, ensuring all of the customizations have been done correctly is very important. This may sound simple but any bug that is missed here, affects the customers reputation badly. Imagine having a logo of a competitor on your page - this could be disastrous. This is an area to be specially looked at, unlike in the traditional on-premise applications where this does not pose much risk.
SaaS applications, when implemented correctly and adequately tested for, have a lot of benefits to offer. Reap those benefits, to make your software a success, by proactively planning for these above areas well in advance.
Lets now look at IaaS. Not long ago, we would all remember the heavy investments software companies would make in setting up labs, hardware, inventory, communication with vendors for hardware shipments, obsolete hardware within years (if not months) of purchase, significant amount of testers time going into setting up the test environment...the list could go on.
The evolution of the cloud in its IaaS form has been a boon to the software industry especially the testing industry. What really is IaaS?
Also referred to as Hardware as a Service (HaaS) sometimes, this is a model where the environment setup is completely outsourced to a vendor where they take care of the machine setup, provisioning and configuration to address your specific needs, scaling the lab to meet your needs and charging you based on your usage.
In areas such as performance testing, compatibility testing - both manual and automated, localization testing, leveraging IaaS is significantly faster and also more economical in the longer run if you take into account hardware depreciation and other associated costs of setting up the inventory in your own premises.
The other advantages this model offers include:
1. Complex setup done by experts accurately represent your test lab requirements
2. Freeing up your testers and build engineers from the mundane setup tasks, motivates and gives them more time to focus on their specialized areas of work
3. Faster lab setup and scaling options result in quicker test cycles and a faster time to market
4. Hardware and Software that is typically hard to procure or which touch upon some remote cases, which would otherwise not be justified in setting up on premise, can now be easily accommodated in your test strategy
5. Easier debugging and troubleshooting, where sharing a common environment with the rest of the product development team is now feasible
While all of this sounds to be a picture perfect scenario, one should be very careful on choosing the right IaaS vendor, carefully thinking through the configuration requirements, and having a plan to validate the environment before actually using it. Remember that although you have the complete control in deciding your lab needs, you are not physically controlling them, rather only virtually controlling them. So, it becomes all the more important to ask the right questions, and consider options in finalizing the vendor of choice.
In addition, also keep in mind that the virtual/hosted infrastructure is no replacement to the testing teams valuable discretion on the optimized test environment to use. The IaaS vendor is only going to work on your instruction set to meet your needs and provide suggestions if you need them: you know your product, its architecture and test scenarios the best, which are very important in deciding the infrastructure setup that optimally meets your needs.
In conclusion, the Cloud helps improve a test teams productivity to a very large extent. Understand what it has to offer, think through how you can use it for your specific project at hand, go in for a customized solution rather than an out of box base solution-with these in place, you can certainly distinguish yourself in the marketplace with your set of offerings regardless of whether you are a product or a services company.
1. Software as a Service (SaaS)
2. Platform as a Service (PaaS)
3. Infrastructure as a Service (IaaS)
From a software testing perspective, SaaS and IaaS become more important than PaaS itself. In this article, I want to discuss with you the elements that are important to take note of and act on, from a Software Testing angle, in the arena of SaaS and IaaS.
Software as a Service (SaaS) is increasing in its popularity by the day and this can be attributed to several reasons such as: evolving cloud technology, need for optimization of resources and on-demand usage, global expansion of business that necessitates a platform to bring its people together, to name a few.
As an independent testing company, that has been testing SaaS applications over the last few years now, I will be talking about how testing for SaaS applications is different from testing a traditional on-premise implementation or rather what more to additionally focus on, in SaaS applications. To do this, let us look at a simple definition of SaaS and derive testable elements from it.
SaaS is a software delivery model, where the software is made available to the end customers over the Internet. The software vendor hosts the software for the customer, without the customer having to build up hardware or software environments, at their end. The customer herein, just provides software customization requirements to the vendor and follows a "pay per use model" to enable optimum use of resources. SaaS offers the customer the flexibility to scale up or down, the resource usage on an "on-demand" basis. Depending on the SaaS implementation model, there are varying levels of SaaS model maturity ranging all the way from:
• Single instance - single tenant (customer)
• Customizable multi tenancy on the base source code
• Multi tenancy on the base source code without any customizations
Let us now draw the testable elements from this definition:
Take away 1:
Software made available to end customers over the Internet: Software has evolved to enable us carry on a zillion different activities all the way from day to day shopping, banking, networking to possibly onetime events like home purchase, finding a match for yourself etc. If you are going to do all of this over the Internet, what is the first concern that comes to your mind? Privacy, confidentiality, security. Are all my transactions going to be secure especially when multiple other tenants use the same instance of the software? Security testing, especially, web application security testing is very important for SaaS applications. Take the initiative of doing the threat model analysis, or if your development team owns it, actively participate to ensure you understand the potential threats so you can test them thoroughly.
Take away 2:
The software vendor hosts the software for the customer: Given the scale of operations involved in the SaaS model, performance is something to definitely plan for in advance, and test. This is something that the end customers have come to expect, however, this can become challenging for the vendor when multiple instances are being planned for. Discuss the performance requirements with the business or product planning team upfront and start this testing early on in the product life cycle to conduct incremental tests. Besides just performance testing, scalability and capacity planning testing is also important in SaaS given that it works on an "on demand" model. Scalability and capacity planning does not always mean throwing in more hardware - doing so, may potentially result in wasted resources. Work with the development team to understand how scalable the product architecture is, to see if any optimizations can be done. Start all these discussions early on, since architectural changes may be very expensive, if not impossible, to implement at later stages.
Take away 3:
The customer herein just provides software customization requirements to the vendor and follows a "pay per use model" to enable optimum use of resources: Besides the core product functionality, two other important aspects to test in SaaS implementations are: right metering and billing and the right customer customizations.
One of the inherent benefits of the SaaS model is the pay-per-use feature. Unless this has been implemented correctly, there is no way to keep track of the customers accurate usage and this may affect the success of the entire application. Test for all the payment and usage algorithms, including boundary cases to ensure the implementation is correct.
Since often times the same application instance is customized and offered to meet the customer requirements, ensuring all of the customizations have been done correctly is very important. This may sound simple but any bug that is missed here, affects the customers reputation badly. Imagine having a logo of a competitor on your page - this could be disastrous. This is an area to be specially looked at, unlike in the traditional on-premise applications where this does not pose much risk.
SaaS applications, when implemented correctly and adequately tested for, have a lot of benefits to offer. Reap those benefits, to make your software a success, by proactively planning for these above areas well in advance.
Lets now look at IaaS. Not long ago, we would all remember the heavy investments software companies would make in setting up labs, hardware, inventory, communication with vendors for hardware shipments, obsolete hardware within years (if not months) of purchase, significant amount of testers time going into setting up the test environment...the list could go on.
The evolution of the cloud in its IaaS form has been a boon to the software industry especially the testing industry. What really is IaaS?
Also referred to as Hardware as a Service (HaaS) sometimes, this is a model where the environment setup is completely outsourced to a vendor where they take care of the machine setup, provisioning and configuration to address your specific needs, scaling the lab to meet your needs and charging you based on your usage.
In areas such as performance testing, compatibility testing - both manual and automated, localization testing, leveraging IaaS is significantly faster and also more economical in the longer run if you take into account hardware depreciation and other associated costs of setting up the inventory in your own premises.
The other advantages this model offers include:
1. Complex setup done by experts accurately represent your test lab requirements
2. Freeing up your testers and build engineers from the mundane setup tasks, motivates and gives them more time to focus on their specialized areas of work
3. Faster lab setup and scaling options result in quicker test cycles and a faster time to market
4. Hardware and Software that is typically hard to procure or which touch upon some remote cases, which would otherwise not be justified in setting up on premise, can now be easily accommodated in your test strategy
5. Easier debugging and troubleshooting, where sharing a common environment with the rest of the product development team is now feasible
While all of this sounds to be a picture perfect scenario, one should be very careful on choosing the right IaaS vendor, carefully thinking through the configuration requirements, and having a plan to validate the environment before actually using it. Remember that although you have the complete control in deciding your lab needs, you are not physically controlling them, rather only virtually controlling them. So, it becomes all the more important to ask the right questions, and consider options in finalizing the vendor of choice.
In addition, also keep in mind that the virtual/hosted infrastructure is no replacement to the testing teams valuable discretion on the optimized test environment to use. The IaaS vendor is only going to work on your instruction set to meet your needs and provide suggestions if you need them: you know your product, its architecture and test scenarios the best, which are very important in deciding the infrastructure setup that optimally meets your needs.
In conclusion, the Cloud helps improve a test teams productivity to a very large extent. Understand what it has to offer, think through how you can use it for your specific project at hand, go in for a customized solution rather than an out of box base solution-with these in place, you can certainly distinguish yourself in the marketplace with your set of offerings regardless of whether you are a product or a services company.
