The Past
Software testing is a necessary method and all people seems to agree on that. Where people vary is the timing and depth of the initiative. The industry has lengthy moved from a degree whilst trying out changed into taken into consideration a put up-development, upkeep activity. We step by step interleaved the improvement and testing system. Both the improvement and the trying out system have a tendency to merge, bringing both new advantages and new demanding situations. The obvious gain is fast feedback. We need to check an mistakes as quick as feasible. The now not-so-obvious venture is protection and overhead. This will come to hang-out us later.
Long since the early days, the enterprise testing adopted Continuous Integration. Whenever a developer is ready to push a few code, he first has to skip it via the Unit Tests. If all is green then only can he genuinely devote the code. Even in line with-commit jobs are gaining reputation. People are delegating the responsibility of code validation to the construct server itself. The concept is to combine the Code - Build - Test - Deploy manner in a unmarried pipelined workflow. Every stage inside the pipeline gives instant feedback to the preceding degree.
On the builders' the front automatic Unit Tests have established themselves as a useful checkpoint. What approximately the managers and the enterprise users? Everyone wants things to be well examined. But Unit Tests and their reports aren't lots meaningful for them. The unit checks also are not dependable in integration checking out. Components having exceeded their unit exams may nonetheless fail to combine. Here comes the want of practical checking out. Organizations have devoted departments of manual-testers. They validate the product at complete construct degree. Detailed reports are generated, modules' fitness is monitored and malicious program monitoring is integrated to this system.
The Present
Bulk of functional checking out has continually been manual. A massive cause changed into lack of gear that could automate applications, detect and record the disasters in doing so. But the final decade saw an exponential boom the automation sector. The authority of pioneering merchandise changed into challenged by way of emerging gear. A majority of them had been open supply, promoting accessibility to even small companies. So whilst we will automate the practical trying out procedure, why now not integrate it to the CI server? Why now not unit test and functional take a look at every code batch? Here comes the hurdle which we knew but neglected on the preliminary stages - maintainability.
Automation scripts are traditionally codes. They are codes which test and validate different code; and they being code, need to be designed, written, maintained and reviewed. But they've a big benefit - they are able to execute simply as every other code. That's why JUnit and similar frameworks are so famous. Problem is that they do not scale properly with useful trying out. For one component, useful exams are very extensive, protecting many workflows, scenarios, check cases and precise steps. Such massive a code is difficult to keep and shortly outgrows the product itself. Another aspect is being code; they don't truely lie in tester's domain. They want seasoned developers. You can have programmers write the check scripts but then a load mismatch remains.
Welcome the generation of Keyword Driven Automated Functional Testing. Encapsulating the programming process in take a look at scripts gives many benefits. First and essential, we not want hard programmers for checking out project. Any programming-literate can do the mission. The Test scripts are greater maintainable and language agnostic. Now we are able to virtually have each unit checks and practical checks which can be induced on each code test in. While unit assessments validate code conformity, useful checks validate product's behavioral consistency. And both paintings unattended. Developers can get remarks meaningful to them from their unit assessments. Business customers can get module fitness popularity meaningful to them from their functional assessments, all in close to real time.
The Future
There is no restriction to innovation. Imagine the improvement of a web software. Let's say the utility needs to be supported on all major browsers, IE, FireFox, Chrome, Safari, and Opera. Across all Platforms: Windows/Linux/Mac Desktops/Laptops. Let us push the limits, ask for help on all important mobile gadgets: Android, iPhone, Blackberry, Windows; talk about orientations, resolutions. Now shall we us ask the check engineers to automate the whole lot. Run all regression flows on all Platforms/browsers on all test-ins. After all we need a strict high-quality manage.
Difficult to accept but this truely can't be done! Not without an automation framework that may do the heavy lifting. None of tools or frameworks inside the marketplace can deal with the necessities that we will be going through in coming years. What the industry desires is a check automation framework which:
Can combine with the existing CI answers.
Perform useful tests as seamlessly as Unit Tests. Run a check every night, each take a look at in or on every Run-Test request.
Scripting system is higher to be key-word pushed than language based. We need one simple ClickOnButton; now not a group of locate, hooks, performs, once more recheck hook and so forth.
Provides excessive maintainability of Scripts. Should now not need programmers to alter that validation.
Can gracefully orchestrate multiple equipment to perform activities which otherwise don't come under ambit of anybody single device. If Selenium can't click on on an image, get it accomplished with Sikuli.
Can launch more than one configurations of a same check in parallel. We need to take the same check on all 5 primary browsers.
Gives a uniform interface over heterogeneous tools. Tools may vary in language support even on platform assist but we want it. We want to check our application on each Linux and Windows.
Generates business oriented reviews. We want reviews in terms of Business flows no longer man or woman test instances. We want health take a look at of modules. We want simple metrics. We want to visualize defects over final 5 builds.
Drill down to step level failures, so that each testers and builders can get manageable statistics regarding the failure.
Integrate with principal Defect management gear like Bugzilla, Jira and so forth. Should be capable of routinely logging confirmed bugs.
Integrate with foremost Application Lifecycle Management software program like HP ALM.
Now that could be a whooping wish listing and no product in the marketplace claims to do it all. But this want is soon going to be our requirement. Success and failure of products will rely on well timed validation on all the target environments. Even if we by some means control the sheer quantity of check scripts and all the plumbing, we're stuck on the check infrastructure. We need to check on 5 browsers, each on cutting-edge model and three backward variations. We need to check those browsers on Windows, Linux and Mac. Now that makes approximately 100 parallel executions on a hundred infrastructure. How are you going to manipulate that infrastructure? What happens when you need a bunch of Mobile devices? Are you going to buy it all? Rent all of it? Of route not.
The future is of cloud computing. We will hire browsers on cloud. We will rent gadgets on cloud. We will rent the execution systems on cloud. Volumes check facts can be stored on cloud, analyzed on cloud structures and reported from cloud. All we need on premises is simply one CI tool and one check-automation-tool. The cloud will offer all of the relaxation. We will lease resources on pay-per-use basic. Pay for the exams we run. No need to inventory the hardware sources, browsers, devices and all.
And our steps to it
That future isn't so remote. You can get the machines from IaaS offers like Amazon or Rackspace. SauceLabs can hire you browsers; Xamarin can run your assessments on rented mobile gadgets. These answers are already to be had. But are these true answers for your pains? No, you need an quit to quit automation solution and these isolated initiatives do not help a whole lot. A carrier is required which can carry a lot of these collectively. A lot of gear have targeted the challenges and maximum have succeeded too. But until now it have been ordinarily isolated, unorganized attempts. Notable efforts are from QTP - which at the side of its Open Test Architecture is the huge brother of the modern-day generation of Test Automation answers and the opposite is OpKey - the following technology of Test Automation Framework.