jameleon.sourceforge.netJameleon - An Automated Testing Tool - Overview

Jameleon Last published: 13 February 2008 11:11 | Doc for 3.3-RC1 Learning Jameleon Jameleon Manual in PDF Introduction Overview XML Basics Getting Started Helpful Links FAQ Changes Download Report a Bug Request a Feature Plug-ins Developing a Plug-in JUnit plug-in Jiffie plug-in Selenium plug-in Watij plug-in Jagacy (3270) plug-in HtmlUnit plug-in HttpUnit plug-in jWebUnit plug-in Technical Documentation Syntax Reference Lifecycle of a Tag Test Case Tag Session Tag Function Points Ant Tasks Assert Levels Global Settings Data-driving Strategies Installation Java API Project Documentation About Jameleon Project Info Project Reports Introduction Jameleon is an automated testing framework that can be easily used by technical and non-technical users alike. One of the main concepts behind Jameleon is to create a group of keywords or tags that represent different screens of an application. All of the logic required to automate each particular screen can be defined in Java and mapped to these keywords. The keywords can then be organized with different data sets to form test scripts without requiring an in-depth knowledge of how the application works. The test scripts are then used to automate testing and to generate manual test case documentation. Jameleon was designed to test many kinds of applications. To make this possible, Jameleon was designed with a plug-in model. Currently, there are five plug-ins offered: A JUnit plug-in which can be used to test at a white box level. All JUnit function tags can be used in conjunction with other plug-ins. A Jiffie plug-in which drives Internet Explorer and can therefore only be run on Windows. Most testers like this plug-in the most. An HtmlUnit plug-in which emulates a browser and supports JavaScript quite well. Because it emulates a browser, tests written in this plug-in can execute on any OS supported by Java. A Selenium Plug-in which drives the most popular browsers. tests written in this plug-in can execute on the most popular OSes. Seee the selenium site for more in-depth information about Selenium-RC. A Watij Plug-in which drives Internet Explorer and can therefore only run on Windows. This plug-in has a lot of potential. An HttpUnit plug-in which emulates a browser, but doesn’t have very good JavaScript support. Developers like this plug-in the most. Because it emulates a browser, tests written in this plug-in can execute on any OS supported by Java. A 3270 (Jagacy) plug-in which is used to automate mainframe applications. A jWebUnit plug-in which is the most basic among the provided plug-ins and currently has no generic tags other than a session tag. This plug-in is available for backwards compatibility only and should be avoided for new projects. If Jameleon doesn’t have some feature or plug-in, please feel free to request it as a feature via the Request a Feature link. Implementing Jameleon plug-ins is simple and can usually be done in less than a hour. Even though it would be possible to write unit tests using Jameleon, Jameleon was designed with integration, regression, functional, and acceptance-level testing in mind. Most bugs are found and fixed by good unit tests. However, this does not eliminate the need to test the application as a whole. Intrinsic Problems with Automated Testing Automated testing does not solve all of the testing needs. In fact, it usually introduces several new problems. While many of these problems may be addressed by being extremely disciplined in an automated testing approach, it is still important to be aware of these problems. Automated Scripts Become Outdated by Frequent Changes to the Application. Unless scripted by a self-disciplined group of testers who understand object-oriented principles, it is likely the automated scripts will contain a lot of duplicated code. Imagine having hundreds or even thousands of automated scripts written against a word processor. It is highly likely that those scripts will end up using several features of the word processor just to arrive at the point being tested. Let’s take editing a file as an example. To test the underline feature, the application must be started, a file must be opened, text must be selected, the underline button must be pressed and finally the text must validated as being underlined. This single test includes four actions; not to mention, validating the successful completion of each of the actions. A change in any of these application features necessitates changes in the underline test as well as many other tests. Flexibility and Power is Often Sacrificed for Ease of Scripting. Many of the available tools have as a selling point that they make writing automated tests quick and easy. To do this they rely on simplified scripting languages, which don’t allow access to external resources or publicly developed and supported libraries. The languages also often limit the extent of abstraction and reuse, or at the least do not encourage good coding practices for maintainability over the long run. To make scripting easier, the tools lose the complexity of a full language, so organizations can only feasibly test the easier problems, missing most of the unique value of automation. Because of the lack of flexibility, it is common to see organizations needing several different tools for web testing, desktop GUI testing, client/server testing, load testing, etc. Now testers have to learn many simplified scripting languages, so in the long run it may not be so simple to learn. Automated Scripts are Hard to Read and External Docs Get Out of Sync. For both automated and manual testing, test cases which document the steps required to pass the business rules, are the base of any test plan. Because automated scripts are usually managed separate from the test cases, the automated scripts and the test cases have a high probability of getting out of sync. Soon, no one knows whether the test case or the test script represent the current state of the application. Too Much Work to Test all Possible Scenarios. In many cases, the data being tested cannot be easily separated from the actual test script. This requires creating several test scripts or requiring the data be included in the code itselft in order to test the many small rules of a single business rule. This not only requires extra work to test a that single feature, but it also makes it a nightmare to maintain the test scripts that only differ in the data being used to test the application. Automated Scripts can be Difficult to Port Between Environments. Many of the automated testing frameworks available don’t take the lifecycle of a product into consideration. A product must go through development, alpha testing, beta testing, and performance testing before it reaches the masses. Because of the infeasibility of editing large numbers of scripts, many organizations only run their automated tests against one environment, thus missing opportunities for validation and regression testing as the application advances in the product lifecycle. Need to Learn Proprietary Technologies. Most testing tools require learning a propriety language or technology. This makes it harder to find experienced people, which restricts a company’s options in selecting testing tools. The narrow applicability of proprietary technologies may also deter current employees from acquiring a more in-depth knowledge of the tool. Jameleon to the Rescue Jameleon attempts to solve many of the intrinsic problems ... Jameleon Takes a More Object-Oriented Approach to Testing Applications. Jameleon separates applications into features that get scripted independently in Java. These features can then be tied together to create a test-case. When a feature in the application is changed, the only file to change is the feature that was coded in Java. All test cases can remain untouched and still pick up the the new changes. Jameleon Supports Scripting at Different Levels of Complexity. Jameleon is an engine, not a specific...