When looking at Feature Matrix of any full-fledged software product I always miss one column called “Does it work as advertised?”. I’m sure it exists but is never displayed on vendors’ web sites. 😉
Thanks to today’s time-to-market pressure, many products are shipped in a condition that appears far from “well-done”. To understand if tool X or component Y meets your requirements, you typically have to spend a sufficient amount of engineering manpower on evaluation.
So did we two years ago when selecting a tool for automation of GUI testing. We thoroughly evaluated 15+ tools, including free and commercial solutions, and found a clear winner. If you have decided to automate your Java GUI testing procedures and looking for a GUI testing tool, our finding may save you some time and money.
QF-Test, a tool that works
You must hear about German quality. That’s it. Created by Quality First Software GmbH (QFS), Germany, QF-Test is a cross-platform tool for GUI test automation of Java and Web applications. It supports regression, stress, and load testing for both Swing/AWT and Eclipse/SWT, the latter is only on Windows and Linux, though. As for web testing, it works with IE and Firefox (for Windows and Linux) with specific support for various AJAX toolkits.
We’ve been using QF-Test for two years for our product, Excelsior JET, a compliant Java SE 6 technology implementation (JVM) with an ahead-of-time compiler and a deployment toolkit.
Excelsior JET comes with GUI tools written in Java. These applications are implemented with Swing and feature a custom look-n-feel specially designed for them. Behind the presentation layer, an advanced logic is implemented to enable the user to accomplish the necessary tasks, such as the creation of a complete installation package, with minimal efforts. That said, we have quite complex testing scenarios and employ the continuous integration approach including automated GUI testing with QF-Test. It’s proved to be a reliable, fully-featured and, at the same time, easy to use tool. And the tool does work as advertised.
How to enable QF-Test for Excelsior JET
This section is for users of Excelsior JET and it will be short.
On the first run, QF-Test prompts you to instrument the underlying JVM. Click Manual instrumentation and select directory
Done. Now everything will work even though the Java applications under test are compiled to Windows or Linux native code executables with Excelsior JET. Just in case, the instrumentation dialog is also available via the Extras->Manage JDK Instrumentation items of the main menu.
Two tips on code testability
Miracles happen but not in the software industry. Whatever test automation tool you go for, your code should be eligible for testing and you should pay attention to it in the course of development. Here go two examples from our practical experience with QF-Test.
First, when testing your Java GUI code you may find out that QF-Test
blames you for reports errors in the code that seems to work (or you just believed so ;)). The matter is that QF-Test is able to detect Swing misuses that lurk in your code and it would be better to fix them before they will bite your end users. Temporarily, you may disable reporting such errors so they will appear as warnings but do not forget to create respective issues in your bug tracker.
The second note is on automatic identification of GUI controls. Remember that aphorism – if it walks like a duck and quacks like a duck, I would call it a duck. If the GUI includes different controls of the same type, that have very similar content, QF-Test may fail to identify them unambiguously. Look, it’s just a Java GUI testing tool rather than an Artificial Intelligence system. To guarantee exact identification, you may simply assign a unique name for each GUI control using the setName() method and take it into account in further development.
On a root cause of quality software
I thought about why QF-Test is so polished and works flawlessly. Finally, I came up with a guess – probably, because it’s tested with QF-Test ;). It’s causa sui of sorts.
P.S. Prepare to be pleasantly surprised by the quality of QFS Customer Support too.