Month: January 2019

Yesterday I did the real demo for my GUI automation proof-of-concept project – the one for my own group.

I explained a little about Robot Framework, and gave an overview of how it works using my demo test case for our Next Generation product. This is what a test suite looks like, and this is how you set variables, and this is how you make your own keywords. That product’s GUI is not very automation-friendly, but see, the tool can cope with that stuff. Plus it may not have to, or at least not as much, as we have a commitment from the GUI Dev manager to have his people make changes for us as we target the parts of that GUI that most need automation. Then I ran it, and of course seeing the browser with the “controlled by automated test software” banner executing the steps of the test was a crowd-pleaser.

Then I switched to the other demo I’d made, which targets the current product. The current product doesn’t have much GUI automation right now because it isn’t needed, because most of its GUI is a very thin layer over the interfaces we use for command line automation. But the GUI itself mostly is automation-friendly, so the sample Robot test case for it just needs obvious-looking SeleniumLibrary keywords with obvious-looking locator values. No xpath expressions, no Execute Javascript with weird incantations. It looks easy … because it is. And it runs on the product we’re all familiar with … that product may not need GUI automation, but there are things it would be nice to have automation for.

Then I switched back to the secondary demo test case for the nextgen product. That one demonstrates the data-driven test case style, where you execute the same test for multiple data sets. One data set is deliberately incorrect, so I could show the reporting and logging for a test failure. People liked that too.

We aren’t ready to start writing production test cases yet. The nextgen product isn’t yet ready for that, though I have a list of areas that will most need GUI automation. Also we need some infrastructure, like describing the directory structure for the Gitlab repo we’ll use for those tests, and writing coding guidelines, and establishing a Jenkins setup for test execution. Also I need to make a Robot custom library to call our command line test framework – I don’t know how soon we will need that, I am sure we will need it. But first, get acquainted.

Three hours after I’d finished my demo, someone had installed the tools, pulled the repo for the current product demo, and made and executed his own first test case, which sets configuration for a test he runs. Hey, no reason not to learn a new tool by making it do something useful to you personally.

GUI testing has never been my thing. I’ve done some – a page is missing this thing, and when I select that thing I can’t set all the options I should be able to, and this other display looks fine for a small amount of data but really terrible when I load my large data set. I’m not comfortable with GUI testing, and my understanding of web pages and the technology used to make them is pretty basic.

But I’ve just done a proof-of-concept project for GUI automation, and am now working on making it go live.

Back up a bit … my current product has a huge library of automated regression tests, almost all of them implemented via command line. There are a few automated GUI tests,and a Selenium Grid to run them on, but GUI automation mostly doesn’t make sense for this product. The GUI interface is relatively small and the GUI is mostly a very thin layer over the same interfaces used by the command line tests. Manual testing (both for new features and regression) for the GUI was a deliberate choice.

At present, I know enough about the current product to be useful doing regression testing for a release, but not enough to be able to efficiently do testing for a new feature. So while we aren’t regression testing a release, I’m assigned to the Next Generation product that is under development, and I’ve been doing proof of concept work.

One of the differences between the current and nextgen products is that nextgen has business logic in the GUI layer, so we’re going to have to write GUI automation. No one thinks the current product’s existing GUI automation is a good thing to build on, though it works well enough for its purpose. A while ago, I was given a copy of a proof of concept project that a colleague had done for a very early version of nextgen. It is Python, runs Selenium, and uses a homegrown framework that someone in another product group had made for use with the target platform, and I experimented with that for a bit.

Well … it actually performs a useful test. But it runs for maybe 40 minutes and then fails, then I restart it and it runs an hour then fails, then I restart it and it runs 15 minutes then fails. It is always the same failure at a different time and I have no clue how to debug the problem. (Nor does my colleague. I’m not dissing her, she did a great job with what she’d been given.) Writing more tests like it would be fine for the programmers among our group, but we aren’t all programmers. And a roll-your-own framework for GUI automation? For all I know that may be appropriate for the product this one was built for, but I don’t think it is needed for our product, and we certainly don’t need the maintenance overhead. I was pretty sure there’s an easier solution. (OK, I admit it – my initial reaction was “this looks way too much like work”, and then I thought about why I had that reaction.)

I remembered that during my job search I’d seen postings that specified experience with Robot Framework, Selenium, and Python, so I downloaded that stuff. Once I had everything set up in Pycharm, getting a trivial Robot test working was … trivial.

Here’s a quote from Joe Colantonio: “If you want to use Python for test automation, you can’t go wrong with the Robot Framework.“ https://techbeacon.com/9-top-open-source-testing-automation-frameworks-how-choose

I put that aside for a while, as there was a release to test, then came back to it and started work on the proof of concept test for the nextgen product. That was not trivial, as I discovered that a lot of the GUI is not automation-friendly. Unsurprising, as “must support GUI automation” has never been on the developers radar. But it was obvious from even a cursory reading of the Robot Framework SeleniumLibrary documentation that it expected to have to deal with unfriendly interfaces.

Here’s one of the selling points for using a mature open source tool. No matter what problem you’re having with it, someone else has already had the same problem, and an Internet search will turn up at least one discussion about how to solve it. If you’re working in an unfamiliar domain, as I am, it may take a while to figure out the correct search terms, but the information is out there somewhere. Robot has been open source since 2008, and was in use at Nokia starting 2005. (The Nokia origin gives me an irrational warm-and-fuzzy feeling that “this originated in *my* world, of course it can do what I need it to”.)

Between Internet searches and help from some of the local GUI developers, I found the correct incantations to deal with the unfriendly-to-automation GUI elements. I got my test working, and showed it off during the latest nextgen product “this is what we have working so far” demo session. It was very well received. When GUI automation is not on your radar, seeing your product’s GUI in a browser window with that “controlled by automated test software” banner up top is definitely cool. The bug I submitted for “please make this GUI more automation friendly” is now officially targeted for work as part of the product.

Plus, that demo uncovered the information that someone in my company who works on a different product has been using Robot for a while, for REST and command line tests rather than GUI. OK! Someone in-house I can talk to about building custom libraries. So far I haven’t needed anything except the SeleniumLibrary keywords, but I’m sure that at some point we are going to need to make Robot tests that interleave GUI and command line actions.

I think this will work for our group. Robot’s keyword-driven approach looks sort-of similar to the test case syntax used with our existing command line test framework, so it won’t spook the non-programmers among us. Our Devs won’t fix all the automation-unfriendly GUI elements any time soon, but we should quickly build up enough examples of how to cope that we won’t be slowed down much once we get past the initial ramp-up. I’m sure we’ll run into problems, but I’m also sure we’ll get past them.

I’ve prepared the detailed demo for my group, and I’m working on the “how to set up your environment” instructions. I have a list of what has priority for GUI automation for the nextgen product, though it is not clear when we will be able to start on that. The next step for me is automation infrastructure stuff, such as modifying the “launch test suite on Selenium Grid” functionality in our existing command line test framework so it can also run the Robot suites, and I’ll want to get Robot suites running via Jenkins as well.

Yes, I am feeling very pleased with myself.

I am also very happy to be working for a manager who gave me a chance to do something so far out of my previous experience.