Friday, October 21, 2011

Monkey Patching Firefox Race Conditions For Watir-WebDriver/Cucumber

If your team is like my team, you're trying to find ways to automate everything. Perhaps you're also using watir-webdriver to run browser based tests with Cucumber. This works great locally, but when you try to run your tests on a CI environment you start to run into some very strange problems with Firefox testing.  You start to get errors around not being able to find the h1 tag on your page.  You look at the screen shot you've generated using a method similar to this, and you find that your screenshot shows the text that contains the h1.  In fact, your error dumped html file also shows the h1 tag:

You've run into selenium issue 2099.  It appears what happens with this error is that the test is executing before the page has fully loaded, causing it to fail.  Old school race conditions.  Nice.

So how do we get around this?  There's been a few postings on how to handle this with Capybara, but what about tools that use selenium-webdriver more directly?  Here's a monkey patch that hacks over the Ruby bindings to enable tools like watir-webdriver to retry.  This patch will retry the find until selenium-webdriver receives an error that is not related to the strange InvalidSelectorError that it gets when the Firefox isn't done loading the page. I just stuff this file into the support directory in a Cucumber suite.

Since this is hacky and a monkey patch, I'm very keen on ideas on how to improve this implementation, since my Ruby skills are very raw.

Thursday, September 22, 2011

Integrating jasmine-maven-plugin With Your Legacy Enterprisey App

Let's say you're working on enhancing an old legacy app with javascript. Doesn't really matter what sort of app, just legacy, with an old source tree and an old deployment style. How can you integrate jasmine within that app and have it also integrate with your new fangled CI environment? What if you've also been asked to use maven since it's "The way forward" for your organization? How can you accomplish this?

Well, there's the jasmine-maven-plugin, which is all sorts of awesome, but because you have a legacy file structure there's no clear way on how to hook it into your legacy app. But when there's a will there's a way.

There are several problems that need to be tackled here. Let's go over them bit by bit.

First, how do we get jasmine-maven-plugin to see our legacy app?

Let's hope that you can create your project somewhere "near" the legacy app in the SCM system. In this example, I'm going to assume that we have a tree structure that looks like this:

Pretty simple, we'll put our specs inside jasmine-tests\src\test\javascript. We'll put any jasmine required javascript inside jasmine-tests\src\test\javascript\lib. We just need to figure out how to tell jasmine-maven-plugin to look at our legacy app for our scripts. Fortunately, it's configurable. Let's look at a snippet of the pom.xml that shows how you would configure the plugin.

In this configuration, we're telling the jasmine-maven-plugin to look outside of our project structure for our javascript files. It even preloads our new fangled jquery thing that we've introduced into our legacy application from our legacy.js.src property. The key to this is that we've kept the configuration of the plugin outside of our execution configuration. When jasmine-maven-plugin runs, maven will actually fork a process to generate the manual runner. It will not receive the configuration settings if you're using an older maven like 2.0.11. I do not know if 2.2.1 or 3.x change this. By placing the configuration at the top level of the plugin, we fix that concern.

At this point, we actually have a plugin that can execute on our system if we've checked out our entire source tree. It should also execute in your CI environment.

However, some teams, like to check things out in different directory structures than what's in the SCM tree due to their tools (like Eclipse). Although your team is excited, they've also noticed that the ManualTestRunner is deprecated. When they try to run jasmine:bdd, they find that nothing gets picked up from the legacy app because the jetty runs from the root of your jasmine-tests directory. It can't see things underneath it. Normally you'd create a symbolic link,'re on Windows. Windows XP. Remember, I said Enterprisey app, not just legacy.

How can you fix the process so you can run jasmine:bdd like good sensible developers? Glad you asked. Maven profiles to the rescue. And, a neat plugin that lets you build symbolic links on crazy things like Windows XP.

Let's look at the profile:

The maven-junction-plugin actually builds symbolic links in both unix and windows. We redefine the location of our legacy.js.src to be a temporary directory inside the jasmine-tests project. The checkedout.legacy.js.src points to where our application has been checked out locally. The plugin is executed in the initialize phase so it occurs before jasmine. And initialize is supposed to setup directory structures, and the like, so it makes sense.

Since we've redefined the legacy.js.src, jasmine:bdd will simply see our files as if they are inside the project, and the jetty will be able to serve them properly.

Our final pom looks like this:

Hopefully this is helpful for you.

Monday, August 01, 2011

The Problem with PageObjects, why PageParts are needed

Over the past year or so, several big dogs in the automated web testing community have been pushing hard with the concept of PageObjects. This is a great thing. As such we now have at least two different page object frameworks to use with Watir-Webdriver thanks to the efforts of Alister Scott and Cheezy.

However, the page object frameworks have placed an item on the backburner. I'm not sure why it is, but the concept of repeating items inside of a page has been ignored at this point. Let's say you have a website like Google Finance. The list of news items is a great example of a what I'm calling a PagePart.

PageParts are items that repeat themselves on a page in terms of content types. They repeat with different content depending upon the user who is logged in, but they all look the same semantically. Here's what a Google Finance News item looks like:

There are four of these items wrapped in a div with market-news-stream as it's id.

Without PageParts you might do this:

So how do we identify these items in the tests? When I first started tackling this sort of problem on my current project I had pass through methods inside my PageObject that went to a simple PagePart. This worked, but didn't give me the cool features of the PageObject frameworks, and wasn't that the whole point of adopting the framework?

Wow, this is more complex than not using page parts.

So, how exactly is this better? It's better when you convert your page parts to use the PageObject framework. Caveat, I do not know how you would do this in Cheezy's PageObjects gem, but it works just fine in the watir-page-helper gem:

Every PagePart extends the BasePart. You'll select your container div in your PagePart object using the part_div method. Obviously this could be any HTML element, but it's most likely to be a div. Since the BasePart will redefine @browser with the div, we'll be able to use the cool shorthand, rather than define every method we might need:

The example may seem a bit contrived, but, maybe you're not working with an interface as clean as Google Finance. If you are, you might get some benefit out of using PageParts.

Wednesday, April 13, 2011

Making Your Cukes More Awesomer With Screenshots

In my last post about Cucumber, I alluded to using watir-webdriver instead of using native watir. There was one very surprising advantage I discovered with watir-webdriver's use of selenium as a backend: Awesome Screenshot API.

Here's how we setup our Cucumber suite to take screenshots (and html/txt "screenshots") using watir-webdriver and celerity.  First, set your tests up to write an error when a test fails:

In this hook, we're passing in an environment, the scenario and the browser object we setup in the prior post.  The environment is really handy since QA folks might be running tests against different regions and would want to pass that information back to the developers to recreate problems.  The TEST_SERVER environment variable is already used to pick this during test startup.

The ErrorWriter module is next.

Several things are going on here.  The write errors message is simple enough.  It creates a directory for the errors to be written, and dumps the context of the browser's text and html representations to file.  What's more interesting are the scenario_name and file_name methods.

The scenario_name method does some interesting things.  We want to name our files by the name of the scenario that fails.  In the else clause, you see how dashes and white space are replaced by a single white space character.  But, in the if clause, we're testing to see if we have an ExampleRow.  These are the Examples in a Cucumber Scenario Outline.  If the scenario is an ExampleRow, we lose the name of the Scenario itself.  Fortunately, the ExampleRow knows what ScenarioOutline it belongs to.  We can get the name of the ScenarioOutline, perform the same transform on the name, and then append the example that is actually running to it's name.

The file_name method simply concatenates the region, time of failure and the scenario name together for the filename to be written to.

This can result in some very long file names, however, their descriptiveness wins overall.  You don't have to open a file to find out what failed.  A simple reading of the directory containing the files lets anyone find the test that they need to examine quickly.

The write_errors method did one other thing. Call capture_screenshot to grab an image of the browser.  When using Watir, you had to do some fancy things with the Win32 API to make the capture work.  We had a lot of issues using it with our QA folks who had SnagIT installed.  So how refreshing it was to have watir-webdriver just work with so little code:

We have another little trick, where in our prior example, we setup a WEBDRIVER variable to determine if we could take a screenshot.  If we're running on Celerity, no harm, no foul, but no screenshot.

Granted, it's really selenium that we're using here, but we get really nice and small png files, which only contain the contents of the browser.  No title bar, no toolbars, nothing. Just the contents of the window.  Our only issue is that IE causes problems on our CI box, but just having Firefox able to run there is an outstanding step forward.

Hopefully this is helpful, please let me know if it is.

Wednesday, April 06, 2011

Cucumber with Celerity and Watir-Webdriver

Earlier this year, I had the problem of trying to enable our QA staff to run our current suite of Celerity driven Cucumber tests through the actual browsers that we were using.  One obvious solution is to use various flavors of Watir.

The main concern with this solution is that Waitr required native ruby instead of jruby.  Not a huge deal, but still an extra annoyance for folks in a java shop.  Requiring QA to switch between jruby and watir wasn't ideal.  Additionally, taking snapshots of the screen during a failed test was far from trivial.

So instead, I embarked on a quest to use watir-webdriver and Celerity together.  The advantage here is that watir-webdriver, through selenium-webdriver, can talk to browsers and runs on jruby.  This combination was alluded to on several blog sites that I had found, but I never quite found something that put it all together for me.   Eventually, i got it working. Hopefully this will at least get you about 90% of the way there.

The first step would be adding watir-webdriver to your Gemfile:

Following that we create a browser.rb file that determines what sort of browser you are running.  I placed this in the support folder in the Cucumber project.  One thing to remember about this, is that you could also throw this method into your env.rb.

What's going on here? We're using the environment variable BROWSER to determine if we want to run something in a browser.  We then use another environment variable to determine which browser to use.  We even have an environment variable to determine what Firefox we want to use, which is a nice feature of selenium-webdriver.  At the end, Browser contains the class of browser we want to instantiate for tests, and BROWSER_TYPE contains the actual type of browser to instantiate for watir-webdriver.

One other thing to note, when we use Celerity, we're setting the offset for index based searches to 0.  Unlike watir-webdriver, Watir uses 1th based arrays.  Celerity, being a wrapper of HtmlUnit, had to translate those arrays from 1 to 0 so that HtmlUnit would run properly.  By setting this value, we can use 0th based arrays, just like watir-webdriver does in Celerity.

Next we setup our Cucumber Before block in hooks.rb:

In here, we simply instantiate the object.  In my case, we also needed to setup some URLs and turn off SSL for our Celerity testing.

The Rakefile is probably a bit more complex due to our desire to run both headless and browser tests on our CI box, but it is helpful, since it uses the :browser task to setup the BROWSER environment variable.

Finally, to help our QA folks, three batch files to execute the tests.

Hopefully you'll find this useful. Please let me know if you do.

Monday, March 28, 2011

My First Code Prison (Coding in the Clink III)

This weekend, I had the privilege of being part of a Code Retreat in Prison. Being that I haven't been to a Code Retreat...period, it might seem odd that I would choose The Clink as the venue for my first Code Retreat.

Along those lines, one of the main questions I got from folks on the outside is why?  For me, this was a fairly simpler question, but the answer is a bit more complex than I let on.

The first reason is, I knew Dan from my time working at CheckFree.  I still remember, and tell stories, of Dan and I being in the same HR orientation together. Dan asked some very good questions surrounding code ownership, and inventions, and who owned inventions made at on our own time at home.  The poor HR staff hadn't a clue how to answer Dan's questions.  I suspect now they do, since they were excellent points.

The second reason is I had the pleasure of working with Joel, while at Nationwide.  Joel became involved in working in the program on Tuesday nights at MCI. I found the stories of his experience fascinating and inspiring.  The fact that Dan was involved, made it all the more interesting.  The third reason, however, is where it gets more complex.

Growing up, I attended mass on a weekly basis. When you attend mass on a weekly basis you get to experience different style homilies from different priests. Each one has their own preffered style. Our pastor tended to simplify things such that a 5 year old could understand it. Other priests essentially took the guidance provided by the church and recited it back such that it.felt like a lecture. The best homilies, like most good programming conference talks, were filled with stories that made the subject matter relevant to your life today.

My favorite homilies contained stories of when the father visited inmates in prison. There was nothing more inspiring than these stories of redemption and understanding.  In fact, it was very obvious, that some of these inmates had achieved a better understanding and faith than I had as a questioning teenager.  I found myself wishing I could do something like that for something I truly understood.  Fast forward twenty years, and I have that opportunity.  I couldn't teach an inmate about faith, but I could bring a skill that I had cultivated over time and maybe do my part.  How could I pass this up?

So on Sunday morning I took the drive up to Marion Correctional Institution.  Myself, and seven other "outsiders" went into the prison and walked to the Life Line center.  One thing that became very apparent during the walk was that you were in prison.  You weren't in a place off to the side of the prison, a visitors area or anything like that.  You walked right past the kitchen and dining hall.  Which was a very good thing actually. Kairos was also there this weekend.  There is nothing that will calm your nerves than hearing a hundred or so men, throatily singing gospel songs as part of that program in the dining hall.

Once we made it to Life Line, we were greeted by the inmates who were very cordial and polite. We had short introductions and offered our skill levels in Java.  We then decided to have each outsider sit at a workstation and have the prisoners pair or triple with each of us.  I had the opportunity to triple with a beginner and a more intermediate to senior level inmate on our first effort on a Tennis Game Kata.

We made a few obvious mistakes.  Trying to cram an enum into the problem wasn't ideal and sidetracked us. We probably should have ping ponged a bit more in our pairing style, as what ended up happening was our beginner programmer drove while myself and the more senior inmate acted in a coaching role.  Not that there's anything wrong with this approach, but we didn't make it very far in our effort to solve the problem.  This, of course, is OK, since you're going to toss the code and start over for round 2 with different pairs.

We then had the opportunity to have lunch, which meant eating with the population in the dining hall.  With four people to a table, we intemingled and chatted over a brunch.  Brunch consisted of sausage, a hashbrown, an english muffin with jelly, some sort of bran flakes, scrambled eggs, a salad and a banana.  Seems like a lot of food until you realize, you don't get to snack on the inside.  One of the more interesting experiences was having a random inmate come and ask me if I wanted my banana.  Once I discovered that was fine, I gave it to him.  It's apparently quite common.

During lunch, we had an interesting discussion about programming and its utility on the outside of being able to earn a living.  This was a popular refrain.  Could you get a job in programming?  I think some of these guys had seen internal programs go away due to foreign competition and they wanted to know if there was a future in it.  Yes, the furniture building program had been halted since the state could get product cheaper overseas.  I think the choice of the prison to shift resources into technology is a very good move, it will help the inmates upon release regardless of programming's place.

What was also interesting was learning that some of the guys had already programmed in prison before. I heard a story of guys working for a client in Arizona.  They ran into issues, however, when the client's programmer had a complex of working with programmers in prison.  Sounds similar to folks having concerns with working with offshore developers, doesn't it? Probably the most interesting thing about this situation was that all communication about the code was done over telephone.  No email, no instant message, no internet.  Just telephone and books.  That's tough!

After lunch I had the opportunity to see a 90 second space battle video that an inmate rendered using Blender and some other graphics tools. It was darn impressive.  There was serious skill involved in making the video.

After the video we switched pairs, and I ended up with two fellows who I would classify as intermediate to advanced in programming skills.  We were able to solve the problem through rotational ping pong pairing.  Probably the most interesting thing about this was towards the end.  We believed we were done, until we read the final step of the kata which told us that the program should ignore all points after a game is won.  Writing tests for that feature drew up a bunch of tiny hidden bugs which gave me the opportunity to talk about self documenting code and extracting methods to handle those problems.  Which, one of my triple knew quite well...probably better than I did, but was a bit too modest.

One of the things that occurred to me after the second pairing was how skilled some of these programmers were.  I can honestly say that some of these guys had more skill than folks who I've been involved in interviews with on the outside.  I believe a lot of it has to do with rote memorization.  They have no Internet.  Only books, Eclipse and each other.  Any plugins and tools have to be brought in from the outside.  They don't have StackOverflow to ask questions.  Just each other, books, and whatever hints the IDE provides. So unlike most developers, who have adapted lazy instantiation of skills in order to adapt quickly to different technologies, these guys have had to battle harden the skills that they have. The other thing that surprised me, is how well they pair.  One thing you might not expect in a prison situation is the ability to share information, reduce your ego, and pass the keys back and forth.  Yet, they did it without concern.  Of the advanced programmers, I had the luxury of pairing with, I saw folks who would have no problem fitting in with an Agile team.  I know they could pass an audition, if they could get one on the outside.

So when I reflect on my experience, it makes me take a step back, and think about my own ego when I pair. Am I giving up the keyboard enough? Am I ping ponging enough?  Am I letting a junior developer work through their problems enough, or do they need to shoot themselves in the foot a few times first?  It was great reflection for me in handling my professional life.

One final note.  The guys in here have made mistakes. Some of these guys have made really big mistakes.  I didn't ask, but one of the inmates I paired with offered that he had been in the system for almost two decades.  You can draw your own conclusions from that information. He had serious questions on if learning Java could help him when he got to the outside.  I think it can, but as Dan put it at our retrospective dinner, the most important thing to learn is pairing and learning to work well with others. I believe the folks in this program were trying to do whatever they can to build skills that can help them later, be it on the inside or outside.

The most interesting thing about this experience, is how well it paralleled with the homilies given in my youth.  It became obvious to me the older I got, that the priest was trying to say that, these prisoners know more about faith that you understand due to their situation.  Similarly, these inmates knew more about programming than many folks I've encountered simply due to their faith that it's a skill that can help them later.  They practiced hard, and were better than they would ever care to admit.  Faith is something that isn't taught, but arrives, and it was inspiring to see.

I strongly encourage any programmer somewhere near Marion (any by near I mean 2.5 hour drive...we had folks from Ann Arbor show up), to try to make an effort to make it to a Code Retreat there.  If you aren't...maybe a prison near you has a programming program you can volunteer in.  It will be worth your time.