Sunday, September 22, 2013

Use ExpectedCondition over Thread.Sleep() in tests, Selenium 2 Webdriver C#

So the theory is you should never use a Thread.Sleep() in automated UI testing because it will be hard to maintain, prone to failure and perform poorly.
  • Whatever you set the value of the wait to, you might have to increase it in the future. On different hardware the "web site" runs slower, when the "web site" being tested changes the waits will need to change.
  • The smaller you set the wait the bigger the chances of future issues, 1,2,3 seconds waits are your worst enemy.
  • If you set the value very high (say 30sec) so you don't need to adjust it in the future you will waste time in execution. Example in 1200 tests it adds up to about one hour of wasted execution time, and you should aim to way more than 1200 tests.
Below are two code snippets that are functionally the same except for the way each waits for the next element.

The wrong way
In this snippet we are clicking the submit button
next we wait a static 1000 milliseconds using Thread.Sleep()
Finally we assert that ByLabel1Div is present by calling ElementExists
My1Page.SubmitButton.Click();
Threading.Thread.Sleep(1000);
Assert.AreEqual(true, Driver.ElementExists(My2Page.ByLabel1Div));

The right way
In this snippet we are clicking the submit button same as above
Finally we assert that ByLabel1Div is present, by calling WaitUntilExists witch return in 0-10 seconds if the element meets the ExpectedConditions of ElementExists
My1Page.SubmitButton.Click();
Assert.AreEqual(true, Driver.WaitUntilExists(My2Page.ByLabel1Div));

In the cases where ByLabel1Div is loaded  by Javascript and Ajax, the time to load the ByLabel1Div  can be affected by many things such as workload on web server, DB server and local machine as well as the network latency.

Selenium 2 Webdriver provides the following 4 ExpectedConditions for you to use (ElementExists, ElementIsVisible, TitleContains, TitleIs). In my open source project I have created additional ExpectedCondition's (ElementNotExists, ElementTextEquals, ElementTextNotEquals, ElementTextContains, ElementTextNotContains, ElementAttributeEquals, ElementAttributeNotEquals, ElementNotVisible)

Here is the magic behind the WaitUntilExists, It is basically a Extension method extending the IWebDriver interface. Its using a WebDriverWait with the ExpectedConditions ElementExists to return as soon as the condition is met.
public static bool WaitUntil(this IWebDriver iWebDriver, Func condition, int maxWaitTimeInSeconds = 10)
{
   var driver = (IWebDriver)iWebDriver;
   var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(maxWaitTimeInSeconds));
   try
   {
      wait.Until(condition);
   }
   catch (WebDriverTimeoutException)
   {
      return false;
   }
   return true;
}

public static bool WaitUntilExists(this IWebDriver iWebDriver, By locator, int maxWaitTimeInSeconds = 10)
{
   return WaitUntil(iWebDriver, ExpectedConditions.ElementExists(locator), maxWaitTimeInSeconds);
}
For more information here's the full file IWebDriverExtension.cs with more WaitUnils*.

In the snippets above you see Click, and then WaitUntilExists, or more generally stated as a pattern: perform action and then validate. This pattern (perform action and then validate) is one you'll want to use often especially for elements that are loaded by Javascript and Ajax.

Friday, September 13, 2013

Working with Selenium 2, WebDriver and .NET. Tips and tricks

This is a supplement to help a person learning Selenium 2, WebDriver and .NET. As a supplement it will be very helpful for you while you're learning and using Selenium to keep these things in mind, but it's not intended to teach you (Selenium 2, WebDriver and .NET), you can get that information elsewhere.

Get the source code
The source code is the best place to answer your questions.  If you're attempting to do Selenium 2 for the first time whether you have UI Automation experience or you're a first timer, youre going have a lot of questions. Understanding the code will not only help you answer specific questions but increase your overall understanding of the different Selenium projects and how best to use them. Also for reasons described below the online information won't always answer your questions.
Update often
Update the Selenium packages you are working with as frequently as you can. People are always making contributions to the Selenium projects and there are frequent updates available, read these change logs and download!  It's super easy to update your Selenium projects and usually it won't impact you tests in a negative way.
Read but don't believe what you see
A lot of the documentation is missing, incomplete, out of date or not in .NET on the Selenium site, try not to complain about it too much, it gets old and doesn't help you solve the problem. This problem is all over the web (including this blog) it's because Selenium is moving so fast (its a good thing!). You can solve the problem by submitting the correct answers to your favorite site. As a .NET SDET you will have a problem no Java SDET will have, that you'll be translating Java into C#, see a blog I wrote in response to this Basic problem.
Don't sleep!
Please don't sleep in your code its a false economy, bottom line if you have a Thread.Sleep(4000) in your code today Murphy's law says it will fail tomorrow. If you honestly think the only way the test will work is if you use a Sleep then you've missed the plot. What's the plot, use the pattern ExpectedCondition instead of Thread.Sleep(). For more information check out this blog article Use ExpectedCondition
Make sure your closing all the browser sessions.
Close all your browsers or call WebDriver.Quit() before you exit your code (think IDisposable). Don't abandon your RemoteWebDriver or other IWebDriver sessions! It will bring down your Selenium Grid, it will eat your hard drives, It will leak memory, and it will reduce the amount of sessions available. For details read this blog entry WebDriver.Quit() and also this StackOverflow question
Think like a web server
You should always be thinking like a user when doing testing because you are user #0, the first user! But you'll also need to think like a Web Server. When a user uses a web page it thinks it working with a synchronized application, not true. Underneath it all, all websites are using REST!  Client side javascript and AJAX are two ways the user experience is enhanced, and can give the illusion of a synchronise responsive application. Selenium will wait for the page to be loaded but it has no mechanism for telling when a random element was added to a page from javascript or AJAX except for the ExpectedConditions.  So many people get lost for hours in their own heads blaming Selenium when they simply missed the plot.
Page factory is good but it will limit you 
If you use a page factory it can be helpful, but keep in mind it will restrict you from being able to use your locators in out side of the page (which you might find very helpful) so its a trade off.
Party like it your birthday

Tuesday, August 20, 2013

Automate like a superhero

I can't help but wonder how would Batman write automation?

First off Batman would know the technology stack like the streets and buildings of Gotham. Batman spends hours studying structural, electrical, maintenance diagrams of the buildings, sewer, water and power systems. How does this translate into knowing the technology stack? Knowing the stack would allow him to pick the best solution from his utility belt for each problem that presented itself so he could write a script or design a program that would not only handle current needs but also leave him in a good position for his next move.

Second, his never ending utility belt and all his cool toys in the the Bat: car, plane, boat and bike. Batman doesn't just make 1 Batarang, he has a multitude of different Batarangs and more than one of each. A sharp Batarang for cutting power lines, an explosive Batarang for taking out support columns, and so on. This means the more tools, Frameworks, API's, languages and patterns you can learn can learn the better your Batarangs are going to be.  Batman did automation he would work smart not hard.

Third Batman is not going to take orders from Superman, if anything he's going to do the opposite of what Superman thinks is a good idea. This is because Superman is a typical hammer and as a hammer all he see's is nails. You wouldn't expect a Web UI developer to drop everything and start doing client applications without a problem. Thats because they're different disciplines. Sure there are some similarities, they both fight crime, but thats about it! Superman can't ask Batman to use super strength to get out of problems and Batman can't ask Superman to think his way out of problems. The Web UI developer and the client application developer can share ideas and metaphors but without knowledge and experience one can not design or architect for the other  unless they want to be driving screws with a hammers.

So after reading all this you're thinking why would Batman do automation, after all Billionaire Bruce Wayne could easily buy every outsourced manual QA team in every developing nation. But you forget Batman refuses to use manual QA because his parents were killed in an alley by manual Tester when he was just 8 years old. It was this event more than any other drove him to become the Batman.

Sunday, June 23, 2013

Best use of webDriver.Quit() Close() Dispose() or it will eat your hard drive


Recently I found that some of VM's I share with coworkers were running out of hard drive space, I was surprised to find that the reason was webDriver.Quit() wasn't being called. My cohorts were running code or NUnit tests in the debugger and forcing the code execution to stop. Stopping execution this way was ensuring that [TearDown] and [TestFixtureTearDown] weren't being called, which is where we had placed the Quit() call. To put this in perspective one of my cohorts had over 18 Gb of data in his appData folder because of this.

What to do when done with the WebDriver
It's important to make sure to free up the driver at the end of the run by calling the Quit() method. Calling the Quit method will clean up temporary files created and in the case of RemoteDriver it will also close the session on the Selenium Server.

I Did some Googleing and didn't really find an answer that I thought I could trust, so I turned to the source code and found the following:

webDriver.Close() - Close the browser window that the driver has focus of
webDriver.Quit() - Calls dispose
webDriver.Dispose() Closes all browser windows and safely ends the session

In summary ensure that Quit() or Dispose() is called before exiting the program, and don't use the Close() method unless you're sure of what you're doing. I found this unanswered question on StackOverflow

What happens if you don't Call WebDriver.Quit() before exiting the program
The result will be that files are abandoned in your appData folder. And in the case of the remote driver the Selenium Server will not have ended the session properly, causing a memory leak in the Selenium Server.

I wouldn't have guessed this to be a problem for several reasons, first I would have thought those files would be cleaned up when doing a disk clean up. That assumption was incorrect even though these files are being written to a temp folder its one that Disk cleanup doesn't work on. Second as I rarely hit the "Stop" button in the debugger and therefore my appData folders aren't blowing up like the housing market. The reason I don't like to stop execution like that is when you let execution continue it ensures we get a NUnit log and all teardowns take place, in our case that means un-managed objects are cleaned up screenshots and logs are written. Having these logs and screenshots make it easier to troubleshoot and do investigation.

Monday, April 22, 2013

I'm confused, How do I get the ABV of mybrew

I thought I could make a quick little post with a simple accurate formula for getting a brews ABV so I can just plug in my numbers and not have to do math. But I cant seem to get a simple accurate formula, I have found multiple formulas. The question becomes witch to use.

The simplest formula I have found is ABV = (og – fg) * 131 but from what I have read  isn't the 100% accurate. Below I'll outline all the formulas I found and apply them to the brew I just bottled.

My Brew is a Belgian farmhouse Ale with a OG: 1.072 and a FG: 1.013 the results range from 7.73% to 8.15 % alcohol, I guess a difference of  0.4% is a lot.


1) According to BrewersFriend this is the two most popular formulas
Basic Formula: = (og – fg) * 131.25
Alternate Formula: = (76.08 * (og-fg) / (1.775-og)) * (fg / 0.794)
2) Wikipedia has a couple formulas
Standard Formula: = (1.05/0.79) * ( (og – fg)/ fg) * 100
Basic Formula: = (og – fg) * 131
3) Homebrew Stackexchange has a a bunch of formulas listed in this question
Formual: = ((76.08 * (og - fg) / (1.775 - og))
Basic Formula 1: = ((og - fg)/.75) * 100
Basic Formula 2: = (og - fg)*131
4) Beer Advocate has similar formula on one of it post, this is really just a place holder for a bunch of different sites, everyone seems to use this formula.
Formula: = (og – fg) * 131

Friday, March 1, 2013

Configure Network Adapter from command line

Here is another useful DOS command  to add to the list, NETSH. If you want configure a network adapter this is the great way to automate it.

To set the address of a Network Adapter you use the set command like this:
NETSH interface ip set address name="Local Area Connection" static 192.168.0.101 255.255.255.0 192.168.0.1 1

Now lets rename it so we don't have to do so much typing.
NETSH interface set interface name="Local Area Connection 2" newname="Looper"

Let say you want to do something fun like add multiple IP's to the loopback adapter, for that you use the add command
NETSH interface ip add address name="Looper" static 192.168.0.102 255.255.255.0
NETSH interface ip add address name="Looper" static 192.168.0.103 255.255.255.0



Monday, January 14, 2013

Telerik, not playing nice with Selenium because it was built on unicorn farts

Everyone knows that if you're using Telerik you're going to have a bad time. Also it's not easy to test with using Selenium.
According to Daniel Levy in a reply to a forum question titled "Automation of Telerik using Selenium",  if your having problems with Selenium, you should use Telerik's Testing software witch I believe was built on unicorn farts. I personally will be pushing my company to stop using Telerik controls as it will increase the overall test-ability of our application.

I did find this forum post from Martin on getting their calender control to work in Selenium  In the post he said "Note that by design RadInput controls set their value after they are blurred." So by that logic one would need to raise the onblur event once done manipulating the control and before the next action. If that doesn't work I recommend directly calling the java script although this can be brittle when the underling java script changes.

The best solution for me, so far, has been to file a defect against the application your testing stating that automated testing of this application is blocked due due to Telerik control. If you do this every time your blocked, It will force whom ever dose the budget for your group get evaluate witch provides more value Automated Testing or Telerik. Regardless of the issue smart people always chose to invest more development time to make an application more testable. Actually I think you can drop the smart and it still applies.

Solutions:

  1. Remove the  Telerik control from the application (best solution)
  2. Add a test Hook in the application mimic what the control dose (worst  solution)
  3. Don't test that specific feature (the cheapest solution)