Sunday, September 22, 2013

Use ExpectedCondition over Thread.Sleep() in tests, Selenium 2 Webdriver C#

So the theory is you should never use a Thread.Sleep() in automated UI testing because it will be hard to maintain, prone to failure and perform poorly.
  • Whatever you set the value of the wait to, you might have to increase it in the future. On different hardware the "web site" runs slower, when the "web site" being tested changes the waits will need to change.
  • The smaller you set the wait the bigger the chances of future issues, 1,2,3 seconds waits are your worst enemy.
  • If you set the value very high (say 30sec) so you don't need to adjust it in the future you will waste time in execution. Example in 1200 tests it adds up to about one hour of wasted execution time, and you should aim to way more than 1200 tests.
Below are two code snippets that are functionally the same except for the way each waits for the next element.

The wrong way
In this snippet we are clicking the submit button
next we wait a static 1000 milliseconds using Thread.Sleep()
Finally we assert that ByLabel1Div is present by calling ElementExists
My1Page.SubmitButton.Click();
Threading.Thread.Sleep(1000);
Assert.AreEqual(true, Driver.ElementExists(My2Page.ByLabel1Div));

The right way
In this snippet we are clicking the submit button same as above
Finally we assert that ByLabel1Div is present, by calling WaitUntilExists witch return in 0-10 seconds if the element meets the ExpectedConditions of ElementExists
My1Page.SubmitButton.Click();
Assert.AreEqual(true, Driver.WaitUntilExists(My2Page.ByLabel1Div));

In the cases where ByLabel1Div is loaded  by Javascript and Ajax, the time to load the ByLabel1Div  can be affected by many things such as workload on web server, DB server and local machine as well as the network latency.

Selenium 2 Webdriver provides the following 4 ExpectedConditions for you to use (ElementExists, ElementIsVisible, TitleContains, TitleIs). In my open source project I have created additional ExpectedCondition's (ElementNotExists, ElementTextEquals, ElementTextNotEquals, ElementTextContains, ElementTextNotContains, ElementAttributeEquals, ElementAttributeNotEquals, ElementNotVisible)

Here is the magic behind the WaitUntilExists, It is basically a Extension method extending the IWebDriver interface. Its using a WebDriverWait with the ExpectedConditions ElementExists to return as soon as the condition is met.
public static bool WaitUntil(this IWebDriver iWebDriver, Func condition, int maxWaitTimeInSeconds = 10)
{
   var driver = (IWebDriver)iWebDriver;
   var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(maxWaitTimeInSeconds));
   try
   {
      wait.Until(condition);
   }
   catch (WebDriverTimeoutException)
   {
      return false;
   }
   return true;
}

public static bool WaitUntilExists(this IWebDriver iWebDriver, By locator, int maxWaitTimeInSeconds = 10)
{
   return WaitUntil(iWebDriver, ExpectedConditions.ElementExists(locator), maxWaitTimeInSeconds);
}
For more information here's the full file IWebDriverExtension.cs with more WaitUnils*.

In the snippets above you see Click, and then WaitUntilExists, or more generally stated as a pattern: perform action and then validate. This pattern (perform action and then validate) is one you'll want to use often especially for elements that are loaded by Javascript and Ajax.

Friday, September 13, 2013

Working with Selenium 2, WebDriver and .NET. Tips and tricks

This is a supplement to help a person learning Selenium 2, WebDriver and .NET. As a supplement it will be very helpful for you while you're learning and using Selenium to keep these things in mind, but it's not intended to teach you (Selenium 2, WebDriver and .NET), you can get that information elsewhere.

Get the source code
The source code is the best place to answer your questions.  If you're attempting to do Selenium 2 for the first time whether you have UI Automation experience or you're a first timer, youre going have a lot of questions. Understanding the code will not only help you answer specific questions but increase your overall understanding of the different Selenium projects and how best to use them. Also for reasons described below the online information won't always answer your questions.
Update often
Update the Selenium packages you are working with as frequently as you can. People are always making contributions to the Selenium projects and there are frequent updates available, read these change logs and download!  It's super easy to update your Selenium projects and usually it won't impact you tests in a negative way.
Read but don't believe what you see
A lot of the documentation is missing, incomplete, out of date or not in .NET on the Selenium site, try not to complain about it too much, it gets old and doesn't help you solve the problem. This problem is all over the web (including this blog) it's because Selenium is moving so fast (its a good thing!). You can solve the problem by submitting the correct answers to your favorite site. As a .NET SDET you will have a problem no Java SDET will have, that you'll be translating Java into C#, see a blog I wrote in response to this Basic problem.
Don't sleep!
Please don't sleep in your code its a false economy, bottom line if you have a Thread.Sleep(4000) in your code today Murphy's law says it will fail tomorrow. If you honestly think the only way the test will work is if you use a Sleep then you've missed the plot. What's the plot, use the pattern ExpectedCondition instead of Thread.Sleep(). For more information check out this blog article Use ExpectedCondition
Make sure your closing all the browser sessions.
Close all your browsers or call WebDriver.Quit() before you exit your code (think IDisposable). Don't abandon your RemoteWebDriver or other IWebDriver sessions! It will bring down your Selenium Grid, it will eat your hard drives, It will leak memory, and it will reduce the amount of sessions available. For details read this blog entry WebDriver.Quit() and also this StackOverflow question
Think like a web server
You should always be thinking like a user when doing testing because you are user #0, the first user! But you'll also need to think like a Web Server. When a user uses a web page it thinks it working with a synchronized application, not true. Underneath it all, all websites are using REST!  Client side javascript and AJAX are two ways the user experience is enhanced, and can give the illusion of a synchronise responsive application. Selenium will wait for the page to be loaded but it has no mechanism for telling when a random element was added to a page from javascript or AJAX except for the ExpectedConditions.  So many people get lost for hours in their own heads blaming Selenium when they simply missed the plot.
Page factory is good but it will limit you 
If you use a page factory it can be helpful, but keep in mind it will restrict you from being able to use your locators in out side of the page (which you might find very helpful) so its a trade off.
Party like it your birthday