Tuesday, July 15, 2014

Part 2 Troubleshooting WebDriver StaleElementReferenceException

StaleElementReferenceException can be a misleading error and people can have a hard time filtering out the actual root cause. The StaleElementReferenceException means the element has been removed or changed in between the time it was referenced and the time it was used.

Let’s walk through some code that might result in a StaleElementReferenceException

  1) We create a variable initialized to a text box
var text1 = driver.FindElement(By.Id("text1"));
  2) Lets imagine some javascript removed it from DOM and then adding it back
  3) Finally the call to SendKeys() will throw "StaleElementReferenceException"
text1.SendKeys("StaleElementReferenceException");

I often see this error on SelectElement's
For example:
A form with dependent SelectElement's: a SelectElement for State that depending on it value populates a SelectElement for City.
When you populate the State SelectElement, the data is loaded into the City SelectElement by removing it from DOM and then adding it back. Lots of apps have these dependent SelectElement's
You can create a variable for the City SelectElement before populating the State but theres might be a race condition if you try t populate it right after populating the State.

Here is a link to the Selenium2 Webdriver documentation on StaleElementReferenceException.

Links to Troubleshooting:
Part 0 Troubleshooting WebDriver issues are easy and so can you!
Part 1 Troubleshooting WebDriver NoSuchElementException
Part 2 Troubleshooting WebDriver StaleElementReferenceException
Part 3 Troubleshooting WebDriver ElementNotVisibleException

Saturday, May 10, 2014

Part 0 Troubleshooting WebDriver issues are easy and so can you!

Seriously troubleshooting issues in WebDriver is easy, if you know what you're doing. Knowing what you're doing isn't easy; it requires a lot of learning and practice. I’m going to attempt to outline some basic steps to troubleshoot some common issues people encounter when using WebDriver.

At this point I'm planning to make 4 blogs on troubleshooting, in the first blog I want to make sure everyone understands at a high level of how the major components interact. The rest of the blogs will show common issues and how to troubleshoot the possible causes.

First, you need to understand that web sites and web browsers are built to give the user the illusion of full synchronized applications. What’s really happening? The “Server” is hosting a “Web Site”. The “Server” serves the “web Site” to a “Client”. The “Client” aka “Browser” renders that “Web Site”.  The communication between the “Client” and “Server” is over a stateless protocol called HTTP. This will be the cause of much confusion when you're working in UI Automation so a good understanding is required if you want to be proficient. Look if you don't understand this, you should before you move on!

Second, you need to understand the fact that WebDriver is an API that allows you to interact with the browser (remember the browser is just one side of that illusion of synchronization). As you might guess when you develop with an API that interacts with an illusion you're likely to have some issues. A good portion of the time when you have a tough issue it’s related to the connection of the following items:

  • Developer uses Tests & Code to interact with WebDriver
  • WebDriver uses native calls interact with Browser
  • Browser uses HTTP interact with Web Site

I have notice Developers who are learning or lazy can be quick to place blame WebDrivers interaction with the browser. More often than not I find the issues to be the test code or the web site code, but if you find a “real” issue with WebDriver, good on you… File a defect with Selenium WebDriver and give yourself a gold star.

Third, you need to keep learning and experimenting.

Last, you need to be able to model these interactions and concepts in your head when come across real tough problems because troubleshooting tough software issues is more than check 1, 2, 3, done. Troubleshooting tough software issues is problem solving and understanding complicated interactions when they have unexpected results, and that ain't easy. People don't naturally consider all possible outcomes of an action, especially if that outcome is unlikely. So when learning a thing it's good to understand it as a whole and just as it applies to your happy path.

Links to Troubleshooting:
Part 0 Troubleshooting WebDriver issues are easy and so can you!
Part 1 Troubleshooting WebDriver NoSuchElementException
Part 2 Troubleshooting WebDriver StaleElementReferenceException
Part 3 Troubleshooting WebDriver ElementNotVisibleException

Other possible exceptions WebDriverTimeoutException, UnhandledAlertException, WebDriverException, InvalidOperationException

Monday, May 5, 2014

Part 1 Troubleshooting WebDriver NoSuchElementException

NoSuchElementException seems obvious but it can be a misleading error and people can have a hard time filtering out the actual root cause. The NoSuchElementException means the specific element did not exist at the time the call was made plus the duration of the implicit wait.

Let’s say the following code results in a NoSuchElementException

driver.Manage().Timeouts().ImplicitlyWait(new TimeSpan(0, 0, 10)); //10 seconds
driver.Navigate().GoToUrl(url);
driver.FindElement(By.Id("text1"));


There are many reasons this might happen, lets walk over some of the common reasons:
There is no web element with an ID of "text1"
What to look for, the element may have been removed or the ID may have been removed or changed.
  • Element doesn't exist, navigate to the URL and view the source code
    • Verify that the web element exists
    • Verify that the ID = “text1”

The element did not load within 10 seconds
What to look for, the element may have been loaded after the 10 seconds
  • Never loads in less than 10 seconds, navigate to the URL and see how long it takes for the element to appear on the page    
    • Verify that it happens in less than 10 seconds, each time you go to a page the time could be different so doing this multiple times could help provide insight. My general rule is if it should take 4 seconds expect it to take 8 seconds and wait as much as 12 seconds. (Should take X,  Expected X*2, Wait X*3)
  • Intermittently loads in less than 10 seconds, Consider each Runtime Context separately, simple example: It works on my laptop but not on build machine   
    • Verify hardware, how long it takes on each machine: laptop 3 seconds, build machine 12 seconds. If this were true you might expect other operations have the same ratio.
    • Verify time of day and day of week, what time are the failures. If fails every Saturday 5:00am, what else is happening at that time, DB back up, system updates, virus scans, etc…

The element could be inside an iframe.
What to look for, the element might not be part of the parent html page it may be contained inside of an iframe.
  • The web element is present but not in the pages DOM
    • Verify if the web element is a dependent of an iframe
So these were just some of the things that could have caused this error, there are a host of other reasons the NoSuchElementException would be raised. I think the most important thing to remember is be flexible and not get locked into a specific way of thinking. Try to visualize what happened over the wire and in the browser. Sadly I have seen people lose hours and days without trying the steps above thats why I finally decided to blog it out.

Here is a link to Selenium documentation on the subject of NoSuchElementException Enjoy it I know I did!

Links to Troubleshooting:
Part 0 Troubleshooting WebDriver issues are easy and so can you!
Part 1 Troubleshooting WebDriver NoSuchElementException
Part 2 Troubleshooting WebDriver StaleElementReferenceException
Part 3 Troubleshooting WebDriver ElementNotVisibleException

Thursday, February 6, 2014

NUnit Test Data in CSV or Excel

     I'm always trying to find ways to build what I call "lightweight data driven tests" and I keep coming back to Excel or CSV as a way to accomplish this quickly.  Why? Because, from thought to execution you can quickly get a large number of tests completed while keeping them easy to maintain. Data driven testing is much like gold mining, some people spend a lot of money and never find the gold others get rich quick and everything between that   The example below will be in CSV in case you don't actually have Excel installed. But that won't matter because Excel and CSV are basically the same in code, Excel is just easier to work with.
     First off here is a sample test "TestCsvData" and a TestCaseSource "CSVDATA" that calls the "TestDataReader" class's "ReadCsvData" method to get its test data.
[TestCaseSource("CSVDATA")]
public void TestCsvData(string a, string b, string c)
{
    Assert.AreEqual(int.Parse(a) + int.Parse(b), int.Parse(c));
}

public IEnumerable<TestCaseData> CSVDATA
{
    get
    {
        List<TestCaseData> testCaseDataList = new TestDataReader().ReadCsvData(@"Data\CSVData.csv");
        if (testCaseDataList != null)
            foreach (TestCaseData testCaseData in testCaseDataList)
                yield return testCaseData;
    }
}

     Next we just need to make an OleDbConnection to the file and select the rows we want to use. For a CSV file the table is the file name and for a Excel file the spreadsheet is the table. The first row is treated as the column names, if you don't want to return all columns as test data then change the select statement .
class TestDataReader
{
    public List<TestCaseData> ReadCsvData(string csvFile, string cmdText = "SELECT * FROM [{0}]")
    {
        if (!File.Exists(csvFile))
            throw new Exception(string.Format("File name: {0}", csvFile), new FileNotFoundException());
        var file = Path.GetFileName(csvFile);
        var pathOnly = Path.GetFullPath(csvFile).Replace(file, "");
        var tableName = string.Format("{0}#{1}", Path.GetFileNameWithoutExtension(file), Path.GetExtension(file).Remove(0,1));

        cmdText = string.Format(cmdText, tableName);
        var connectionStr = string.Format("Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};Extended Properties=\"Text;HDR=Yes\"", pathOnly);
        var ret = new List<TestCaseData>();
        using (var connection = new OleDbConnection())
        {
            connection.ConnectionString = connectionStr;
            connection.Open();

            var cmd = connection.CreateCommand();
            cmd.CommandText = cmdText;
            var reader = cmd.ExecuteReader();

            if (reader == null)
                throw new Exception(string.Format("No data return from file, file name:{0}", csvFile));
            while (reader.Read())
            {
                var row = new List<string>();
                var feildCnt = reader.FieldCount;
                for (var i = 0; i < feildCnt; i++)
                    row.Add(reader.GetValue(i).ToString());
                ret.Add(new TestCaseData(row.ToArray()));
            }
        }
        return ret;
    }

    public List<TestCaseData> ReadExcelData(string excelFile, string cmdText = "SELECT * FROM [Sheet1$]")
    {
        if (!File.Exists(excelFile))
            throw new Exception(string.Format("File name: {0}", excelFile), new FileNotFoundException());
        string connectionStr = string.Format("Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};Extended Properties=\"Excel 12.0 Xml;HDR=YES\";", excelFile);
        var ret = new List<TestCaseData>();
        using (var connection = new OleDbConnection(connectionStr))
        {
            connection.Open();
            var command = new OleDbCommand(cmdText, connection);
            var reader = command.ExecuteReader();
            if (reader == null)
                throw new Exception(string.Format("No data return from file, file name:{0}", excelFile));
            while (reader.Read())
            {
                var row = new List<string>();
                var feildCnt = reader.FieldCount;
                for (var i = 0; i < feildCnt; i++)
                    row.Add(reader.GetValue(i).ToString());
                ret.Add(new TestCaseData(row.ToArray()));
            }
        }
        return ret;
    }
}

So why Excel? Its just easier than anything else.... 

  • Excel was built for this type of data manipulation 
  • Excel has lots of functionality to help you get work done quick such as pattern copy
  • Using a file instead of DB makes the test suite more accessible to non-technical people
  • SQL will require a lot more time to develop
  • SQL will require a lot more time to maintain
  • CSV not very good readability or data entry 
Generic methods like these return all the data as string parameters so its up to the test method to correctly type the data.

In the past I have had a hard time convincing people of the benefits of testing this way, I'll do my best here to allay their concerns and yours.
Concern: Putting the test data in Excel or CSV will somehow make the data become irrelevant especially over time. I think they have the perception that having the data in a file external to the test class will cause a data issue or slow down the time to find and fix issues with the data.
Answer: False, the data will become irrelevant no matter where you store it, so focus on changing/fixing it as soon as it happens regardless of data location.
Concern: By putting the test data in Excel or CSV the data will lose visibility, meaning the PM's, managers or manual testers won't have visibility to the data.
Answer: False, we are talking about moving the data from source code to Excel or CSV file and it will most likely be in an adjacent folder. That means the data is going to be in the same source control as before so no access control will change. Furthermore, you might grant write permissions to manual testers on an excel file to add more test cases where you won't want them touching source code.
Concern: Shouldn't we put this in SQL, reasons: (reporting, visibility, easy data entry, etc)
Answer: False, if you put this data in SQL you need to build a data access layer and this will make changing the data more difficult and rigid. Also, if you don't already have one, using SQL now will add a new dependency on SQL.  Rule of thumb SQL is great for enterprise software but not at all lightweight.
Concern: Excel or CSV  will not scale to meet the data needs, or will have poor performance.
Answer: If you have this problem with your test data, you have other more important problems. I believe if you could actually find enough relevant tests to make Excel or CSV perform poorly enough to bother you then you hit a testing gold mind.