UI Testing with Nightwatch.js – Page Objects

I have already written a little about UI testing with Nightwatch.js. This was a little while ago and nightwatch.js has changed a little since then. In v0.7.0 they changed their implementation of page objects and added enhanced support. Nightwatch is a great framework for writing UI tests, and it easy to pick up an write some basic tests. But, if you are going to be writing lots of tests which you are more than likely going to, to ensure your UI is working as planned then you should really be making use of page object within the Nightwatch framework. Page objects are going to save you a lot of time, and prevent duplicated test code. By using page objects you able to abstract away a lot of the HTML actions needed when you are accessing and manipulating your pages. Also by using page objects you can increase your test coverage as you reuse the objects in multiple tests.

There are two parts to the page objects in the Nightwatch framework. Elements and Sections. For me, you need to be using both combined to get the most from the page objects. This is where you will get the most of them. To highlight this I’ve put together a couple of examples to explain how elements and sections work. Following on from my first post: UI testing with Nightwatch.js. I will be using the test from that post, and making two more test files, one that uses elements, and another that uses elements and sections together.

Getting started with page objects

Before you can get using page objects you need to update your config file to tell Nightwatch where to look for your page object files. In your Nightwatch config file you need to update the property:  page_objects_path. Mine looks like: 

"page_objects_path": "page-objects",

I am putting all my page objects inside of one folder so I use a single value in my config. Nightwatch allows you to set the value of page_objects_path as an array, with each element in the array a folder path. As your UI tests grow it might be worth looking at splitting them up into multiple folders for better maintainability. Each page object should be within it’s own file. Nightwatch will read the contents of each folder and these will become your page objects you can use within your tests.

Using Page Objects in your tests

Once you have updated your config to pull in the page objects you need to reference them inside of your tests to be able to call the commands or using the elements. You can name your files how ever you like, I name mine all one word using camel case if needed. You can use hyphens, underscores or even spaces in the file names if you so wish. Depending on how you name your files will matter, as you will need to reference the page object differently inside of your test files. All your page objects are available to all your test on the function argument .page context. I stick with the argument being named browser as per the Nightwatch documentation, but you can use whatever you wish.

The reason I keep the filename to camelCase is because the browser.page is a JavaScript object of all your page object files and folders. I have a page object file named simpleLogin.js which I reference in my tests as:

var login = browser.page.simpleLogin();

If you where to use hyphens in your file name you would need to use the following syntax.

var login = browser.page['simple-login']();

You can make your page object folder have nested folders. This would result in the browser.page object have a key for the folder name which is an object containing the page object files within that folder, you could then call that object like:

var login = browser.page.folder['simple-login']();

Page Elements and Sections

Page elements, or elements as they are referred to within the code are a way to reduce to keep the selectors you use in your tests DRY (Don’t repeat yourself). Rather than having to write the same selector over and over again in you test file you can add it as an element. An element is a property of a page object that have a selector attached to them. You can then use the element name in your test to reference to the selector of the HTML element you want to test for. This allows us to remove the duplicated selector references from our test files and move them into a single file. By doing this we only have one location to change if at a later point in time you are refactoring you application and change the class or ID of the element you are using in your test, you just need to update the selector value in your object file and all the test’s using that element will get the new value.

From the tests in my previous post, switching the selector I was using into page elements gives me the following elements

elements: {
  username: {
    selector: '#username'
  },
  password: {
    selector: '#password'
  },
  submit: {
    selector: 'input[type=submit]'
  },
  error: {
    selector: '.error'
  }
}

If you look at the Nightwatch documentation for page elements you will see there are a few different ways to set up you elements object. I go with the name of the element then an object containing selector and value. As you can see from my example we have four elements, username, password, submit, and error. Each have a selector value that corresponded to an HTML element within our page. You can split your elements down into sections if your object is dealing with a lot elements, you could split your elements object into sections to allow for better maintainability. See the Nightwatch documentation for more on using element sections.

To show you how you could use page elements in a test, here’s our initial test written without using page objects:

'Login Page Initial Render': function(browser) {
  browser
    .init()
    .waitForElementVisible( 'body', 1000 )
    .verify.visible('#username')
    .verify.visible('#password')
    .verify.value( 'input[type=submit]', 'Log In' )
    .verify.elementNotPresent('.error')
    .end()
}

Here’s the same test written using a page object that contains just page elements:

'Login Page Initial Render': function(browser) {
  var login = browser.page.simpleLogin();

  login.navigate()
    .waitForElementVisible( 'body', 1000 )
    .verify.visible('@username')
    .verify.visible('@password')
    .verify.value( '@submit', 'Log In' )
    .verify.elementNotPresent('@error')

  browser.end();
}

As you can see it’s not any shorter in length, possibly longer if anything. First off you need to setup the page object you wish to use in your test, you do this by setting a variable with the value being the page object. Then you start your test chain using this new variable, you can then make use of the elements from the page object by using the name of the element prefixed with an @, as you can see I’ve replaced #username with @username.

Page Commands

Page Commands, or commands called by the Nightwatch documentation is what makes using page objects all the worth while. By using commands you can make your test files really DRY, and move the bulk of your test logic into the page objects allowing future developers or tests who need to contribute to your test to reuse blocks of test’s without having to reinvent the wheel.

Commands are functions that contain logic for reuse in your tests, if you find yourself writing the same assertions or verify statements over and over, you should look at using a command. A really common example would be having to click submit on a form, rather then each test having the same code to click and verify something to do with the form submission you can wrap this up in a command and call the command inside of your test. Commands will still output to the terminal or your report for assertions and verify statements, there is no drastic difference between using a command verses having the test code within the test file. Using commands inside of your page objects make things more maintainable, and helps others who may need to write tests that touch parts of the page you have worked on.

If we take our first test, that’s not using pages objects and show you how we could write it using a page object that makes use of elements and commands

'Login Page Initial Render': function(browser) {
  browser
    .init()
    .waitForElementVisible( 'body', 1000 )
    .verify.visible('#username')
    .verify.visible('#password')
    .verify.value( 'input[type=submit]', 'Log In' )
    .verify.elementNotPresent('.error')
    .end()
}

Here’s the same test written to use page objects with elements and commands:

'Login Page Initial Render': function(browser) {
  var login = browser.page.commandsLogin();

  login.navigate()
    .validateForm()

  browser.end();
}

Sure makes our test a lot smaller, that’s because we have moved the logic into a page object command named “validateForm()”, which now contains the test logic for validating the form is present, inside of the page object we have the following

var loginCommands = {
  validateForm: function() {
    return this.waitForElementVisible('body', 1000)
      .verify.visible('@username')
      .verify.visible('@password')
      .verify.value('@submit', 'Log In')
      .verify.elementNotPresent('@error')
  }
};

module.exports = {
  commands: [loginCommands],
    url: function() { 
      return this.api.launchUrl; 
    },
  elements: {
    username: {
      selector: '#username'
    },
    password: {
      selector: '#password'
    },
    submit: {
      selector: 'input[type=submit]'
    },
    error: {
      selector: '.error'
    }
  }
};

As you can see, we have page elements and commands combined in our page object file for our login page. Overall a bit more code than our very first test, but this is a simple example, if you look at our other tests which is all on my GitHub, you will be able to see how I have used page objects that combine both elements and commands.

Take a look around the updated nightwatch-demo repository to see the other test’s and how I’ve switched them over to use page objects. You can even clone it or download the repository, I’ve updated it to contain all it’s required dependencies, and all runs from an npm install (tested on a Mac and Travis-CI only)

UI Testing with nightwatch.js

Update 26 December 2016: The code in the repo https://github.com/matthewroach/nightwatch-demo, has been updated to run from a clone/download. All dependencies are part of the repository or part of the npm install.

Nightwatch.js is an easy to use Node.js based End-to-End (E2E) testing solution for browser based apps and websites. It uses the powerful Selenium WebDriver API to perform commands and assertions on DOM elements.

Testing your website as a user may interact with it may not be your first thought when talking about testing code. We can unit test our code to ensure it’s doing as it’s supposed to but the more code we apply to our UI the more dependency it will have from the UI and the code powering it.

Most of the code and web apps I create are driven from a server side language with a front end interface, and then JavaScript applied on top to add a nice user experience.

With UI testing or integration testing we can test all the parts of the web application are working as intended. Ever upgraded a third party library or decided to adjust a little bit of code on a single page to later get a bug report from someone that another page in the app is now not working? YES, unit testing should of caught these issues, one of the added benefits from using a UI testing tool is you can be 100% sure that the integration from the code and tests you have written work at the final stage of delivery.

Installing nightwatch.js

If you have node and npm installed on your machine, it’s pretty simply to get nightwatch set up, you can install it local to your application or as a global module (-g). I have installed mine globally. Just run the following from your command line (you may need to do this as an admin or sudo on a Mac):

npm install nightwatch -g

Once you have it installed you can confirm you have it install by checking the version

nightwatch -v
nightwatch v0.6.8

Setting up

The code and test’s are available on Github if you wish to look: https://github.com/matthewroach/nightwatch-demo

When using nightwatch.js you will need a nightwatch.json file, this is the config file that contains a lot of different options, for this post I’ll not cover them all, just the one’s I’ve used, changed or added. I have this file located at the route of my project, you can place it anywhere under your project but you’ll need to update config items to point to the relevant locations of your project.

I am using browserstack’s automate service to run my tests with. You don’t have to use browser stack to run these test’s you can use selenium to run them locally. Using selenium takes a bit more set up to get running, so for the purpose of this post we will use browser stack.

Below is the nightwatch.json file I have located at the root of my project. The src_folders config points to my tests folder that’s also located on the root of the project. You can change this to point to anywhere you like.

You will also notice some references to hub.browserstack.com, this tell’s nightwatch to use browser stack as our selenium running. The two items you need to update to use this is the browserstack.user and browserstack.key. You will find these in your browser stack account.

{
  "src_folders": ["tests"],
  "output_folder": "reports",
  "custom_commands_path": "",
  "custom_assertions_path": "",
  "page_objects_path": "",
  "globals_path": "",

  "selenium": {
    "start_process": false,
    "server_path": "",
    "log_path": "",
    "host": "hub.browserstack.com",
    "port": 80,
    "cli_args": {
      "webdriver.chrome.driver": "",
      "webdriver.ie.driver": ""
    }
  },

  "test_settings": {
    "default": {
      "launch_url": "http://hub.browserstack.com",
      "selenium_port" : 80,
      "selenium_host" : "hub.browserstack.com",
      "silent": true,
      "screenshots": {
        "enabled": false,
        "path": ""
      },
      "desiredCapabilities": {
        "browserName": "chrome",
        "javascriptEnabled": true,
        "acceptSslCerts": true,
        "browserstack.user": "",
        "browserstack.key": ""
      }
    }
  }
}

Tests

Test’s are written in JavaScript, I separate out my tests into a tests folder to keep separation from the app code. On bigger projects I’ve added extra folders with the tests folder to mimic the web app structure, so you are able to run selected areas on their own.

Each test file can have one or multiple test’s. Each test is a JavaScript function. A sample test is shown below:

module.exports = {
  'Login Page Initial Render': function( _browser ) {
    _browser
    .url('http://dev.matthewroach.me/login/')
    .waitForElementVisible( 'body', 1000 )
    .verify.visible('#username')
    .verify.visible('#password')
    .verify.value( 'input[type=submit]', 'Log In' )
    .verify.elementNotPresent('.error')
  }
}

The test shown above is very simple example that will open the URL: http://dev.matthewroach.me/login/ and check that the username and password fields are visible, the submit button has a value of Log In, and the error element is not visible.

One thing to note here is I am using the .verify object rather than the .assert object. The reason I prefer to use verify over assert is that the test’s will abort if you use assert, where as using verify it will record a fail on the test but carry on running through the other tests.

Running Tests

Now you have your first test and nightwatch all set up you can just call nightwatch from terminal in your project root, and you will see output like the following, you will noticed that it still outputs the test details even tho you are running test’s on browser stack. The output is also saved within browser stack, it looks slightly different in browser stack.

Nightwatch Output

When a test causes an error the output looks as follows

Nightwatch error output

 

Now you have the basics down, look at using page objects to make your tests DRY and reusable, see my post on UI Testing with nightwatch.js – Page Objects.

Basic jQuery testing with Jasmine – Spies – Part 2

In my previous jQuery testing with Jasmine I went over the basic’s of getting everything set up and even getting a few tests running, and as these couple of post’s are about testing a jQuery plug-in rather than standard JavaScript we are using jasmine-jquery. Couple of reason I am using jasmine-jquery is mainly I am testing a jQuery plug-in, so that makes sense, and as my plug-in has click events in it we can test these events.

The examples in the post relate to the testing of my simple jQuery tabs plug-in (fluabs) which is available on GitHub with all the test’s.

Following on from the previous post, we tested the plug-in was set up and initatied on the fixture we loaded, and the default actions of the plug-in had happened. In this case only the one content area should be shown.

Now we know the plug-in can do more than that, and it should do more than that. We need to know test against the click events that the plug-in should of bound to the element to hide/show the relevant tabs.

For me I set up another describe block for testing events, happily name fluabs events:

describe('fluabs events', function() { });

In here I will test the plug-ins events for clicking on the tabs.

Test One

The first test I preform is to check if any of the tabs have a click event bound to them. As the plug-in does the event binding we can test if the event has been bound properly.

My test looks like:

  it("click tab", function() {
    var spyEvent = spyOnEvent('#tabs li a', 'click' );
    $('#tabs li a').click();
    expect( 'click' ).toHaveBeenTriggeredOn( '#tabs li a' );
    expect( spyEvent ).toHaveBeenTriggered();
  });

Ok, so let’s take a look at what this test is doing. This test is the basic example that is shown on the jasmine-jquery readme on their github page. I have modified it to work with the fluabs plug-in.

First we are assigned a spyOnEvent() function that has the element I want to perform the event on and the type of event I want to perform, to the variable spyEvent, we then trigger the event using the jQuery trigger function.
The next two lines are the test’s check a click event has happened on the element we have specified. So we expect a click to happen on the a inside the li under the #tabs element, and the second expect is to check that our spyOnEvent has been triggered

This first test is a bit open ended as all its checking is if a click event has been fired, so the plug-in could be binding the click event to the correct element but does not me the click event is working properly, we will do that next. But this is a good test to ensure we have go our event bound properly.

Test Two

The second test I preform is to check if the click event we bind in the plug-in are bound properly and do what they are supposed to do.

My test for this looks like:

it("click tab 2 - hide all tabs except tab 2, tab 2 to have class current", function() {
   var spyEvent = spyOnEvent('#tabs li a[href=#two]', 'click');
    $('#tabs li a[href=#two]').click();
    expect( 'click' ).toHaveBeenTriggeredOn( '#tabs li a[href=#two]' );
    expect( spyEvent ).toHaveBeenTriggered();
    expect( $('[data-tab="#two"]') ).toBe( ':visible' );
    expect( $('#tabs li a[href=#two]') ).toHaveClass( 'current' );
    expect( $('[data-tab="#one"]') ).not.toBe( ':visible' );
    expect( $('[data-tab="#three"]') ).not.toBe( ':visible' );
});

You will see some similarities to the first test, as we are again testing the click event, but this time we are not testing just for a click event to happen, we are testing the click event on a certain element and if it has performed the click events functions as the plug-in should be set up to do.

This means with the spyOnEvent() function will have to be more specific on the element we wish to target the click event on, so that the actual plug-in click event happens.

As this a tabs plug-in, and the way I have wrote it is, that the href part of the tabs unordered list elements is used to target the div within the .tabcontent div. The tabs list looks like:

  <ul id="tabs">
    <li><a href="#one" class="current">Tab One</a></li>
    <li><a href="#two">Tab Two</a></li>
    <li><a href="#three">Tab Three</a></li>
  </ul>

and my tabs content div looks like:

  <div class="tabcontent">
    <div class="tab" data-tab="#one">
      <p>
        This is some sample tab content in <strong>Tab One</strong>
      </p>
    </div>
    <div class="tab" data-tab="#three">
      <p>
        This is some sample tab content in <strong>Tab Three</strong>
      </p>
    </div>
    <div class="tab"data-tab="#two">
      <p>
        This is some sample tab content in <strong>Tab Two</strong>
      </p>
    </div>
  </div>

So the href part corresponds with the data-tab attribute of a div with the .tabcontent div

With that all sorted, back to the test. We have set up the click event to happen on the second tab like so :

$('#tabs li a[href=#two]').click();

Then the follow expects check that the element has been triggered on, and the spyEvent was triggered, then we check to make sure the elements on the page have changed based on what the plug-in is supposed to do.

We expected the data-tab=”#two” to be visible as we click the link with href of #two, and the plug-in adds a class of current to the a element, so we check to ensure this has happened.

Now we have checked the relevant tab is visible and the link we triggered the event on has the right class, we need to check to make sure none of the other tabcontent div’s are showing we check them to make sure they are not visible.

Basic jQuery testing with Jasmine Part 1

I have had JavaScript testing on my radar for a good few months, and in the past few months I’ve made it a big proirity to learn. So after rewriting my blog system to give me the kick up the backside, the next on my list was JavaScript Unit testing.

So for the past couple of months in my spare time I have been playing around with the JavaScript testing framework. I am going run through setting up Jasmine and jasmine-jquery and also how I tested one of my simplier plug-ins. The plug-in I am going to use in this article is a simple jQuery tab’s plug-in named fluabs. It’s available on github along with all the test’s which are not all covered in this article

What is Jasmine?

Jasmine is a behavior-driven development framework for testing JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM. And it has a clean, obvious syntax so that you can easily write tests.

Jasmine on Github

Why Jasmine?

I do not have a huge big pro’s and con’s of why I choose Jasmine. I was doing some research in the different JavaScript testing frameworks, and Jasmine seemed to come up alot, along with QUnit and Mocha. I tried Jasmine, then had a play with QUnint. I never actually tried Mocha as it just did not sit right with me for some reason.

After playing a little with Jasmine and QUnit, and hitting a couple of little issue’s with QUnit I just decided to stick with Jasmine, rather then fully evaluate them both and make a proper decision. So that’s how I chose Jasmine, probably not the best but I am getting on really well with Jasmine, at first the whole testing thing was really alien to me and it was Jasmine that actually made it click, or that fact I stuck at it.

On top of Jasmine, I am also using jasmine-jquery.

jasmine-jquery provides two extensions for Jasmine JavaScript Testing Framework

1, a set of custom matchers for jQuery framework

2, an API for handling HTML fixtures in your specs

Getting Started

As I am using jasmine-jquery, you need to download jasmine, aswell as jasmine-jquery. Also, as we are testing jQuery you will need a copy of jQuery. You could use a CDN, but I have downloaded it and put it in with my test’s so I have the correct version that I have used to develop the plug-in with. This would then allow us to re-run our tests when a new version of jQuery is released to check if our plug-in is compatable with it.

My folder structure for my plug-in is, which I use for all jQuery/JavaScript plug-in’s/code:

  • demo – contains a HTML file that demo’s my plug-in pulling the JS from the other folders
  • dist – contains a production ready minified version of the plug-in
  • src – this contains the unminfied development plug-in code
  • tests – this contains Jasmine/ jasmine-jquery, and my test stuff

Getting the files

  • First thing to do is download the latest standalone jasmine package from their github downloads page,
  • Extract and place somewhere. I put everything under a folder named tests within my plugin folder that I going be testing,
  • Download jasmine-jquery from their github downloads page – jasmine-jquery is a single file, this needs to be put in the lib folder that is in side the test folder we created in step 2,
  • Put a copy of the version of jQuery you need for your plug-in in the lib folder to

Adding jasmine-jquery and jquery

You now have jasmine and jasmine-jquery downloaded, you will now need to update the SpecRunner.html file to work with jasmine-jquery. It’s not too difficult just a case of adding a couple of script tags

You need to add the jasmine-jquery.js file on the line after the jasmine-html.js file like so:

<script type='text/javascript' src='lib/jasmine-jquery.js'></script>

And as we are testing jQuery you need to add jQuery to you page, which you will be familiar with. You add this on the line below jasmine-jquery.js like so:

<script type='text/javascript' src='lib/jquery.js'></script>

Now we have the testing lib’s all set up we need to add our test specs and jQuery plug-in so we can run test’s against it.

Adding plug-in and Spec

You need to add your jQuery plug-in code just like you would if you where putting it on any site, using the script tag. Rather than duplicate copies of the plug-in I reference the plug-in from the src folder as this is the development code and this code we want to test against. So I add the plug-in to the SpecRunner.html like:

<script type='text/javascript' src='../src/fluabs.js'></script>

We also need to create a Spec file that will contain our tests, I called it SpecFluabs.js and saved it in the spec folder, and referenced it in the SpecRunner.html file like:

<script type='text/javascript' src='spec/SpecFluabs.js'></script>

You will notice in the SpecRunner.html file there is the following code:

<!-- include source files here... -->
<script type='text/javascript' src='src/Player.js'></script>
<script type='text/javascript' src='src/Song.js'></script>
<!-- include spec files here... -->
<script type='text/javascript' src='spec/SpecHelper.js'></script>
<script type='text/javascript' src='spec/PlayerSpec.js'></script>

This is the default spec code and JavaScript code that is tested against, I remove the script tags and replace them with reference to my JavaScript and Spec files.

Testing the plug-in

As I had already written the plug-in, I’ve just added tests to prove it’s working. We could then use the testing framework set up to do test-driven development if I or someone else would like to contribute to the plug-in.

Now we got the test folder all set up we can start writing some test’s to check the plug-in does what it’s supposed to do.

The basics of Jasmine. Jasmine uses’ Suites to describe your tests, Specs which contain one or more expections, Expections which take a value, which your matcher is responsilbe for checking if your expection is true or false.

For complete documentation see the jasmine documentation

Tests

Right we have everything set up, lets write some tests. Ah one more thing, as we are testing a jQuery plugin we need to load the HTML required for the plug-in and bind the plug-in to it to simulate it being in the DOM. With jasmine-jquery you get the ability to have HTML fixtures, which is way you can load HTML content that is needed for your tests.

For this plug-in we need a fixture that contains the sample HTML needed for the plug-in. I put my fixtures in a folder named fixtures within the tests/spec/. You need to update jasmine-jquery.js to reflect where you have put your fixture folder, this is on line 76 ofjasmine-jquery.js

So we create a file named fluabs.html inside of the fixtures folder and add the HTML needed for our plug-in.

We now need to make this fixture avaiable to our test’s, which you do using loadFixtures() in your test spec.

For testing this plug-in we want to reset the state of our test code and plug-in, so that it’s as if the page has just loaded and nothing has happened. We do this using a beforeEach(). So we want to load our fixture inside of the beforeEach() function and initaite the plugin. We do this like:

var fixture;
beforeEach(function () {
  loadFixtures('fluabs.html');
  fixture = $('#tabs');
  fixture.fluabs();

});

We have defined a variable name fixture which we can use later on to test againt in our Specs. We then use the beforeEach() to load our fixture and then to initaite our plug-in. As we have told the plug-in and HTML to load beforeEach(), it will load before each spec in the describe, so we want to reset the code by using afterEach(). So we set our Suite up and theafterEach() function:

  afterEach(function () {
    $('#tabs').remove();
  });

Now we have the plug-in and the HTML needed for our plug-in getting loaded beforeEach test and our plugin is removed after each, we can start writing some tests.

We use the describe() function to set up a suite which will contain multiple specs and a nested suite with more specs.

describe('fluabs', function() {
});

Now we need to add our first spec, so the first test we need to do is to check if the plug-in has been defined and we do this like:

it('to be defined', function() {
   expect( fixture ).toBeDefined();
});

Open SpecRunner.html in your browser (use FireFox or Safari, or read the documentation on how to use Chrome), and this should pass, if not we have an issue!

Our second spec would be to check that the first content area is shown and the others are not, as we are using the plug-in with default option’s we know this is what should happen so we set our spec, expections and matchers to check all this:

it('tab one to be shown and others to be hidden', function() {
    expect( $('[data-tab='#one']') ).toBe( ':visible' );
    expect( $('[data-tab='#two']') ).not.toBe( ':visible' );
    expect( $('[data-tab='#three']') ).not.toBe( ':visible' );
  });

As we are using jasmine-jquery and jQuery we can use jQuery selectors in our expect.

Refresh your SpecRunner.html (sometimes you may need to clear your cache) and see all green.

So we have some basic test’s set up, for our plug-in, in another article I will go into more depth on testing with spies to test click events and make sure everything works.