Why I chose WebdriverIO

I've been using angular for some time now and I have always utilized protractor for e2e tests. Protractor makes testing angular very simple and stable with its inbuilt support angular bindings and automatic waiting for network requests and promises to finish. I've been looking for something similar in the ReactJs ecosystem. Two tools i have looked at are nightwatch and WebdriverIO. After some quick analysis I chose WebdriverIO for a few reasons

• Its incredibly simple to setup
• It has lots of plugins that work with my other tooling, such as Webpack
• It allows you to write in a tdd/bdd fashion and doesn't try to force you into using a custom testing style like nightwatch does
• The documentation is easy to understand

Configuration

WebdriverIO uses a file called wdio.conf.js to store its configuration. My current file looks like this

exports.config = {
  specs: [
    './test/**/*.spec.ts'
  ],
  exclude: [],
  maxInstances: 10,
  capabilities: [{
    maxInstances: 5,
    browserName: 'chrome'
  }],
  sync: true,
  logLevel: 'silent',
  coloredLogs: true,
  screenshotPath: './test_results/e2e/screenshots',
  baseUrl: 'http://localhost:1025',
  waitforTimeout: 10000,
  connectionRetryTimeout: 90000,
  connectionRetryCount: 3,
  services: ['webpack', 'spa-server', 'phantomjs', /*'selenium-standalone'*/],
  webpackConfig: require('./webpack.config.prod.js'),
  spaServer: {
    path: './dist',
    port: 1025,
    fallback: 'index.html',
  },
  framework: 'mocha',
  mochaOpts: {
    ui: 'bdd',
    compilers: ['ts:ts-node/register'],
    requires: ['should']
  }
};

Lets break this down a bit.

Services

{
   ...
   services: ['webpack', 'spa-server', 'phantomjs', /*'selenium-standalone'*/]
}

The services config defines the plugins that WebdriverIO will run. It runs in the exact order that each service is defined. The services I am using above are

• wdio-webpack-service builds the application using webpack. For example above I am using the production webpack config file
• wdio-spa-server-service runs next and this hosts a little server on the specified port that server the built assets from the previous step
• wdio-phantomjs-service uses phantomjs to run our tests headlessly
• wdio-selenium-standalone-service is used to run our tests in actual browser such as Firefox and Chrome

Mocha and Typescript

Mocha and Typescript work wonderfully together and lucky both are supported in WebdriverIO. The configuration which controls this is below

{
   ...
   framework: 'mocha',
   mochaOpts: {
      ui: 'bdd',
      compilers: ['ts:ts-node/register'],
      requires: ['should']
   }
}

Also note that you don't have to use mocha or typescript. WebdriverIO lets you mix and match whatever suits your particular needs, for example you can use babel/chai/cucumber etc...

Running headlessly with Travis CI

Travis allows us to run our builds for free when you have an open source project which is awesome. All you just need to do is create a .travis.yml file and signup and link your github here

language: node_js
node_js:
  - "6"

install:
  - npm run e2e

cache:
  directories:
    - node_modules

And we need to add a new e2e script to our package.json like so

{
   "scripts": {
      "e2e": "cross-env NODE_TLS_REJECT_UNAUTHORIZED=0 wdio",
   }
}

This is using a plugin called cross-env to add a new environment variable NODE_TLS_REJECT_UNAUTHORIZED. This is needed because in the company I work for I ran into some issues with SSL when it tries to download phantomjs-prebuilt binaries or chrome webdriver, depending on your companies network you may not need this at all, in fact it's probably recommended not to have it. finally we just run wdio which runs WebdriverIO with our configuration file.

Using the above script and configuration Travis is going to build your application with webpack and then host a lightweight server which serves your built assets, next WebdriverIO will run our headless browser (PhantomJS) and run your tests against the lightweight server.

{
   ...
   spaServer: {
      path: './dist',
      port: 1025,
      fallback: 'index.html',
   }
}

This is our config in wdio.conf.js for our single page application server. The important thing here is that it runs on port 1025, Travis doesnt seem to allow you to bind to any ports below 1024 without sudo access.

If you need web server to be listening on port 80, remember to use sudo (Linux will not allow non-privileged process to bind to port 80). For ports greater than 1024, using sudo is not necessary (and not recommended).

Path just defines the path to your built assets and fallback should be familiar to anyone who has built and run a SPA.

Writing Tests

My application is fairly simple, it just displays some TODO items in a list. I created a simple folder called test and added 2 subfolders called pages and integration. Next I added 2 files the first is the Page Object which describes the functionality of our TODO items page. The second is an actual test. The final structure is this

• test
  • pages
    • todo-list.po.ts
  • integration
    • todo-list.spec.ts

I briefly mentioned the Page Object above, This is a common pattern you should utilize when created e2e tests. More info on the Page Object Pattern can be found here here.

Our todo-list.po.ts looks like this

class TodoListPageObject {

  go() {
    browser.url('/');
  }

  async getListItems() {
    return browser.elements('.todo-item');
  }

  itemExists(name: string) {
    return browser.isExisting(`div*=${name}`);
  }
}

export const todoListPageObject = new TodoListPageObject();

And our todo-list.spec.ts looks like this

import {todoListPageObject} from '../pages/todo-list.po';

describe('test1', () => {

  beforeEach(() => {
    todoListPageObject.go();
  });

  it('should have the correct title', () => {
    browser.getTitle().should.eql('Webpack-React-Typescript-boilerplate');
  });

  it('should have a list of todo items', async() => {
    let items = await todoListPageObject.getListItems();
    items.value.length.should.eql(4);
  });

  it('should have correct name on each todo item', async() => {
    todoListPageObject.itemExists('test1').should.be.true();
    todoListPageObject.itemExists('test2').should.be.true();
    todoListPageObject.itemExists('test3').should.be.true();
    todoListPageObject.itemExists('test4').should.be.true();
  });
});

There's nothing major going on in these tests, the first test simply tests the title is correct the second checks that there are 4 items in the list and the last checks that the correct text is displayed in the list.

Selenium Grid

As your tests grow your probably going to see things slow down dramatically. PhantomJS does not seem to handle parallel test runs very well. I tested with 3 parallel instances and there were a lot of random failures. Eventually your going to want to use the selenium grid. How to achieve this is outlined here. If you do not have time to setup your own grid then maybe check out saucelabs. Ive never used saucelabs personally but it seems to have some nice featues

Conclusion

I'm really liking what I see so far with WebdriverIO. Coming from Protractor though I am interested to see how this scales and how flaky these tests could potentially get. I'm really hoping I don't end up with loads of wait statements everywhere as I know from experience what a nightmare this can be. If WebdriverIO works out then I will happily add it to my react testing arsenal. For a working example please check out webpack-react-typescript-boilerplate