Configuring BackstopJS with Javascript

Author picture
Tom Phippen
Posted by
Tom Phippen
Date:

BackstopJS is a great bit of software for front-end developers (Breaking CSS is easy. Checking every responsive page element is hard.). But there's a few things we needed it to do that aren't obvious.

What did we want to achieve?

Different environments

By default Backstop assumes you'll either be testing a url against a previous reference image. Or that you'll be testing a specific url against a specific reference url.

For most of our work this would be fine (checking the local environment where we've done the work against live). But there are circumstances where we'd want to test a staging environment against the live site, or the current live page against an older reference image.

Different pages

The default is also for 1 url per scenario, which is quite limiting on big sites.

We want to be able to check

  • a handful of pages by default (to make sure that a change hasn't broken something unexpected)
  • a specific page we're working on
  • and a bunch of pages at the same time if we're working on a section or content type

How we'll do this

The BackstopJS GitHub page offers this tantalizing hint on how this will be possible

JSON-based configs cramping your style? Well, here's some good news – BackstopJS allows you to import all config parameters as a node module (as an option instead of JSON) which allows you to use comments, variables and logic etc. inside of your config.

But doesn't actually make it clear how to do this. Thanks to some Sherlock level sleuthing - and accidentally clicking the wrong link on a Google search - I found there used to be a configExample.js in the Git Repo, which was removed back in September.

How we want to work

The plan is to enable us to use a couple of arguments to get the variation we need, firstly to chose the reference/test environments like so:

backstop reference --configPath=backstop-settings.js --refhost=http://example.com
backstop test --configPath=backstop-settings.js --testhost=http://example.com

Then to choose the paths to test, either supplying an array in the command line:

backstop test --configPath=backstop-settings.js --paths=/,/contact

or if we're testing a lot of pages choose a file that'll load in the array:

backstop test --configPath=backstop-settings.js --pathfile=paths

and have some sensible defaults so we don't need to specify everything every time.

Setting up Backstop

If you haven't already you'll need to install Backstop, the full instructions are on GitHub, but you'll probably just need to install it globally.

npm install -g backstopjs

Then you'll need to create a directory for your settings, tests, and reports. Once you've created that you'll need to run the following command:

backstop genConfig

This creates a backstop.json file (which we won't be using) and some folders for test data.

And the last bit for our setup is installing minimist

npm install minimist

Creating our Javascript settings file

We'll be using a new file backstop-settings.js for our settings.

Step 1 is getting a few variables set up, this include the default paths array, which in this case is just the homepage.

/*
  Set up some variables
 */
var arguments = require('minimist')(process.argv.slice(2)); // grabs the process arguments
var defaultPaths = ['/']; // By default it just checks the homepage
var scenarios = []; // The array that'll have the pages to test

Step 2 is the environments. If these aren't set in the arguments it will use the defaults, on a real test they're likely to be the live site for the reference and your local install for the test.

/*
  Work out the environments that are being compared
 */
// The host to test
if (!arguments.testhost) {
  arguments.testhost  = "http://local.example.com"; // Default test host
}
// The host to reference
if (!arguments.refhost) {
  arguments.refhost  = "http://example.com"; // Default test host
}

Step 3 is to do the same again for paths, there will either be:

  • a --paths argument, that's split into an array
  • a --pathfile argument, that sets which file to load an array of paths from
  • or nothing, which uses the defaultPaths array
if (arguments.paths) {
  pathString = arguments.paths;
  var paths = pathString.split(',');
} else if (arguments.pathfile) {
  var pathConfig = require('./'+arguments.pathfile+'.js');
  var paths = pathConfig.array;
} else {
  var paths = defaultPaths; // keep with the default of just the homepage
}

If loading from a file it looks like this

var pathConfig = {};

pathConfig.array = [
  '/',
  '/contact',
  '/about',
  '/work'
]

module.exports = pathConfig;

Step 4 Is to populate the scenarios array

for (var i = 0; i < paths.length; i++) {
  scenarios.push({
    "label": paths[i],
    "referenceUrl": refhost+paths[i],
    "url": testhost+paths[i],
    "hideSelectors": [],
    "removeSelectors": [],
    "selectors": [],
    "readyEvent": null,
    "delay": 500,
    "misMatchThreshold" : 0.1,
    "onBeforeScript": "onBefore.js",
    "onReadyScript": "onReady.js"
  });
}

Step 5 the rest of the file is the standard Backstop settings, our example has a handful of breakpoints

module.exports =
{
  "id": "prod_test",
  "viewports": [
    {
      "name": "small",
      "width": 320,
      "height": 480
    },
    {
      "name": "mediumish",
      "width": 568,
      "height": 760
    },
    {
      "name": "medium",
      "width": 768,
      "height": 1024
    },
    {
      "name": "large",
      "width": 1024,
      "height": 768
    },
    {
      "name": "xlarge",
      "width": 1440,
      "height": 900
    }
  ],
  "scenarios":
    scenarios
  ,
  "paths": {
    "bitmaps_reference": "backstop_data/bitmaps_reference",
    "bitmaps_test":      "backstop_data/bitmaps_test",
    "casper_scripts":    "backstop_data/casper_scripts",
    "html_report":       "backstop_data/html_report",
    "ci_report":         "backstop_data/ci_report"
  },
  "casperFlags": [],
  "engine": "phantomjs",
  "report": ["browser"],
  "debug": true
};

Next steps

The next obvious step would be to add variable breakpoints as each one means a significant increase in the time to run the test.

Download

The files are available from this git repo