How to Build Tests for SiteCheck

What are Tests?

Tests are a simply a way to test the structure of your website. Running a test will either return a pass or fail result. Additionally a message will be returned indicating the success of a test or detailing the reason why it failed.

At the core, these tests are simply regular expressions. If you're not familiar with the subject I would highly suggest reading up a bit here.

What does a test look like?

Tests are provided in json format. A simple test would look something like this: {
    "testId" : 1,
    "name" : "Document Type",
    "expression" : "!DOCTYPE\\s+html",
    "description": "This site is using HTML5",
    "image" : "http://sitecheck.uphouseworks.com/images/check.png",
    "failure" : "A failure message"
}


Fairly straight forward, but let's break this down a bit first.

"testId" : 1 The unique id of this test, can be anything you like! Just be sure not to give two tests the same id.

"name" : "Document Type" The name of this test to display to the user.

"expression" : "!DOCTYPE\\s+html" The regular expression to run for this test. Here we're simply checking for an HTML5 doctype, allowing one or more spaces between DOCTYPE and html.

"description" : "This site is using HTML5" The description of this test. Generally a brief 1 - 2 line synopsis about what it means if this test passes.

"image" : "http://sitecheck.uphouseworks.com/images/check.png" The pass image to show for this test. A checkmark will do in almost all cases, but if you wish to provide your own 'pass' asset you can do so here.

"failure" : "A failure message" The message to show when a test fails. Generally a 1 - 3 line synopsis about why this is important to have and what you can do to fix it.

How do I make my own tests?

To make your own suite you have to wrap your own tests as follows. The example here uses a couple tests in the default suite. {
    "testCases": [
        {
            "testId" : 1,
            "name" : "Document Type",
            "expression" : "!DOCTYPE\\s+html",
            "description": "This site is using HTML5",
            "image" : "http://sitecheck.uphouseworks.com/images/check.png",
            "failure" : "This page does not use HTML5. You are either using an existing version of HTML or missing your DOCTYPE declaration."
        },
        {
            "testId" : 2,
            "name" : "Language",
            "expression" : "<\\s*html.*lang\\s*=\\s*[\"'][^'\"]+[\"']",
            "description": "Specifies the language this page is written for",
            "image" : "http://sitecheck.uphouseworks.com/images/check.png",
            "failure" : "Having this helps better declare what language this page supports when you intend to serve content in multiple languages"
        }
    ]
}


Additionally, if you've used the app, you'll notice a version file is also referenced. Every time SiteCheck runs a test it will check to see if the version file has changed, and if so will proceed to update, rather than downloading the entire test suite every time to check for changes.

You will need to create this version file and put something simple in it. 1 That's it, and in this case all we can keep track of how times we've changed our tests.

Note that you don't have to build a static file to perform this, you could instead create a server script file (ruby, php, node, etc.) which computes a hash of the local file and returns that for instance.

Finally, as you build your own tests it can be useful to leverage services like JSON Lint to check your json for errors.

How do I use my tests?

To use your own tests you need to open up the SiteCheck app and access the settings. You can find the settings by clicking the (• • •) at the bottom of the home page.

Once you've arrived there you need to change your Test Suite URL and Test Suite Version URL to where you are currently hosting your tests.

That's about it! At this point you're all set and using your own test suite. Now you can incorporate custom approaches to analyzing sites with the convenience of an app, enjoy!

FAQ

My tests won't change!

Make sure that you've changed both your Tests and Version urls.

If you still are having issues it is possible that your device has cached prior results for efficiency reasons. In this case you may try force restarting the app, but most likely you will need to restart your device. Ultimately be patient if this happens, it normally doesn't take too long for the device to request a new version of a cached resource.

I want the old tests back, what were the urls again?

The old test urls are as follows:

Test Suite URL
http://sitecheck.uphouseworks.com/schema.json

Test Suite Version URL
http://sitecheck.uphouseworks.com/version.html

My tests aren't working

Make sure that you've run your json tests through JSON Lint or a similar service to verify there isn't an error in your json. If you're still having issues you will need to go over your json carefully, particularly regarding escaped characters. For example trying to match whitespace, \s, will not work unless you are also escaping the slash. A properly escaped instance would look like \\s.

I'm still having issues.

You can reach out to me at ben@axolsoft.com. I'll see if I can't help out!