Guardian Pub

The Guardian is the continuous testing and continuous integration facility. It runs all the stories virtually all the time - whenever there is a change to either the stories or specs, or to the target environment.

The status is either green or red and the detailed report is available if you click on it.

When to test

The Guardian has a few operating modes:

  • manual, from the menu - you can start a few testing scenarios
  • auto, when some configuration (specs, stories) changes
  • scheduled, when monitoring a service and detecting a change

Note that auto is generally turned off for cloud deployments.

Configuring the scope to test

The Guardian will look at all the topics in the current reactor that match a pre-configured Tag query++ guardian.settings.query, configured in the Reactor configuration.

A good example would be (a few stories tagged sanity is a great idea for automated testing):

guardian.settings.query=Story/sanity/-skip

You can override this setting and pass this tagQuery argument into the polled message when you poll (the drawback is that this setting will only override the scheduled tests, not others - the guardian may also start automatically or on demand and those only use the guardian.settings.query):

$when diesel.guardian.poll(env)
...
=> diesel.guardian.polled(env, stamp="Etr:${t1}", tagQuery="story/sanity/-skip")

Also, the Guardian will re-run all tests when a topic matching these criteria is edited/changed:

  • are in the Spec or Story categories
  • are tagged spec or story

Note the tag query above story/sanity/-skip will include all stories tagged sanity and NOT tagged skip.

Polling and scheduling

Note: this is specific to the Guardian. For a more generic scheduling mechanism, see diesel.cron.set.

The Guardian can watch several systems and trigger specific test suites when these systems are updated. This is how the guardian keeps everything on the level via continuous testing but without overwhelming all systems.

The Guardian poller will keep a stamp for each environment and it will poll the environments on a schedule, to get this stamp. When the stamp changes, the poller will initiate a test and record the test results.

Good values for the stamp are build numbers, release numbers, build timeStamps, git commit IDs etc.

These are set for each Environment settings and require three parts:

Here's an example of schedules, configured per environment when the realm is loaded:

$when diesel.realm.loaded
=> diesel.guardian.schedule (env="sandbox", schedule="5 minutes")
=> diesel.guardian.schedule (env="dev", schedule="5 minutes")

Using a differnt realm: you can run the schedule in a different realm by passing inRealm=whichone - this only works for trusted realms, see setting diesel.trust in Reactor configuration. This option is good when testing a production reactor from a schedule running on a development reactor.

Also, by passing in inLocal=yes you can have schedules that only run in a "local" deployment, i.e. not in the dieselapps cloud. This is very useful when accessing environments only available from the local - these tests would always fail ran from the cloud!

When the schedule kicks in, every 5 minutes, a diesel.guardian.poll message is created and you have to handle it to poll the external system.

This is a typical poll, which you have to configure as the implementation for the reserved message diesel.guardian.poll - this one pings an environment-specific URL which returns a json document with a buildTimestamp:

$when diesel.guardian.poll(env)
=> diesel.setEnv(env, user=diesel.user)
=> snakk.json (url=GUARDIAN_PING_URL)
=> (payload = payload.buildTimestamp)
=> diesel.guardian.polled(env, stamp=payload)

The target system in this case implements a GUARDIAN_PING_URL which returns a JSON document. This JSON document has a field buildTimestamp which can be used to determine when new code has been deployed. When new code is detected, then the guardian will automatically start a test.

The internal implementation of diesel.guardian.polled will compare the current timestamp with the previous stored values and decide if a new test should be executed or not. If true, it will schedule a new test in that environment.

  • for diesel.guardian.polled you can add inRealm=other to notify it to be ran in another trusted realm, see diesel.trust.

At the end of a test run, the Guardiwn will decide if a notification needs to be sent, depending on the current test status and the previous tests (if it's still successful, then no notification is sent).

If the Guardian needs to send out a notification, you have to configure the notification. A typical notification goes to an email - note that you can send to different emails based on the environment, in this case sandbox:

$when diesel.guardian.notify (realm, env == "sandbox", oldStatus, newStatus, errors, total, report)
=> diesel.mail.send (
    to="a@b.ca,c@b.ca", 
    subject = "${errors} / ${total} Guardian errors in "+env, 
    body=report)

Sorting

When running a test, the stories are ran in alphabetical order, so the easiest way to organize them is with their name. It is important to be able to control the order of the tests, so that creation tests for instance run before query tests etc.

However, you can also use tags: any tag starting with "testOrder" will be used to sort, before the name. So for instance story "A2" with tag "testOrder.1" will run before "A1" with tag "testOrder.2" and before "A0" with no tags.

Note: all stories, if not having an "testOrder" tag, are considered to be tagged with "testOrderLast" - this should come after any reasonable order tag convention that you might use, like:

  • testOrder.1
  • testOrder
  • testOrder.group1
  • etc

Automatically testing all fiddles

The Guardian will also look for fiddles if this property is set to "true": guardian.settings.fiddles, in the reactor configuration. It will add any sections tagged dfiddle of type story to the list of stories to test.

TODO maybe this particular setting should be automatically set to "true"?


Was this useful?    

By: Razie | 2017-05-01 .. 2021-04-05 | Tags: academy


Viewed 197 times ( | Print ) this page.

You need to log in to post a comment!

© Copyright DieselApps, 2012-2021, all rights reserved.