This dialog is displayed when you click the Add Test button in the Create Batch Test Specification dialog. Here you define the Trading Spec that will be tested, the test period and the list sampling required. It also allows you to define any parameters that should be varied over multiple iterations of this one back-test.

Important note: Most of the settings on this dialog default to the settings in the Test System tab of the Dashboard, so you can save yourself a lot of time by setting these first. The first Trading Spec shown is also drawn from the Dashboard setting. However, each time you add a new test to the batch, this dialog will display the next Spec that follows in the drop-down list, which is useful when you have a number of Spec variants to test.

The steps are as follows:

1. Enter a name for the test. This will be displayed in the left hand column of the summary report displayed at the end of the batch test, so its worth keeping it short.
2. Specify what, if anything, is to be varied in the  back-test using the ‘Test varies’ drop-down list, then press the Update Dialog button, which will adjust the  settings displayed on the dialog according to your choice. The test variation options are discussed below.
   Important note: The dialog’s Ok button won’t be displayed until the Update Dialog button has been clicked.
3. Select the trading specification that will be tested
4. Unless your test varies the ‘Test start date’, you’ll now want to add the test start and stop date and specify how frequently the rules should be evaluated
5. As with any back-test, the next thing to specify is the source of the shares that will be tested and any list sampling you’ll require (see the Test System settings for further information on this)
6. If you specified that ‘Nothing’ was to be varied in the ‘Test varies’ drop-down (e.g. if you’re just testing various versions of a Spec), then press Ok to save the test to the batch. This will take you back to the Create Batch Test Specification dialog, where you can press Add Test again to add another test.
7. If you didn’t choose ‘Nothing’, then read on to learn the method for defining how your test will be repeated over multiple iterations, while varying the test criteria that you’ve specified.

Other than ‘Nothing’, the ‘Test Varies’ drop-down list offers 5 different ways of testing different criteria over multiple test passes. These are listed and described next.

It’s not a bad idea, when you’ve developed a Trading Spec, to test it over different time periods. This helps ensure you’ve not overly optimised it for the time-period you were testing it over, while you were developing it. When you choose this option an extra group of settings is added to the bottom of the Batch Test Definition dialog:

As shown above, fields are provided for you to define the first and last dates of the test, the duration of each test in years, and the step-size in months. The first iteration of the test will run from the start-date for the specified duration. The second iteration will run from the start-date plus the step-size, for the same duration. The third will run from the start-date plus 2x the step-size, and so on. This continues until a test is executed, whose duration takes it to or beyond the stop-date. This test then stops and the next test in the batch is executed.

It can be hard to know how many positions to aim for with your Trading System. Too few and the lack of diversification increases the risk; too many and the lack of good opportunities impacts the returns you’ll see. To answer this question you can vary the trade-size, which determines the number of affordable positions. This is done by varying the parameters normally set in the Dashboard’s Trade Sizing tab. You have two options: to vary the trade-size value, or the trade-size smart percentage. In either case, the following settings group is added to the bottom of the dialog:

Here you specify the first value to test, the step-size to add to that value with each iteration of the test, and the value to stop the test at.

The whole point of developing a Spec using one cohort of shares from sampled source share lists, is that you then have fresh cohorts of shares to test it against, when you think it’s ready. This is the purpose of varying the ‘List sampling start index’, as the start index determines the cohort that is picked. Selecting this option adds the following dialog group, whose settings are set in much the same way as for varying the trade-size, above.

This option allows you to vary the value of one or more numeric, percentage or condition terms in your Spec. Only constant terms can be used, which are terms that are defined with a specific, invariant value i.e. which is not dependent on an expression, whose value may change from time to time.

Varying the definition’s value is a very useful way of determining what the value should be (if it’s a threshold for example), or for testing how sensitive your Spec is to that value (which may indicate a risk of over-optimisation). If you choose this option, the following settings group is added to the dialog:

First, select a term to vary (from the Trading Spec defined in the dialog’s Trading System group) using the ‘Parameter to vary’ drop-down list. If you change the selected Spec, you’ll need to click the ‘Update Params’ button, next to the specification list, to update the parameters displayed here.

Once you have selected a term, click Add Parameter, which will open a small dialog where you can enter the start, stop and step-size values to be used to vary the term’s value. If the term is a condition, you’ll be asked to specify the first and last state (true / false) for the condition.

Each parameter you add will be listed at the top of the dialog’s Trading Spec Parameters To Be Varied group, along with a brief description of how it will be varied