When you evolve your own Trading Spec and investigate the performance of its back-tests, you’ll find you get many ideas for refining your Spec. To conform with golden rule B: Change one thing at a time, you’ll want to test these ideas one by one, which means copying, modifying and testing multiple versions of your Spec. However, waiting for each individual back-test to complete would be mind numbingly tedious.

Fortunately you don’t have to, because RuleTrader allows you to batch-up a series of tests, then leave them to run unattended. It then generates a table of results from each test for you. The same system also allows you to automatically vary the start-date of a test (useful for checking for temporal over-optimisation), or the nominal trade-size (to get a feel for the best balance between diversification and risk), or the cohort of sampled shares that is used from your Trading List (also useful to avoid over-optimisation).

It also allows you to automatically vary the value of a constant term in your Spec, which is very handy when you don’t know what the value should be (e.g. if it’s a threshold), or if you want to test how sensitive your Spec is to that value – which again might indicate possible over-optimisation. This is another reason why it’s a good idea to break expressions down into Named Terms.

To learn how to create and run batch-tests, we’ll use this latter feature to investigate the threshold level for our ‘Stop-Loss’ rule, which is set by the aptly named ‘Stop-Less Threshold’ term. You’ll remember that this term (after we corrected the error) is defined as:

This is a constant term – a term that is defined with a fixed percentage, number or conditional state, rather than by the variable output of some expression. These are the terms that can be varied using batch-tests, in the following manner:

1. Open the Dashboard and click the Reset Back-Test button, to reset the system, then re-open the Dashboard and click the Batch Back-Tests button, which is also on the right of the Dashboard
2. This opens the ‘Batch Tests For Unattended Execution’ dialog. In the Batch-test file field enter the name for the batch-test e.g. Stop-Loss Threshold Test. Then click the Create/Edit button below that field

3. The next dialog is where you set global parameters for the test and add individual tests to the batch. You won’t normally need to change the parameters, so leave the checkmarks checked and just click the Add Test button, to add the first test.

4. The ‘Create Batch-Test Definition’ dialog now opens. Enter a name for the test in the Test name field, such as Stop-Loss Threshold =  (you’ll see why we added the = when you see the table of results for the test).

5. In the Test varies drop-down list below, select ‘Trading spec term(s)’ then click the Update Dialog button. The dialog wil update and an Ok button will now appear in the top-right.

6. The settings in the Trading System, Test Scope and Trading List Sampling groups should already be set up, as they are taken from the current settings in the Dashboard, so it’s a good idea to configure the Dashboard settings before you start creating a batch-test.

Just check that the Securities to test setting is set to ‘All securities loaded from trading-list file’ and the List file-name is correctly set, as this will allow the batch test to run as quickly as possible.

7. When you clicked the Update Dialog button you’ll have seen an extra section was added to the bottom of the dialog:

In the Parameter to vary drop-down list, pick the ‘Stop Loss\Stop Loss Threshold’ term (RuleTrader always uses this MainDefinition\PrivateTerm notation when it is referring to Private Terms).You’ll see it’s the only term offered, as it is the only constant term defined in the Spec. 

8. Now click the Add Parameter button and the ‘Parameter Variation Settings’ dialog wil open, where you’ll specify the values you want the ‘Stop Loss Threshold’ term to be set to, for each back-test. Let’s start with a value of -10%, then decrease it by -5% in each subsequent test, until a value of -30% is reached, which will be the value used for the last test. If you then click the Update button, you’ll be told the number of back-tests that will be run (the iterations) is 5 to cover each threshold of: -10%, -15%, -20%, -25% and -30%. So the settings are:

9. Click Ok, then Ok again and you’ll see your test has been added to the main batch-test dialog. We still need to set up the reports that will be run after each test, so click Save & Exit, then ensure you have the Performance by Account, Period, Trading Rule and Trade Type reports selected.
10. Then go back to the Dashboard, click Clear Console, then Batch Back-Tests and in the next dialog click Execute to start the batch-test. You’ll be asked to confirm the start so just click Ok, and you’re off!

The test will take about 7 minutes to run through each test. If ShareScope’s progress bar / window freezes, because you are doing other things with you computer, you can still see how your batch-test is getting on, by going to the directory: \ShareScope\ShareScript\Output\RuleTrader\Test Sessions\Batch Tests\Results

Once there, open the file with the same name as your batch-test in your favourite note-pad app. In our case this file was ‘Stop-Loss Threshold Test Results.txt’. This file contains the table of results for each test and will become available once the first test is complete. Both NotePad and NotePad++ will automatically add a new row of results (though you may have to click away from them then back on them), after each test completes.

When we ran the test, the results were pretty much the same irrespective of the stop-loss level, though given the test was only over 8 months in a rallying market, it’s not possible to read too much into this. It’s always a good sign, though, if your Spec’s performance doesn’t change too much when some parameter is varied, as it means it hasn’t been over-optimised for a particular value of that parameter.

While you’re waiting for the batch-test to complete, may we suggest you review the various Help articles on creating Trading Specifications, perhaps with a focus on the Definition Constructs available to you for creating Strategy, Rule, Term and List Selector definitions.

As you create multiple variants of your Trading Spec, they will start to clutter up the Specification drop-list in the Test System tab of the Dashboard. The way to deal with this is to use RuleTrader’s Import/Export facility, the button for which is immediately below the drop-list. This opens the ‘Trading Specification Import/Export’ dialog.

Here you can import and export Specs to archive files, before deleting them. This reduces the clutter, while allowing you to recover a Spec from your archive, should you wish to do so in the future.

You can also import Trading Specs from the RuleTrader Test session to the Live session and vice-versa. We’ll do this now as we show you how to deploy the Trend Trader v1.01 Spec for live trading.