Delayed Derivation

Triggers derivation processing after a delay.

Introduction

The delayed derivation type creates components that specify calculations to be performed after derivation, but not at the same time as derivation. The nodes that trigger the derivation are known as input nodes. The nodes that are recalculated are known as output nodes.

A typical scenario is a long-running reporting calculation. This needs to be triggered when data changes, but:

• It should be triggered after a short delay to ensure that other derivation has had a chance to complete before the reporting, and to pick up changes related to the initial change. This is known as the calculation delay, and is specified in seconds.
• There is no need to respond to every single change. If multiple changes occur, the calculations can be performed periodically. This is known as the calculation period, and is also specified in seconds.

The delayed derivation uses the ScheduleService service to initiate a calculation script held in the Script (system.NODE_SCRIPT) member of the delayed derivation instance. It retains a record of the last run timestamp (and any errors) within user data for the node. The logic of the delayed derivation is:

• If the calculation delay is less than zero, exit the derivation. (This is useful for turning processing off, for example during development.)
• Look up last run timestamp from user data.
• If last run timestamp is not present, or more than calculation period seconds ago, schedule the script to run after calculation delay seconds.
• If the last run timestamp is less than calculation period seconds ago, schedule the script to run after calculation period seconds. (If calculation period is less than calculation delay, use calculation delay.)
• If ScheduleService receives more than one request for the same node, it rejects the later dated one. This means that, where changes take place occasionally, the calculated data is never more than calculation delay + calculation period seconds old, and calculations are carried out at maximum once every calculation period.

The scheduled service executes the delayed derivation instance. The execution script is defined on the type. This cancels any outstanding schedule requests (which only does anything when run manually), updates the last run timestamp, and then calls the script on the instance. If the script returns an error, it updates the error message on the user data also.

The delayed derivation script can do anything. It can force derivation on output nodes, generate original data in output nodes, or do any other processing.

Using delayed derivation

Configuration

A delayed derivation requires a script, and should have a name and description.

Delayed derivations have two Bindings members types:

• The first Bindings (library.parts.DynamicBindings), on the data tab, are dynamic bindings, and are retrieved in the script using context.getBinding(). Use this for:
  • Bindings for the output nodes.
• The second Bindings (system.BINDINGS), on the script tab, are static bindings, and are retrieved in the script using application.getBinding(). Use this for:
  • Type, node type and constant bindings.
  • Bindings for the input nodes, unless these are specified on the dynamic bindings.

It is very important not to list output nodes in the static bindings, or every time the script is run it will re-trigger itself.

Dynamic bindings do not have to be specified on the delayed derivation node itself, but can be specified on a package node.

If an input node is read from a dynamic binding, then it should be listed in the Depends On member type, so that changes to the input trigger the calculation.

Dependency management

The delayed derivation node acts as a go-between between the input nodes and the output nodes. It keeps the output nodes up-to-date while reducing the impact that changes to the input nodes have on the output nodes.

For this to work effectively, it is very important that the output nodes and their dependents have no dependency on the input nodes. The output nodes and their dependents should only reference the input nodes through dynamic binding, or through member types with the Ignore Target Change (system.IGNORE_TARGET_CHANGE) tag set.

Inheritance

Some solutions will be able to use the delayed derivation type directly. Most solutions will benefit from inheriting from the type.

Inherited types should inherit from this type, but only inherit the member types that they need.

Inherited types can set the derivation script in the type or the instance. In most cases, adding it to the type will be simpler. If it is set on the type, this will override a script set on the instance. (This may seem the wrong way round, but is more secure.)

Monitoring

Instances of delayed derivation support the Delayed Derivation Tuple, which can be retrieved for monitoring purposes. Remember that the instance of delayed derivation is dependent on the input nodes, so output nodes should only reference this through dynamic binding.

The Calculation Summary (library.parts.DelayedDerivationSummaryTuple) is calculated each time, and provides the following fields:

• Last run timestamp
• Last run period
• Next run timestamp
• Next run period
• Message

The last run period is the number of seconds since the last run timestamp. If the last run timestamp and period are missing, the calculations has never been performed (or the user data has been deleted).

The next run timestamp is the schedule timestamp returned from ScheduleService, and next run period is the number of seconds between now and the next run timestamp. If no run is scheduled, these fields will not be present.

The message describes when the calculations were last performed, whether they are up to date, and when they are next scheduled. It also reports any errors.

Manual updating

The processing can be forced to run now by executing the delayed derivation instance directly, using method Run Delayed Derivation Method (library.parts.RunDelayedDerivationMethod). This will cancel any outstanding requests and run the script now.

The transaction Refresh calculations (library.parts.RefreshDelayedDerivationTransaction) can be called to offer the user the option of performing a manual update.