Using incremental execution can radically reduce the time and resources needed for the execution of a calculated feature. Without incremental execution the Application Server will delete all existing observations of the calculated feature first and then (re)generate observations from all the source features used in the calculation algorithm. For huge amounts of observations this process is obviously very time and resource consuming.
Incremental execution does not delete existing observations and generates new observations only from observations that were added to the source features after the last execution of the calculated feature. In many cases this approach will need a lot less time and resources to complete. Incremental calculation can only be performed if the calculated feature and the source features of this calculated feature fulfill following criteria.
•All calculation types used in the expanded algorithm support incremental calculation.
•If the calculated feature has been executed before, no observations with a sampling timestamp less than the highest sampling timestamp detected on last execution of the calculated feature are present
•If the calculated feature has been executed before, no observations with a creation timestamp before the last execution of the calculated feature were modified after the last execution of the calculated feature
•Last observation purge is not after last execution
Some calculation types need to always operate on all observations of the source feature (or calculation), e.g. calculating the median of a numerical property within the property structure of observations will need to read all values of this property. Other calculation types (like the Copy calculation type) can easily operate only on the newly added observations.
Refer to the documentation of calculation types to see if a specific calculation type supports incremental calculation.
The user can set the incremental execution option of each calculated feature to one of the value described in table 1. By default each calculated feature will be set to Incremental if supported by algorithm and no modifications detected.
Incremental execution |
Description |
||
Incremental if supported by algorithm and no modifications detected |
The calculation will be performed incremental if supported by the algorithm and if no changes (modification, delete) were made to the observations of source features that were handled in previous executions. |
||
Incremental if supported by algorithm |
The calculation will be performed incremental if supported by the algorithm. If changes were made to observations of source features handled in previous executions the calculation will still be performed incremental. This option will only generate consistent results if no observations of the source features used in the expanded algorithm were modified or deleted since the last execution of the calculated feature. This is usually the case if the feature records sensor data from some automated sensor. When your calculated feature is based on features with a huge number of observations and if those features are usually not modified this option is optimal.
|
||
Non-Incremental |
The calculation will not be performed incremental even though the algorithm would support it. This option can be useful if you previously used the Incremental if supported by algorithm option. Also when the calculated feature operates on a small number of observations the this option might be faster. |
Table 1: Possible settings for the incremental option
