Execution Rules

Resources

In Oracle databases, too much parallel activity can consume all temp space and cause the database to abort tasks. The Resources : Temp Space parameter in vv_sqts_config sets the maximum temp space in bytes. The scheduler records how much temp space each task used on its last run. Before starting a task, it checks whether the sum of the demand of running tasks plus the last consumption of the new task would exceed that limit. If so, the task is not started.

Sensible initial values come from the environment at installation time; adjust them later from operational experience.

Parallelism

The Parallel : Threads parameter in vv_sqts_config limits how many jobs the scheduler may start at once. When that number is reached, it waits until a job finishes before starting another.

Execution Order

Transformations are executed in the following order (evaluation is done in the specified order):

• Priority (see below)
• Runtime. Transformations that have run longer in the past are started earlier
• Sequence number of the transformation (sqts_action_id)

Priority

A priority can be assigned to a batch element and also to a connection. This is used when there are more waiting transformations than free processes. Transformations with higher priority (= lower number) are then preferred.

When you set a priority for a connection, all tasks that use this connection receive the same priority. This achieves in the DWH that all tables of a source are transferred at approximately the same time. By using different priorities for the respective connections, you can control the order in which data is loaded.

Since this method does not create dependencies (i.e. instead of artificial layer dependencies), everything is always processed as far as the dependencies between the individual transformations allow, even if higher-priority transformations have failed or depend on failed transformations.