Skip to content

Shift Target Column Service User Manual

Introduction

The Shift Target Column service allows to prepare a dataset for time-series forecasting applications.
The operation consists of "shifting" one or more target columns by a certain number of time steps (hops), thus generating the dataset needed to train predictive models.

Service Features

  1. Management of datasets registered in Data Analytics System
    The input is a dataset already recorded in Data Analytics System with MinIO storage.

  2. Shift target columns
    The selected target columns are moved of a number of steps (hops) defined by user.

  3. Flexible output modes

  4. Adding new shifted columns with suffix (_shifted),

  5. Or replace the original columns with the shifted versions.

  6. Layering and sorting support
    It is possible to apply the shift:

  7. separately for groups defined by a selection column,

  8. after sorting the dataset based on a sort column.

  9. Output as MinIO dataset
    The result is saved on MinIO and automatically registered as a new dataset that can be used in pipelines.

  10. Progress messages
    During processing the service sends log messages to monitor the status of the operation.

Use of the Service

Configuration

In the user interface they must be configured:
- Input dataset: a dataset recorded with MinIO storage,
- Hop size: number of time steps for the shift (≥1),
- Target columns: list of target columns to shift (comma-separated),
- (Optional) Selecting column: column to stratify the process,
- (Optional) Sorting column: column to sort the data before applying the shift,
- (Optional) Replace target columns: if active, the original columns are replaced; otherwise, new columns are added _shifted.

Execution

At the RUN of the service:
1. The input dataset is read by MinIO,
2. The shift transformation is applied to the target columns,
3. The last incomplete lines (equal to the hop size) are removed,
4. The transformed dataset is saved on MinIO as output.

Output

The result is a dataset containing the new shifted (or replaced) columns, ready to be used in forecasting models.

Advantages

  • Automated preparation: it is not necessary to write code to generate shifted datasets.
  • Flexibility: supports adding or replacing target columns.
  • Compatibility with forecasting: The resulting datasets are ready to be used with time-series predictive models.
  • Pipeline integration: Input and output managed via MinIO dataset.

Useful References