Data processing processes

You do not need to know programming languages ​​to create processing processes on your own and extract from that data most important to you and your production plant. The ETL (Extraction, Transformation and Load) wizard will help you, in which you can create process on your own (or with the support of our experts) using elements (the so-called processing nodes) that operate on the basis of drag & drop. You can also use ready-made templates built into the IPLAS system.

After starting the process from the incoming industrial data streams, you will receive what is most important to you – knowledge that will allow you to improve production in your plant. This is where you can create an individual alarm and notification system, verify and aggregate data. What if at some point you find that you would like to analyse something else in one process? Absolutely no problem! You can flexibly modify it. See how it works.

Example of building ETL process

See how we have built a simple ETL process that generates and consecutively saves to the database the totals of manufactured items as part of a shift for a specific machine.

A few words of explanation for the video above:

  • Input data
  • Used nodes and their application
  • Effect

Variable with the number of items produced in certain periods.

Node 1. Input channel, specifies the input data and the source thereof, transfers the variables on the basis of which the rest of the processing is created.

Node 2. Time stamp, the user specifies additional variables based on the frame generation time. The node adds time variables to the processing.

Node 3. Work shifts, the working hours of individual shifts are determined therein, and the result variables that relate to them are selected. Generates new variables that allow you to specify change data.

Node 4. Grouping, selecting the type of aggregation and grouping variables that divide data into certain groups, e.g. the sum of variable values ​​for the given shift numbers.

Node 5. Output table, here you define the database, destination and assign variables to specific columns. Based on the data stored in the database, analyses are generated.

The result is the processed values ​​in the database that are needed to create the analysis. The data presents the sum of the items produced for each shift.

The data generated by the ETL process can be presented in the analysis, which was shown in the video.

Documentation

In the graphic ETL wizard, you select and then drag elements to the workspace from which you create your own data processing.  All these “blocks”, “squares” or whatever else you define them, are called processing nodes. They can be divided into 4 categories by action: input nodes, filters, transforms, and output nodes. Below you will find a description, available options and an example of how to use each of them. If something is still unclear, please do not hesitate to write to us: iplas@spiid.pl.

  • Input nodes
  • Filters
  • Transformations
  • Output nodes

Input nodes (Input channel) are responsible for introducing input variables to the processing process.

Filters (Last Frames, Required Values) allow you to keep only the current data, optimize processing.

Transformations (Grouping, Determining Increments), process data, generate new variables on the basis of input variables.

Result nodes (Output table) are used to save the results of the processing.