Introduction to Engineering Data

Prev Next
What is Engineering Data?
Data engineering is the process of transforming data. It involves creating and maintaining processes to make data usable for analysis and decision-making.

There are multiple approaches for engineering - or transforming - data in DataJet.  These can be broadly grouped as:

  1. Scripting
  2. Using DataJet Dektop

However, it should be noted that the two groups are not exclusive of each other - engineering that is done via the User Interface is automatically recorded in the project history, and User Interface data objects - such as (transformed) tables and fields - can be added to scripts via drag and drop.

With that in mind, the following workflows are all possible depending on the project approach and purpose:


These workflows can be grouped into 5 stages:

  1. Importing Data
  2. Inspecting, Analysing and Engineering Fields
  3. Turning insight into repeatable actions
  4. Testing and Refining repeatable engineering actions
  5. Automated engineering

The most common workflow is a mix of Interactive UI and Scripting.

Importing Data

For details on how to get started with Importing Data, see Creating a Project and Uploading Data or for more information see Introduction to Loading Data.

Inspecting, Analysing and Engineering Fields

Inspecting and Analysing Fields

Before engineering a field, it is often necessary to inspect it.   DataJet offers many different routes to inspection, including:

Engineering Fields

The following broad functional groups are available for engineering fields via the User Interface Menu options:

Often it is necessary to design and create a Data Model/Schema.    The following functional areas support this activity:

Turning Insight into repeatable actions

Testing and refining repeatable actions

  • Controlling Scripts
  • Techniques for efficient script development

Automated Engineering

  • Scheduling Load and Engineering
  • Triggers
  • Notifications
  • Data Audits