Introduction to Engineering Data

There are multiple approaches for engineering - or transforming - data in DataJet.  These can be broadly grouped as:

1. Scripting
2. Using DataJet Dektop

However, it should be noted that the two groups are not exclusive of each other - engineering that is done via the User Interface is automatically recorded in the project history, and User Interface data objects - such as (transformed) tables and fields - can be added to scripts via drag and drop.

With that in mind, the following workflows are all possible depending on the project approach and purpose:

These workflows can be grouped into 5 stages:

1. Importing Data
2. Inspecting, Analysing and Engineering Fields
3. Turning insight into repeatable actions
4. Testing and Refining repeatable engineering actions
5. Automated engineering

The most common workflow is a mix of Interactive UI and Scripting.

Importing Data

For details on how to get started with Importing Data, see Creating a Project and Uploading Data or for more information see Introduction to Loading Data.

Inspecting, Analysing and Engineering Fields

Inspecting and Analysing Fields

Before engineering a field, it is often necessary to inspect it.   DataJet offers many different routes to inspection, including:

• Data View
• Data Audit
• Context Panel
• Profiling
• Modelling
• Statistics
• Field Analyzer

Engineering Fields

The following broad functional groups are available for engineering fields via the User Interface Menu options:

• Decodes
• Banded Fields
• Functions
• Expressions
• User Defined Fields
• Aggregations

Often it is necessary to design and create a Data Model/Schema.    The following functional areas support this activity:

• Joins
• Key Tables
• Time Series Tables
• Data Model Scripting Methods

Turning Insight into repeatable actions

• Script Editor
• Scripting Methods
• Project History and Import Definitions

Testing and refining repeatable actions

• Controlling Scripts
• Techniques for efficient script development

Automated Engineering

• Scheduling Load and Engineering
• Triggers
• Notifications
• Data Audits