Contents x
      Introduction to Engineering Data
      • 1 Minute to read
      • Print
      • Dark
        Light
      • PDF
      Contents

      Introduction to Engineering Data

      • 1 Minute to read
      • Print
      • Dark
        Light
      • PDF
      Article summary

      What is Engineering Data?
      Data engineering is the process of transforming data. It involves creating and maintaining processes to make data usable for analysis and decision-making.

      There are multiple approaches for engineering - or transforming - data in DataJet.  These can be broadly grouped as:

      1. Scripting
      2. Using DataJet Dektop

      However, it should be noted that the two groups are not exclusive of each other - engineering that is done via the User Interface is automatically recorded in the project history, and User Interface data objects - such as (transformed) tables and fields - can be added to scripts via drag and drop.

      With that in mind, the following workflows are all possible depending on the project approach and purpose:


      These workflows can be grouped into 5 stages:

      1. Importing Data
      2. Inspecting, Analysing and Engineering Fields
      3. Turning insight into repeatable actions
      4. Testing and Refining repeatable engineering actions
      5. Automated engineering

      The most common workflow is a mix of Interactive UI and Scripting.

      Importing Data

      For details on how to get started with Importing Data, see Creating a Project and Uploading Data or for more information see Introduction to Loading Data.

      Inspecting, Analysing and Engineering Fields

      Inspecting and Analysing Fields

      Before engineering a field, it is often necessary to inspect it.   DataJet offers many different routes to inspection, including:

      Engineering Fields

      The following broad functional groups are available for engineering fields via the User Interface Menu options:

      Often it is necessary to design and create a Data Model/Schema.    The following functional areas support this activity:

      Turning Insight into repeatable actions

      Testing and refining repeatable actions

      • Controlling Scripts
      • Techniques for efficient script development

      Automated Engineering

      • Scheduling Load and Engineering
      • Triggers
      • Notifications
      • Data Audits
      Was this article helpful?
      What's Next