Release Note: DataJet v5.10.19 20/10/2023

6 Minutes to read

Print
Dark
Light
PDF

Release Note: DataJet v5.10.19 20/10/2023

6 Minutes to read

Print
Dark
Light
PDF

Article summary

Did you find this summary helpful?

Thank you for your feedback

Key Features

The key features of this release are:

BREAKING CHANGES

Note: this release contains breaking changes. See details:

ExportIntoTable - v5.10.13
ExportGenProfileIntoTable
Dataset overwrite

Overview

Documentation

Administration

API

Modified methods

ExportIntoTable
Export
CreatePackage
InjectPackage
DeleteFields

New Methods

CopyUpFromTemplate
Exists
Empty
CheckDiskSpace
CopyFile

Engineering and Analytics

Reporting

Scripting

Field templates can be modified in script editor by dragging fields from Project Explorer
"targetTable" and "template" keys can be modified using right-click
Run-time execution history tracking in Script Editor
Tooltips can be added to scripts
Folded script contents can be viewed in popup up editor
Improved error propagation for nested scripts (REF: AMP-12)
Helper for IF conditions (REF: AMP-23)

Database Build, Export and Management

General Desktop UI

Breaking Changes

The default behaviour of export functions have changed and existing scripts may require modification.

See ExportIntoTable - v5.10.13 for details of these changes

Changes to API Methods

Modified

The following existing API methods have been modified for this release

Category	Method	Description
Export	Export	Support for templates
Export	ExportIntoTable	Support for templates Change to empty table behaviour (see ExportIntoTable - v5.10.13 for details)
Tools	CreatePackage	If Storage Hub is configured, will add created package and metadata to the hub.
Tools	InjectPackage	Option to look in Storage Hub for packages New Helper
	DeleteFields

New

Category	Method	Description
Data Model	CopyUpFromTemplate	Copies fields in field template from foreign table to primary table
	CheckDiskSpace	Checks that minimum specified disk space is available an raises an error if it isnt.

Feature Details

CreatePackage option added to Project menu

Project | Create Package will create a package file and associated metadata for the currently open project.

The package details will be added to the storage hub.

For further details on packages, see:

InjectPackage wizard lists available packages and contents

The InjectPackage wizard will interrogate the Storage Hub for details of available packages. Selecting a package will populate the helper with details of the package contents.

Choosing "Apply" will add Selected options which will then be added to the method JSON.

Automated mass profiling allows quick repeat build of profiles for a set of dimensions or filters

Rapid creation of similar profiles is now possible through automated mass profiling.

To create a mass profile dashboard:

Create a profile template using one of:
Create one of the following:
1. Dataset Collection
2. Field Template
Drop either a "Dataset Collection" or "Field Template" onto the profile:
1. Dataset Collection - this will create a duplicate copy of the profile template for each Dataset in the collection and add it to a dashboard of the selected layout. The dataset filter in the profile template will be replaced with the datasets in the collection
2. Field Template - this will create a duplicate copy of the profile template for each FIELD in the field template and add it to a dashboard of the selected layout. The dimension in the profile template will be replaced with the fields in the template.

See Dashboard - Auto Generation for more information on auto-generating dashboards.

Combined Profile supports auto-repeat for selected field values

Combined profiles allow multiple profiles with common row labels to be joined into a single result set.

It is now possible to auto-iterate a combined profile by dropping a dataset collection onto an existing profile within the combined profile report:

See Combined Profile for further details

Copy Script option added to managed reports

Standardized script JSON to support automation of standardized reports can be extracted from the Manage Standardized Report dialog by:

opening the dialog
selecting the report and choosing copy script
going to script editor and pasting the clipboard content into the script.

A "Save Standardized Report" method will be added to the script.

Helper for IF conditions

See IF ELSE ENDIF for further details

New standardized report layout

Previously, standardized reports had fixed layouts that could not be modified once the report had been created. Standardized reports now use a mechanism more like that of dashboards, with configurable slot sizes.

Auto-generate standardized report from a dashboard

It is now possible to convert dashboards into standardized reports. This allows 2 views of the same reporting data to be created, making use of the different features of standardized reports and dashboards.

(For a summary of the differences between the two display types, see Standardized Report)

Read-only standardized reports available in a browser (BETA)

DataJet Web allows standardized reports and dashboards to be viewed in a browser by selecting the "preview" option on the toolbar:

(NOTE: This requires installation and configuration of datajet.app)

(NOTE: This is BETA functionality and may not be configured for your system)

Field templates can be modified in script editor by dragging fields from Project Explorer

The AddTemplate scripting method can be modified in script editor via drag and drop.

To add a field to an existing scripted template definition, drag a field object from the database explorer and drop it onto the JSON panel. The following options will be available:

Replace definition - this will replace the existing field definition list with the new field name.
Add field - add the new field to the list. (Note this will not verify for duplicates)

"targetTable" and "template" keys can be modified using right-click

Methods which contain "targetTable" and "template" will provide a selector for those keys:

Run-time execution history tracking in Script Editor

Script Editor run-time execution history is now automatically recorded. This means results and timings generated during script development are always available.

The history viewer has the option to display JSON details (Tools | View JSON)

The execution history can be saved as a JSON file, or exported as tab delimited (ready for copy and paste into Excel)

For further details on available script logs, see:

Tooltips can be added to scripts

Each script method now supports a "tooltip" key:

Content entered into the body of remarks and navigation points will also display as a tooltip:

Folded script contents can be viewed in popup up editor

Sections of script which are complete, and logically associated with each other, can be collapsed into folds in order to help readability of long scripts.

Folds can be expanded and collapsed as needed.

Methods within a fold are read-only and cannot be changed until unfolded.

To copy methods from within a fold, select the fold and choose "Peek Fold" from the script editor right-click menu - a script dialog will be opened showing the fold contents.

Export, ExportIntoTable and ExportIntoTableEx can take an optional "template" key

Export field lists can be controlled via field templates.

Extended Join dialog - dataset information available in layers view

Details of any foreign table filter applied to control record selection are now shown in the JoinLayers dialog:

Project descriptions can be changed for open projects

Project descriptions are displayed in:

To edit a project description, select the project in Manage Projects, right-click and choose Edit Description.

Upgraded JSON connector

A new version of ndjsonproc.exe is provided. This includes an overwrite flag which forces a reprocessing of all previously processed files in the "output" folder.

{
  "method": "XProcess",
  "name": "NDJSONProc",
  "wait": true,
  "param": {
    "root": "%SOURCE%",
    "output": "%PROCESSED%",
    "delim": "HSTART",
    "extension": ".load",
    "stopFile": false,
    "overwrite": true,
    "definition": {
      "id": "integer",
      "code": "string",
      "shop_id": "integer",
      "packet": "var",
      "simple_packet": "var",
      "ts": "datetime",
      "detail": "var"
    }
  },
  "example": {
    "ts": "2022-11-21 22:59:58.937",
    "id": 789456,
    "code": "EMAIL.send",
    "shop_id": 100,
    "guid": "3395556CC43E49DCA5A5B0350202F809",
    "packet": "\"8F837EF47ABF4EA7AD2EAA21455F8029\"",
    "simple_packet": "\"8F837EF47ABF4EA7AD2EAA21455F8029\"",
    "origin": "8F837EF43ABF4457AD2EAA21455F8029",
    "detail": {}
  },
  "tooltip": "Process JSON source data into format ready for load",
  "description": "JSON source data preparation ",
  "project": "JSONLoad1"
}