--- title: "Release Note: DataJet v5.10.19 20/10/2023" slug: "release-note-datajet-voct2023" updated: 2023-10-21T18:59:28Z published: 2023-10-21T18:59:28Z canonical: "docs.datajet.app/release-note-datajet-voct2023" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.datajet.app/llms.txt > Use this file to discover all available pages before exploring further. # Release Note: DataJet v5.10.19 20/10/2023 ## Key Features The key features of this release are: - [Enhancements to and Web Delivery of Standardized Reporting](/v1/docs/release-note-datajet-voct2023#reporting) - [Script Editor Productivity Enhancements](/v1/docs/release-note-datajet-voct2023#scripting) - [Enhancements to Shared Data packages](/v1/docs/release-note-datajet-voct2023#administration) - [Enhanced JSON loader](/v1/docs/release-note-datajet-voct2023#upgraded-json-connector) - [Automated Mass Profiling](/v1/docs/release-note-datajet-voct2023#automated-mass-profiling-allows-quick-repeat-build-of-profiles-for-a-set-of-dimensions-or-filters) - [API to support bulk copy up of fields via template](/v1/docs/release-note-datajet-voct2023#new) - [Performance enhancements and improved error handling for ExportIntoTable](/v1/docs/release-note-datajet-voct2023#breaking-changes) - Improved support for extensible plugins BREAKING CHANGESNote: this release contains breaking changes.  See details: - [ExportIntoTable - v5.10.13](/v1/docs/exportintotable-v51013) - ExportGenProfileIntoTable - Dataset overwrite ## Overview #### Documentation - [Getting Started with DataJet](/v1/docs/home-page-1) - [Introduction to Scripting](/v1/docs/introduction-to-scripting) #### Administration - [CreatePackage option added to Project menu](/v1/docs/release-note-datajet-voct2023#createpackage-option-added-to-project-menu) - [CreatePackage adds new packages to Storage Hub](/v1/docs/release-note-datajet-voct2023#createpackage-option-added-to-project-menu) - I[njectPackage wizard lists available packages and contents](/v1/docs/release-note-datajet-voct2023#injectpackage-wizard-lists-available-packages-and-contents) #### API [*Modified methods*](/v1/docs/release-note-datajet-voct2023#modified) - ExportIntoTable - Export - CreatePackage - InjectPackage - DeleteFields [*New Methods*](/v1/docs/release-note-datajet-voct2023#new) - CopyUpFromTemplate - Exists - Empty - CheckDiskSpace - CopyFile #### Engineering and Analytics - [Automated mass profiling allows quick repeat build of profiles for a set of dimensions](/v1/docs/release-note-datajet-voct2023#automated-mass-profiling-allows-quick-repeat-build-of-profiles-for-a-set-of-dimensions-or-filters) - [Combined Profile supports auto-repeat for selected field values](/v1/docs/release-note-datajet-voct2023#combined-profile-supports-autorepeat-for-selected-field-values) #### Reporting - [Copy Script option added to managed reports](/v1/docs/release-note-datajet-voct2023#copy-script-option-added-to-managed-reports) - [New standardized report layout](/v1/docs/release-note-datajet-voct2023#new-standardized-report-layout) - [Auto-generate standardized report from a dashboard](/v1/docs/release-note-datajet-voct2023#autogenerate-standardized-report-from-a-dashboard) - [Read-only standardized reports available in a browser](/v1/docs/release-note-datajet-voct2023#readonly-standardized-reports-available-in-a-browser-beta) #### Scripting - [Field templates can be modified in script editor by dragging fields from Project Explorer](/v1/docs/release-note-datajet-voct2023#field-templates-can-be-modified-in-script-editor-by-dragging-fields-from-project-explorer) - ["targetTable" and "template" keys can be modified using right-click](/v1/docs/release-note-datajet-voct2023#targettable-and-template-keys-can-be-modified-using-rightclick) - [Run-time execution history tracking in Script Editor](/v1/docs/release-note-datajet-voct2023#runtime-execution-history-tracking-in-script-editor) - [Tooltips can be added to scripts](/v1/docs/release-note-datajet-voct2023#tooltips-can-be-added-to-scripts) - [Folded script contents can be viewed in popup up editor](/v1/docs/release-note-datajet-voct2023#folded-script-contents-can-be-viewed-in-popup-up-editor) - Improved error propagation for nested scripts (REF: AMP-12) - [Helper for IF conditions](/v1/docs/release-note-datajet-voct2023#helper-for-if-conditions) (REF: AMP-23) #### Database Build, Export and Management - [](/v1/docs/release-note-datajet-voct2023#export-exportintotable-and-exportintotableex-can-take-an-optional-template-key)[Changes to Export behaviour](/v1/docs/release-note-datajet-voct2023#breaking-changes)  (REF: AMP-8, AMP-9, AMP-14, AMP-16, AMP-19 ) - [Export, ExportIntoTable and ExportIntoTableEx can take an optional "template" key](/v1/docs/release-note-datajet-voct2023#export-exportintotable-and-exportintotableex-can-take-an-optional-template-key) - [Extended Join dialog - dataset information available in layers view](/v1/docs/release-note-datajet-voct2023#extended-join-dialog-dataset-information-available-in-layers-view) - [Project descriptions can be changed for open projects](/v1/docs/release-note-datajet-voct2023#project-descriptions-can-be-changed-for-open-projects) - [InjectPackage supports "templates" key, allowing subset(s) of fields in the package to be imported.](/v1/docs/release-note-datajet-voct2023#injectpackage-wizard-lists-available-packages-and-contents) - [Upgraded JSON connector](/v1/docs/release-note-datajet-voct2023#upgraded-json-connector) - [Quick delete of multiple fields in Details view](/v1/docs/release-note-datajet-voct2023#quick-delete-of-multiple-fields-in-details-view) #### General Desktop UI - [Additional columns added to list-view - discrete count and and field-type](/v1/docs/release-note-datajet-voct2023#additional-columns-added-to-listview-discrete-count-and-and-fieldtype) - [Column Limited warning on DataSet View](/v1/docs/release-note-datajet-voct2023#column-limited-warning-on-dataset-view) ### Breaking Changes The default behaviour of export functions have changed and **existing scripts may require modification**. See [ExportIntoTable - v5.10.13](/v1/docs/exportintotable-v51013) for details of these changes Changes to API Methods #### Modified The following existing API methods have been modified for this release | Category | Method | Description | | --- | --- | --- | | Export | [Export](https://preview.portal.document360.io/v1/docs/export-1) | - Support for templates | | Export | [ExportIntoTable](/v1/docs/exportintotable) | - Support for templates - Change to empty table behaviour (see [ExportIntoTable - v5.10.13](/v1/docs/exportintotable-v51013) for details) | | Tools | [CreatePackage](/v1/docs/createpackage) | - If [Storage Hub](/v1/docs/storage-hub) is configured, will add created package and metadata to the hub. | | Tools | [InjectPackage](/v1/docs/injectpackage) | - Option to look in Storage Hub for packages - New Helper | | | DeleteFields | | #### New | Category | Method | Description | | --- | --- | --- | | Data Model | [CopyUpFromTemplate](/v1/docs/copyupfromtemplate) | - Copies fields in field template from foreign table to primary table | | | [CheckDiskSpace](/v1/docs/scriptingmethod-2) | - Checks that minimum specified disk space is available an raises an error if it isnt. | ### ### Feature Details ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697553519713.png) #### CreatePackage option added to Project menu **Project | Create Package** will create a package file and associated metadata for the currently open project. The package details will be added to the [storage hub](/v1/docs/script-hub-overview). For further details on packages, see: - [CreatePackage](/v1/docs/createpackage) - [Deep Dive - Sharing Data by Injection](/v1/docs/deep-dive-sharing-data-by-injection) ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697202700170.png) #### #### InjectPackage wizard lists available packages and contents The [InjectPackage](https://preview.portal.document360.io/v1/docs/injectpackage) wizard will interrogate the [Storage Hub](/v1/docs/storage-hub) for details of available [packages](/v1/docs/deep-dive-sharing-data-by-injection). Selecting a package will populate the helper with details of the package contents. Choosing "Apply" will add Selected options which will then be added to the method JSON. ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697202957990.png) #### Automated mass profiling allows quick repeat build of profiles for a set of dimensions or filters Rapid creation of similar profiles is now possible through [automated mass profiling.](/v1/docs/dashboard-auto-generate) ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697204914299.png) To create a mass profile dashboard: 1. Create a **profile template** using one of: 1. [Profile](/v1/docs/profile) 2. [Index Profile](/v1/docs/index-profile) 3. [Multi-Function Profile](/v1/docs/multi-function-profile) 2. Create one of the following: 1. [Dataset Collection](/v1/docs/dataset-collections) 2. [**Field Template**](/v1/docs/template-collection) 3. Drop either a "Dataset Collection" or "Field Template" onto the profile: 1. **Dataset Collection** - this will create a duplicate copy of the profile template **for each Dataset in the collection** and add it to a dashboard of the selected layout.   The dataset filter in the profile template will be replaced with the datasets in the collection 2. **Field Template** - this will create a duplicate copy of the profile template for each **FIELD in the field template** and add it to a dashboard of the selected layout.   The dimension in the profile template will be replaced with the fields in the template. See [Dashboard - Auto Generation](/v1/docs/dashboard-auto-generate) for more information on auto-generating dashboards. #### Combined Profile supports auto-repeat for selected field values [Combined profiles](/v1/docs/combined-profile) allow multiple [profiles](/v1/docs/profiles-1) with common row labels to be joined into a single result set. It is now possible to auto-iterate a combined profile by dropping a [dataset collection](/v1/docs/dataset-collections) onto an existing profile within the combined profile report: ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697658652300.png) See [Combined Profile](/v1/docs/combined-profile) for further details #### Copy Script option added to managed reports Standardized script JSON to support automation of [standardized reports](/v1/docs/static-standardized-report-quick-reference) can be extracted from the Manage Standardized Report dialog by: 1. opening the dialog 2. selecting the report and choosing copy script 3. going to script editor and pasting the clipboard content into the script. A "Save Standardized Report" method will be added to the script. ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697726746649.png) #### Helper for IF conditions See [IF ELSE ENDIF](/v1/docs/if-else-endif-1) for further details ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697815953803.png) #### New standardized report layout Previously, standardized reports had fixed layouts that could not be modified once the report had been created. Standardized reports now use a mechanism more like that of dashboards, with configurable slot sizes. ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697658739642.png) ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697735380769.png) #### Auto-generate standardized report from a dashboard It is now possible to convert dashboards into standardized reports. This allows 2 views of the same reporting data to be created, making use of the different features of standardized reports and dashboards. (For a summary of the differences between the two display types, see [Standardized Report](/v1/docs/static-standardized-report-quick-reference)) ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697659188790.png) #### Read-only standardized reports available in a browser (BETA) DataJet Web allows [standardized reports](/v1/docs/static-standardized-report-quick-reference) and [dashboards](/v1/docs/dashboard-quick-reference) to be viewed in a browser by selecting the "preview" option on the toolbar: ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697735810126.png) (NOTE: This requires installation and configuration of datajet.app) (NOTE: This is BETA functionality and may not be configured for your system) ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697818505607.png) #### Field templates can be modified in script editor by dragging fields from Project Explorer The [AddTemplate](/v1/docs/addtemplate) scripting method can be modified in script editor via drag and drop. To add a field to an existing scripted template definition, drag a field object from the database explorer and drop it onto the JSON panel. The following options will be available: - Replace definition - this will replace the existing field definition list with the new field name. - Add field - add the new field to the list.  (Note this will not verify for duplicates) ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697702261718.png) #### "targetTable" and "template" keys can be modified using right-click [Methods](/v1/docs/scripting-methods-overview) which contain "targetTable" and "template" will provide a selector for those keys: ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697702927688.png) #### Run-time execution history tracking in Script Editor Script Editor [run-time execution history](/v1/docs/execution-history-1) is now automatically recorded. This means results and timings generated during script development are always available. ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697703019047.png) The history viewer has the option to display JSON details (Tools | View JSON) ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697793108259.png) The execution history can be saved as a JSON file, or exported as tab delimited (ready for copy and paste into Excel) ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697716947899.png) For further details on available script logs, see: - [Introduction to Scripting](/v1/docs/introduction-to-scripting) - [Execution History](/v1/docs/execution-history-1) #### Tooltips can be added to scripts Each script method now supports a "tooltip" key: ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697795076935.png) Content entered into the body of remarks and navigation points will also display as a tooltip: ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697795566490.png) #### Folded script contents can be viewed in popup up editor Sections of script which are complete, and logically associated with each other, can be collapsed into folds in order to help readability of long scripts. Folds can be expanded and collapsed as needed. Methods within a fold are read-only and cannot be changed until unfolded. To copy methods from within a fold, select the fold and choose "Peek Fold" from the script editor right-click menu - a script dialog will be opened showing the fold contents. ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697703740239.png) #### Export, ExportIntoTable and ExportIntoTableEx can take an optional "template" key Export field lists can be controlled via [field templates](/v1/docs/template-collection). ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697712693321.png) #### Extended Join dialog - dataset information available in layers view Details of any foreign table filter applied to control record selection are now shown in the [JoinLayers](/v1/docs/count) dialog: ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697812178883.png) #### Project descriptions can be changed for open projects Project descriptions are displayed in: - [Project | Manage Projects](/v1/docs/manage) - [Project | Open](/v1/docs/open) To edit a project description, select the project in Manage Projects, right-click and choose Edit Description. ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697715044660.png) #### #### Upgraded JSON connector A new version of ndjsonproc.exe is provided. This includes an overwrite flag which forces a reprocessing of all previously processed files in the "output" folder. ```json { "method": "XProcess", "name": "NDJSONProc", "wait": true, "param": { "root": "%SOURCE%", "output": "%PROCESSED%", "delim": "HSTART", "extension": ".load", "stopFile": false, "overwrite": true, "definition": { "id": "integer", "code": "string", "shop_id": "integer", "packet": "var", "simple_packet": "var", "ts": "datetime", "detail": "var" } }, "example": { "ts": "2022-11-21 22:59:58.937", "id": 789456, "code": "EMAIL.send", "shop_id": 100, "guid": "3395556CC43E49DCA5A5B0350202F809", "packet": "\"8F837EF47ABF4EA7AD2EAA21455F8029\"", "simple_packet": "\"8F837EF47ABF4EA7AD2EAA21455F8029\"", "origin": "8F837EF43ABF4457AD2EAA21455F8029", "detail": {} }, "tooltip": "Process JSON source data into format ready for load", "description": "JSON source data preparation ", "project": "JSONLoad1" } ``` #### Quick delete of multiple fields in Details view Multiple fields can be selected for deletion using the Table | Details view. ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697716170326.png) #### Additional columns added to list-view - discrete count and and field-type ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697717009644.png) ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697717054808.png) #### #### Column Limited warning on DataSet View If more than 512 fields are added to a dataview, the "Column Limited" warning will be displayed, and only the first 512 fields will be included. ![](https://cdn.document360.io/73b6469b-c1e4-47e9-aa65-986c8ebd17ca/Images/Documentation/image-1697816853888.png)