Release Note: DataJet v6.7.1 (1/July/2024)
  • 5 Minutes to read
  • Dark
    Light
  • PDF

Release Note: DataJet v6.7.1 (1/July/2024)

  • Dark
    Light
  • PDF

Article summary

Key Features

Breaking Changes

  • NONE

Bug Fixes

  • AMP-134: Unable to load excel file that has a cell with unsupported content:  An improved error message is now returned - if possible, the problem cell location is reported.
  • LEFTOF(A,B) on null string field was generating an error - will now generate a null field.

New APIs

Modified APIs

Core Module Updates

The following core modules have been updated:

  • AWSSDK.S3 3.7.308
  • MongoDB.Bson 2.25.0
  • MongoDB.Driver 2.25.0
  • MongoDB.Driver.Core 2.25.0
  • MySql.Data 8.4.0
  • SharpCompress 0.37.2
  • SSH.NET 2024.0.0

Feature Additions and Improvements


Extensions to conditional script processing and variables

Script variables (once string only) can now be numeric or boolean

Previous versions allowed string variables to be defined using the DEFINES method:

{
  "method": "Defines",
  "description": "Read-only variables",
  "variables": [
    {
      "key": "%LoadMonth%",
      "value": "04"
    },
    {
      "key": "%LoadYear%",
      "value": "2024"
    },  
    {
      "key": "%LoadAction%",
      "value": "1"
    },
  ],
  "project": "ACX_Load_Local"
}

When evaluating these variables, the >, <, <= and <= operators could not be used as the variables were always evaluated as strings:

//Valid - true if %LoadYear% = 2024
{
  "method": "IF",
  "condition": "%LoadYear% = 2024",
  "project": "Load_1"
}

//Invalid - always evaluates to false
{
  "method": "IF",
  "condition": "%LoadYear% >= 2024",
  "project": "Load_1"
}

It is now possible to define numeric and boolean variables, and evaluate them as such within IF conditions:

{
  "method": "Defines",
  "description": "Read-only variables",
  "variables": [
    {
      "key": "%LoadMonth%",
      "value": "04"
    },
    {
      "key": "%LoadYear%",
      "value": 2024
    },  
    {
      "key": "%LoadAction%",
      "value": 1
    },
  ],
  "project": "Load1"
}

//Valid - true if %LoadYear% = 2024
{
  "method": "IF",
  "condition": "%LoadYear% = 2024",
  "project": "Load_1"
}

//Valid - evaluates to true if %LoadYear% is 2024 or higher
{
  "method": "IF",
  "condition": "%LoadYear% >= 2024",
  "project": "Load_1"
}

AMP-127: Script IF/STOPIF now supports multiple conditions

Multiple conditions can now be evaluated in conditional statements using the following logical conjuntions:

  • ||  (OR)
  • &&  (AND)
Operators
The following operators should be used:
  • ==
  • !=
  • >=
  • <=
  • >
  • <


[
  {
    "method": "STOPIF",
    "condition": "1==1 && 2==3",
    "project": "Rg2-StopIf"
  },
  {
    "method": "STOPIF",
    "condition": "1==2 || 2==3",
    "project": "Rg2-StopIf"
  },
  {
    "method": "STOPIFN",
    "condition": "1==2 || 2==3 || 5=5",
    "project": "Rg2-StopIf"
  }
]

Script now supports IFN and STOPIFN

New methods have been added:

  • IFN  Evaluates as TRUE if the condition is NOT met (IF NOT)
  • STOPIFN Stops if the condition is NOT met (STOP IF NOT)
{
  "method": "STOPIFN",
  "condition": "%THOUSAND%==1000",
  "project": "Rg2-StopIf"
}

[
  {
    "method": "IFN",
    "condition": "exists([sales]) && exists([customer])",
    "project": "Demo"
  },
  {
    "method": "IncrementProjectGlobal",
    "key": "myCounter",
    "increment": 1,
    "project": "Demo"
  },
  {
    "method": "ENDIF",
    "project": "Demo"
  }
]


Script now supports DynamicDefines [variables that get assigned/reassigned only at point of execution)

Run Time Script variables (also known as DynamicDefines) work in a similar way to DEFINES in that they allow scripts to be developed with variable input. 

Unlike DEFINES, DynamicDefines are evaluated as the script processes.   The content of a Dynamically Defined Script Variable therefore depends on the conditional execution path of the script

A variable can be defined as part of a DEFINES method and then over-written using DynamicDefines.  The same variable can be over-written many times in the same script.

{
  "method": "DynamicDefines",
  "description": "Set values as multiple of 1",
  "variables": [
    {
      "key": "%Var1%",
      "value": 1
    },
    {
      "key": "%Var2%",
      "value": 10
    },
    {
      "key": "%Var3%",
      "value": "defaultString"
    }
  ],
  "project": "LargeDataSource"
}

For more information on conditional defines, see DynamicDefines


New Join Builder - Can join multiple new field and data types.

  • Support has been added for 1 billion + primary keystring joins.
    • NOTE: Depending on the size of the key field, joins of this size may require additional memory to be added to the server 
  • The following Join Types are now supported:



Improved local and shared system script execution history views - calling JSON on error entry, variables, passed variables

Ctrl-M for more detailed results of executed script line.



Enhanced administrative control of client user experience

Lock Project with user definable message -   "method": "LockCodeAcquire",   "message":"Database being updated. Try again later.",

At the start of script processing, use LockCodeAquire to provide a message that will be displayed to any user who tries to connect to the project.  If provided, this message will replace the default "Project is locked" message of previous versions.

{
  "method": "LockCodeAcquire",
  "message":"Database being updated. Try again later.",
  "project": "Demo2023"
}

Once processing has completed, release the lock by using LockCodeRelease:

{
  "method": "LockCodeRelease",
  "project": "BikeData3",
  "newInstance": true
}


Realm/Project global config variables API (globalManagerRootPath configured in djclient.cnfg)

PROTOTYPE - DO NOT USE without guidance

globalManagerRootPath is a path in the djclient.cnfg file used for storage and retrieval of Global Project and Realm variables.

"globalManagerRootPath": "/mnt/home/engine/globalManager"

This path must be a location that is accessible by all servers and realms and is used by DataJet Web in order to coordinate across projects and realms.  The following methods are used to access global variables:

  • SetProjectGlobal - Add or overwrite a project variable in the globalManager: bool, string or integer.  Available to all scripts within the active project.
  • ProjectGlobal() - Retrieve a project variable value from the globalManager.  
  • SetRealmGlobal - Add or overwrite a realm variable in the globalManager.   Available to all realms configured to use the same globalManagerRootPath
  • RealmGlobal() - Retrieve a realm variable value from the globalManager.  
  • IncrementProjectGlobal - Increments an existing global variable by the specified amount.

If the globalManager has not been configured, the following error message will be seen:

"GlobalManager not active"


Example Use:



OPEN_WARNING in 'ProjectGlobal' variables

When a project is opened, this message will be shown if it exists.  If its blank no message is displayed.

{
  "method": "SetProjectGlobal",
  "key": "OPEN_WARNING",
  "value": "This project has not been correctly processed",
  "project": "LargeDataSource"
}

HOME_REPORT in 'ProjectGlobal' variables

If this value has been set and the report exists, explorer will open the report when the application opens.

{
  "method": "SetProjectGlobal",
  "key": "HOME_REPORT",
  "value": "Client1_Homepage",
  "project": "LargeDataSource"
}






Was this article helpful?