Key Features
The focus of this release is on improving analytical workflows for audience creation. Analytical reports and algorithms have been extended to provide additional processing capability and to allow for scripted flow of data from one process to another.
- Enhanced Segment Correlation (AnalyseSegments): Now includes:
- Indexing Columns to show over- and under-representation.
- Direct extraction of underlying records from the generated output report via Data Table Viewer.
- Multiple output formats: parquet, excel, table, data table.
- Integration with Multi-Field Statistics report
- Custom processing of results available via Secondary Processing, allowing for thresholds and other filters to be applied (i.e., integration with python model processor)
- Index Profile: Now includes:
- Custom processing of results available via Secondary Processing, allowing for custom thresholds and other filters to be applied (i.e., integration with python model processor)
- Target Dataset Iteration - repeat the calculation for a set of target datasets and output results in a single grid, or as a dashboard
- Dimension Iteration - repeat the calculation for a set of dimensions (i.e., columns) and output results to a dashboard, or extract segments that meet specified criteria into a dataset collection
- Caching - Integration with Data Table Viewer
- Productivity - Field Template can be used to quickly provide Multiple Dimensions
- Overlap Report: New Analysis Report. Allows the calculation of intersections across multiple datasets. Supports:
- Extended intersections - Up to 15 datasets, base filter, target filter, record extraction, result filters, resolution tables, measures, export to table, export to file, copy and paste, hit counts
- Visualisation - Thematic grid of segment inclusion/exclusion and hit-count pie-chart.
- Caching - Integration with Data Table Viewer (note: extraction not yet implemented)
- Target Dataset Iteration - repeat the calculation for a set of target datasets and output results in a single grid
- Integration with Venn Report - switch between Venn and Overlap report for first 3 segments in list
- Data Table Viewer: Provides the ability to view pre-calculated data tables as grids. Multiple models - AnalyseSegments, Overlap, Index Profile - can be calculated during data refresh/load, or as part of ad-hoc data analysis, ready for exploration and review by analysts.
- With Data Extraction: AnalyseSegments
- Without Data Extraction: CreateSegmentModel, GenProfile, IProfile, Overlap, Query Matrix
- Segment Dataset Type: Enables segments to be created directly from the Campaign report and used like any other dataset type in all analysis and reporting workflows. Removes the need to pivot data from the campaign system to the Primary Contact Table. Drag segments directly from Campaign report into Venn, Overlap, Index profile etc.
- Venn: Improved labelling, overlap analysis grid, integration with overlap report
- Workflows: Sample workflow scripts and documentation for Audience Creation, Target Selection, Statistical Analysis...etc.
- Automated Data Generation: multiple methods of automating and modifying repeated output from analytical processes are available
- foreach : Automates calculation of analytical processes and reports by iterating through input lists of fields and datasets. Creates groups of objects (fields, templates and dataset collections) for use in downstream processing. Available initially for:
- Index Profile - Iterate by Target Dataset or Dimension
- Overlap Analysis - Iterate by Target Dataset
- BuildReport: Builds a multi-report dashboard by iterating through a supplied field list. Initially for:
- Index Profile
- secondaryProcess: Runs custom processing on output from analytical processes. Initially for:
- Index Profile
- Profile
- AnalyseSegments
- Overlap
- Query Matrix
- CreateCollectionFromProcess: Takes an Analytical Object and creates datasets from output rows. Includes support for secondary processing. NOTE: Only extracts data from single-dimension axes. Data is taken from first column. Generic support for all processes that create a data table model.
- foreach : Automates calculation of analytical processes and reports by iterating through input lists of fields and datasets. Creates groups of objects (fields, templates and dataset collections) for use in downstream processing. Available initially for:
- Administration:
- Direct access to Data Tables and Context sensitive menus from Runner.
- Packages available in Remote File Manager.
- "help" command added to command line.
- "state" command added to command line.
- Datasets, Templates, DatasetModels and Reports can be imported from packages.
- Pre-Load Processing: Various improvements to improve handling of poor quality data during load, or as part of pre-load processing:
- Delimiter check during load, filename prefix,
- DelimiterCheck,
- MultiDelimiterCheck,
- Load from Parquet,
- fileAnalysis
- Extended Python Integration:
- Analysis reports support export to parquet file
- Export to Parquet: AnalyseSegments, ExecuteReport, GenProfile, IProfile, Overlap
- Model Processoraccepts multiple report types.
- Model Processor Data Inputs: discrete fields, multi-function profiles, profiles. Now also accepts: index profiles and query matrices
- Analysis reports support export to parquet file
- General UI & Productivity:
- Templates
- CreateTemplateFromFile - allows a template to be created from a client-side spreadsheet
- Display of Dataset Collections and Field Templates in Context Panel - contents can be accessed without opening the template/collection
- TranslateDataSetCollection - creates a copy of a dataset collection replacing source table with target table (note - no verification is done)
- TranslateTemplate - creates a copy of a field template replacing source table with target table (note - no verification is done)
- Open in Browser - open current project in Web Browser
- Templates
- Module Updates: Mongo P1 Security patch
AMP+ Issues Resolved in this release
- AMP 257, 187 – Export to file silent failure - error message wil be raised if export fails because of lack of disk space.
- AMP 253 – Bug LEFT(A,B) - LEFT(A,B) will not fail on fields with short field names
- AMP 252 – Enhancement – JoinLayers - JoinLayer will not fail on single layer join.
- AMP 248 – Enhancement, Venn, set resolved at. Resolution level can now be set before segments are dropped.
- AMP 221, 165 – Reports, Templates, Datasets not imported when injecting
- AMP 219 – Quantile over continuous field - Use Remap function to turn continuous field into discrete.
- AMP 159 – Pin Horizontal scroll bar to window so always visible
Other Changes
- DM-639 P7 AMP-220 Extend support for ranking to support the use of a continuous fields as the ranking field (REMAP function). Ranking is limited to discrete fields. To rank a continuous field, first remap the field to reduce the number of possible values, and then use the remapped field as the ranking field.
- DM-777 post profile process date columns can handle year month columns
- DM-778 INJECTION 'debugIgnoreInjectionErrors' added to InjectPackage to help debug project when repo is not available
- Data Model Table display added to Runner output - Allows runner output to be viewed in grid form
- Templates can be added as default views to datasets
- Scripts support %CURRENT_VERSION% - allows DataJet server version to be used as needed in object descriptions and script development
- DATEFROM supports constants (previously only supported fields)
- Expression "Replace" allows an empty string for "replacement_string"
- Expression "lhash" allows an 8 byte longinteger key to be built from a set of string fields. (Also function String:LHASH(A))
- Expression "md4" allows a 16 byte string key to be built from a set of string fields. (Also function String:MD4(A))
New API Calls
- DataTableQueryModel
- ListDataTableModels
- DropDataTableModel - Drops an existing data table model.
- GetDataTableModel -
- ExportDataTableModelIntoTable
- SetDefaultTemplate
- GetDefaultTemplate
- Overlap - Calculates and visualizes inclusion and exclusion counts and metrics across all combinations of user supplied dataSets simultaneously
- TranslateDatasetCollection - Creates a copy of a dataset collection and applies it to an alternative table
- TranslateTemplate - Creates a copy of a field template and applies it to an alternative table
Modified API Calls
- Analyse Segments - Extended to support indexing, greater output options
- IProfile - Extended to support field templates, iteration and secondary processing