Key Features
The focus of this release is on improving project wide search capabilities. Various Tech-ops bugs and minor feature requests are also addressed.
- Advanced Search - Now Includes: the ability to search across the entire project for Field Names, Discrete Values, Descriptions, Meta data (Note: to choose which metadata columns to include in the output grid, use "select metadata")
- Improvements to Signals- reductions in disk space, processing time and query times as well as additional metrics
- PreProcessSegments - pre-processes *.tsv files, after which tsv files can be removed. Output files are approximately 1/3 the size of input tsv files, and are processed by ProcessSegments in approx 1.2 the time.
- ProcessSegments - Performance improvements:
- Processing time is reduced if PreProcessSegments is run
- Segment query times are improved if segment accelerator files are created - once the first signal segment is calculated, query times take approx 1/2 the time to calculate.
- Additional metrics- basic metrics for records in the primary contact table are available:
- entries: total number of times hash code appeared in input files
- total_segments: total number of segments in which hash code appears
- unique_segments: unique number of segments in which hash code appears
- Primary Contact / Signal synchronisation check
- If "segmentMetafile" property is added to CreateTableFromFile when creating the Primary Contact Table, the unique GUID associated with the loaded maindatafile.txt will be stored with the Primary Contact Table. Signal (segment) queries will generate an error if the project signal data does not match the Primary Contact data.
- Analyse Segments Report - provides user interface for setting up AnalyseSegments calculations
- Contingency Management Layer - Phase 1 thread management to add queuing to engineering functions on large (100m+) tables.
- Extension to System Variables: %CURRENT_DATE%
- Improved Error Handling in XProcess - errors that are written to the error file will be reported in the same way as engine errors.
- Various AMP+ Issues
API Calls
| Type | API | |
|---|---|---|
| MODIFIED | CreatePackage | if tables defined and table does not exist then error (AMP-287) |
| MODIFIED | InjectPackage | packages will have an injectionId assigned (but optionally passed as 'instanceId':string). This id is stored in the definition of all injected objects. FieldTemplate objects can be imported and accessed as Template objects |
| MODIFIED | AnalyseSegments | "target_nontarget" mode outputs universe as count of individual records when outputting to Excel |
| NEW | AddNullField | Allows a null field of specified data type to be created |
| NEW | AddNullFieldLike | Allows a null field to be created where type matches another field |
| MODIFIED | CopyDownFromTemplate | now collects subordinate errors. |
| NEW | CopyDatasetCollection | copies a dataset collection to a linked table |
| NEW | PreProcessSegments | Pre-processes *.tsv files to produce *.tsv.hash and *.tsv.hash.seg files. Once processed, *.tsv files can be removed, reducing required disk space to ~1/3 original disk requirements |
| MODIFIED | ProcessSegments | Can now process *.tsv.hash and *.tsv.hash.seg files. Processing time is reduced by half. Query times are improved Additional metrics are generated - number of times a hash (unique SHA256) appears in the input data, total number of segments in which hash appears, unique number of segments in which hash appears. |
| MODIFIED | CreateTableFromFile |
|
AMP+ Issues Resolved in this release
- DM-447 / AMP-64 - CreateTableFromFile corrupts data when there is a tab - tabs replaced with spaces in file stream when delimiter is not tab
- DM-487 / AMP-78 - Modify InjectPackage so that FieldTemplate objects can be imported and accessed as Template objects.
- DM-589 / AMP-81 - Auto-guess breaks if single mismatched quote - auto-guess program modified so it can be cancelled. Return automatically if line exceeds a specified length.
- DM-680 / AMP-216 - Issues with nulls in Decodes (nulls not being included in dialog) - the null row is now included in the decode dialog
- DM-840 / AMP-267 -Large Load "No line terminal in file" - filename has been added to error message
- DM-918 / AMP-290 - CopyUpFromTemplate does not support datasets (Function does not support universal and individual datasets)
- DM-933 / AMP-295 - AnalyseSegments not displaying target amount properly in output - correct value now displayed
- DM-927 / AMP-296 - Add ability to verify entire project, not just single table or field - use command verify in command line to iterate through all project objects
- DM-900 / AMP-287 - CreatePackage allows a non-existent table to be specified in the table list - CreatePackage fails with an error if table doesn't exist
- DM-934 / AMP 308 - Allow text to be pasted in to Add Dataset Row dialog for IN operator when dataset is first created
- DM-930 / AMP-310 - Postman not sending emails after upgrading to unencrypted djserver.cnfg - new email module implemented and documentation updated.
- DM-943 / AMP-315 - Ability to create sha256 as base64
- DM-938 / AMP-316 - Eyeota-prd01 recursive fail in Export Method
- DM-939 / AMP-317 - yearmonth() expression - INCORRECT RESULTS when source field is DATETIME
- DM-944 / AMP-319 - Improve error handling in XProcess - XProcess now returns an error if an error has been added to the error handling file.
- DM-956 / AMP 323 - error in stored script causes project to remain locked
- DM-962 / AMP 329 - RenameTable fails with "field not found" after doing BulkDecode - fixed
Other Issues Resolved in this release
- DM 936 - Templates not refreshed on open/close project
- DM 925 - Function: AddNullField, AddNullFieldLike - allows null fields to be created, to assist with table merges.
- DM 919 - Venn : Values in dragged segment do not always have the correct filter applied
- DM 917 - MULTI-FIELD STATS - Ability to reset report/clear all fields without needed to open a new report
- DM 916 - VENN - When dragging from Counts Grid, can drag source or resolution table records
- DM 914 - Project Explorer: Add ability to view selected dashboard in Discovery - right-click report, select "View Report in Discovery"
- DM 904 - MULTI-FUNCTION PROFILE (MFP) - Desktop allows string fields to be added to MFP setup grid and counted (e.g, counting cities in a zip code)
- DM 895 - OVERLAP - Once a segment has been dragged from grid, can't select grid scroll bar without triggering select
- DM 893 - AnalyseSegments - When dragging a dataset from Data Table Viewer Grid, give the dataset a useful name
- DM 891 - DATASET COLLECTIONS - Individual datasets can be dragged from the collection dialog
- DM 868 - TEMPLATES - Fields that have been hidden using Project Explorer | Show Hide Fields will not be available in Template dialogs or Dataset Row dialogs
- DM 867 - DATASET TAB - Datasets and Collections can be searched for by name
- DM 855 - REPORTS TAB - Data Table mode output is accessible from the Reports | Models tab in Project Explorer