- Overview
- Getting Started
- Training Series
- Quick Snippets
- Quick Questions
- Quick Questions
- QQ - Data Ingestion
- QQ - Engineering
- QQ - How do I change a discrete string to continuous?
- QQ - How do I sort a table?
- QQ - What is the difference between ICAST and WILDICAST?
- QQ - How do I flag records that contain numeric data for a specific field?
- QQ - How do I transform data loaded as string into a double?
- QQ - How do I create a join between a field that is continuous on one table and discrete on the other?
- QQ - How do I use the current date or time in expressions and calculations?
- QQ - How do I get the name of the nearest city from a table of Geo Data?
- QQ - How do I use Conditional and IF statements in DataJet?
- QQ - How do I get detailed statistics for continuous data fields?
- QQ - How do I dynamically create a banded field in script?
- QQ - How do I round up a decimal number to 1 decimal place?
- QQ - How do I rename a table?
- QQ - How do I cast between different data types?
- QQ - General
- QQ - How do I find a package file to download?
- QQ - Why is the record count in the tree one less than the count in the hotbar?
- QQ - Why is the last record missing from my table?
- QQ - What is the difference between a DataSet and a DataView?
- QQ - How do I view NULL counts for continuous fields?
- QQ - How do I select the "best" record according to some numeric score?
- QQ - How do I download large source files from the DataJet Server?
- QQ - How do I find out the cardinality of a continuous field?
- QQ - Modelling
- QQ - Reporting
- QQ - Why isn't my Standardized Report applying all filters to configured slots?
- QQ - Why are my crosstab columns in the wrong order?
- QQ - How do I add a Standardized Report to my script?
- QQ - Can I automatically display a specified report when a project loads?
- QQ - How do I add a report created using Discrete Statistics to script?
- QQ - How do I edit a Standardized Report?
- QQ - How do I create a scatter plot?
- QQ - Scripting
- Rapid Training Series
- Introduction To DataJet User Interface
- Creating a new project from a text file
- Introduction to DataSets and DataViews
- Combining single Profile reports into a Multi-Function Profile
- Creating a Simple Multi-Function Profile
- Basic Engineering Functions
- Creating connected reports from simple index profiles
- Database Updating
- DataSet Resolution Methods
- DataView and Templates
- Engineered Fields and First Discrete
- Loading multiple tables and data audit
- Multi-Function Profiles
- Creating Resolving Tables
- Using a MySQL data source
- Introduction to the DataSet HotBar
- Creating simple profiles and Tab navigation
- Using the DataJet Administrator
- Tutorials
- DJDemo2023 Demo Database Schema
- Create DJDemo2023 Project
- Basic Sales Dashboard
- Tutorial overview
- Step 1 - Prepare and Upload Data
- Step 2 - Create Project and Load Data
- Step 3 - Create Reporting Tables
- Step 4a - Prepare Data for Reporting - Decodes
- Step 4b - Prepare Data for Reporting - Create Reporting Dimensions & Metrics
- Step 4c - Create Customer Segmentation
- Step 5 - Create Dashboard
- Step 6a - Create Reports - Profiles & Multi-Function Profiles
- Step 6b - Create Reports - Panel Forms
- Step 6c - Create Reports - Profiles Continued
- Step 7 - Create and Publish Standardized Report
- Deep Dive
- How To
- DataJet Desktop
- Desktop Overview
- Project
- Analytics
- Introduction to Analytics
- Analytics - Functional Overview
- Combined Profile
- Analyze Field
- Connected Report
- Dashboards
- Data Audit Updated
- DataSet Scratch
- Index Profile
- Multi-Field Profile
- Multi Function Profile
- Multi-Field Statistics
- Query Matrix
- Profile
- Statistics
- Venn
- Reporting
- Engineering
- Introduction to Engineering Data
- Aggregate
- Banded Fields - Custom
- Copy Down (P-F)
- Copy Up (F->P)
- Fields from Values
- Decode Field - Custom
- Decode Fields - Wild
- Expression Builder
- Functions
- Function Overview
- General
- CombinedDimension
- CombinedDimensionNS
- CombinedIndex
- CompositeLongKey
- CopyDiscrete
- CopyDiscreteIndex
- DescendingIndex
- DiscreteIntegerSequence
- DistinctByKey
- FirstDiscrete
- FirstDiscreteByIndex
- FirstDiscreteByInvertedIndex
- IntegerSequenceKey
- JoinLayers
- LongSequenceKey
- MakeContinuousKey
- Quantile
- SessionIdentifier
- DateTime Transforms
- Date Transforms
- Date Transform Overview
- ABSDAYSDIFF(A,B)
- AGE(A)
- AGE(A,B)
- ANNIVERSARYDAYS(A,B)
- DADD(A,B)
- DATE(A)
- DATEFROM(A,B,C)
- DAY(A)
- DAYSAFTERMIN(A)
- DAYSBEFOREMAX(A)
- DAYSDIFF(A,B)
- DAYSDIFF_MXONB(A)
- DOY(A)
- MAX(A,B)
- MIN(A,B)
- MONTH(A)
- SMONTH(A)
- SWEEKDAY(A)
- WEEKDAY(A)
- WEEKOF(A,B)
- WOY(A)
- YEAR(A)
- YEARMONTH(A)
- YEARQTR(A)
- YEARWOY(A)
- DMath
- FLogic
- Geo
- LMath
- Statistics
- String
- Import List
- Joins
- Make Key Table
- Model Processor
- Extensions
- Function Field
- Waterfall
- User Defined Fields
- Tools
- Script
- Introduction to Scripting
- Script Editor
- Script Editor - Overview
- Script Hub - Overview
- File
- Methods
- Administration
- Audit
- Config
- Control
- Connections
- Data Model
- AddJoin
- AddJoinLayer
- CopySourceTable
- CreateKeyTable
- CreateTableFromFile
- CreateTableFromWorkbook
- CreateTableFromWorkbooks
- DeleteField
- DeleteTable
- DeleteTemplate
- DetachAllTables
- DetachTable
- InjectTable
- InjectTables
- LoadTableFromFile
- MapFieldNames
- MapFieldNamesFromTable
- MapMissingFields
- MapMissingFieldsFromTable
- MoveFields
- RenameTable
- RequiredFields
- RequiredFieldsFromTable
- SetHiddenTables
- Engineering
- Export
- General
- Logging
- Tools
- Internal Only
- Project
- Tools
- Importing
- Examples
- Misc
- Admin
- Misc
- Intrinsic Functions
- Settings
- Trouble-shooting
- Indications of Disk Space issues
- Error [Join does not verify] when joining injected tables
- DataJet Explorer stuck on "opening project" message
- Performance and shutdown issues on Windows 11 22H2
- Error [load: not allowed] when loading data from file
- Modelling menu does not contain Helper menu and sub-menu
- Template Resolution Failed
- Plugins
- Engineering Functions
- Field and Table Names
- Post Functions
- Regex
- System Variables
- in Development
- Inline Help
- Download
- DataJet Server
- Campaign Selection System
- Audience Solution and DataHub
- API
- AnalyseSegments Updated
- AudienceDelete
- AudienceLS
- AudiencePush
- AudienceRename
- BulkExportAudiences
- CreateAudiencesFromWorkbook
- CreateSegmentModel New
- CountCampaignDataset
- CreateCampaignDataSet
- ExportAudience
- ExportAudienceDefinitions
- ExportAudienceRules
- GetCampaignHistory
- ImportAudienceDefinitions
- NotificationsLS
- ProcessSegments
- SetAudienceState
- Release Notes
- Release Note: DataJet v7.2.17.1 Updated
- Release Note: PATCH 6.7.26.2
- Release Note: PATCH 6.8.7.1
- Release Note: DataJet v6.7.26 - 29/07/2024
- Release Note: DataJet v6.7.1 (1/July/2024)
- Release Note: DataJet v6.5.23 - 23/05/2024
- Release Note: Q1 Patch 1: v6.03.28
- Release Note: v6.02.20
- Breaking Changes
- Release Note: DataJet v5.11.27 - 27/11/2023
- Release Note: DataJet v5.10.19 20/10/2023
- Release Note: DataJet v5.9.22
- Release Note: DataJet v5.8.16
- Release Note: DataJet v5.7.18.1 20/07/2023
- Release Note: DataJet v5.6.30
- Release Note: DataJet v5.5.13.1 - 13/05/2023
- Quick Vids
- Coming Soon
- 3 Minutes to read
- Print
- DarkLight
- PDF
Deep Dive - Sharing Data by Injection
- 3 Minutes to read
- Print
- DarkLight
- PDF
Article In Progress
PROTOTYPE
Overview
What is injected data?
Injecting data allows data to be shared across projects and realms. Depending on system architecture, data may be injected across:
- projects,
- multiple realms on the same server,
- multiple DJ servers.
Injecting data avoids loading multiple instances of the same dataset. Using this technique, multiple projects can use the same data source.
Features
Once the injection has completed, the injected table will appear in the target project. Injected tables are green, and are read-only within the target project.
Example: sales table injected from Demonstration project into BikeData2 project:
Notes:
- Deleting the injected table in the source project will also remove it from all target projects
- To remove an injected table, right-click the table in Database Explorer and choose "detach". Alternatively use the DetachTable API.
- It is possible to create derived data on an injected table within a target project.
- Derived data that has been created on an injected table will remain on the table even after the table has been detached
- Additions and changes to the source table will NOT be reflected in the injected table. It is important to have a process in place that prevents source data from being modified once it has been injected in another project.
Approach
Whichever injection paradigm is used, the overall approach will be the same:
- Prepare Source Data
- Create, Load, Engineer
- Create Injection Packages (if injecting across realm or server)
- Create a source project - make sure data files are accessible from Target machines.
- Backup method
- CreatePackage method
- Prepare Target Project to receive data
- Create Project
- Inject Data into Target Project
- InjectTable - Injects a single table into a project on the source realm
- InjectTables - Injects a set of tables into a project on the source realm
- InjectPackage - Injects a set of tables, joins and reports into a project on source realm or other connected realms
- Coming soon... InjectFromWorkbook
- Coming soon... InjectReports
Scripting Methods
CreatePackage
This will create a package file that can be injected into a target project
{
"method": "CreatePackage",
"path": "%DATAPATH%shareddata/Package_05_withmetadata.zip",
"system": false,
"includeProjectMetaData": true,
"targetProject": "Package_05",
"tables": [
"DATA_S1",
"DATA_S2",
"DATA_S3"
],
"project": "CreatePackage_05"
}
InjectTable
This will inject a single table from a source project into a target project. Both source and target projects must be on the same realm. See InjectTable for further details.
{
"method": "InjectTable",
"description": "Inject Data From Demonstration into BikeData2",
"sourceProject": "Demonstration",
"sourceTable": "Sales",
"targetProject": "BikeData2",
"targetTable": "sales",
"project": "BikeData2"
}
InjectTables
This will inject a set of tables from a source project into a target project. Both source and target projects must be on the same realm. See InjectTables for further details.
{
"method": "InjectTables",
"project": "test",
"sourceProject": "DataSource_01",
"sourceTables": [
"Customers",
"Transactions"
],
"targetTables": [
"Cust_DS01",
"Trans_DS01"
],
"targetProject": "AllDataSource_01",
"includeJoins": true
}
InjectPackage
InjectPackage will inject a static set of Data Objects and Data into one or more projects. InjectPackage works across:
- Projects on the same realm
- Projects on the same server, but on different realms
- Projects on different servers (and hence different realms)
The process is as follows:
- Prepare Data -create a project containing only the data that is to be injected
- Create the package and make sur the project data is accessible from all target realms
- Run InjectPackage on the target realm
See InjectPackage for more details
{
"method": "InjectPackage",
"project": "activeproject",
"path": "%DATAPATH%sharedbackup/realm1_packageABC.zip",
"homePath": "%DATAPATH%shareddata",
"targetProject": "ClientA"
}
Methods
- Injecting tables within a realm
- Injecting a package within a realm
- Injecting Packages across realms.
Injecting tables within a realm
To inject a table from one project to another, where both projects are within the same realm, the InjectTable or InjectTables methods can be used. Note that this method is injecting "live" project data into a target project - it is possible for the source data to be modified if processes are not in place to prevent this.
Injecting a package within a realm
To inject a table across realms on the same server, use the InjectPackage methos.
Injecting Packages across realms
To inject a package across realms , first make sure that the project data that is to be injected (i.e., the loaded project files *.uvl, *.dat) are accessible from all realms which will be using the package. This may involve moving or copying data, depending on the various realm configurations. Then use InjectPackage on the target realm.
Step 1: CreatePackage
Create package on the source machine/realm:
{
"method": "CreatePackage",
"path": "%DATAPATH%shareddata/Package_05_withmetadata.zip",
"system": false,
"includeProjectMetaData": true,
"targetProject": "Package_05",
"tables": [
"DATA_S1",
"DATA_S2",
"DATA_S3"
],
"project": "CreatePackage_05"
}
Step 2: Make sure data is accessible.
The data in project "Package_05" must be visible from the target machine, and have all the necessary permissions. In this instance, the data will be in
- /home/repos/realm-name/Package_05
And the package file will be at:
- "%DATAPATH%shareddata/Package_05_withmetadata.zip"
Let's assume that a drive on the target machine has been mapped to this location, and is configured as:
- /home/datasources/shareddata/Package_05
These will give the following parameters for InjectPackage:
- "path" : "%DATAPATH%shareddata/Package_05_withmetadata.zip"
- "homepath" : "%DATAPATH%shareddata/Package_05"
Step 3: InjectPackage
Run the following on the target machine/realm:
{
"method": "InjectPackage",
"path": "%DATAPATH%shareddata/Package_05_withmetadata.zip",
"homePath": "%DATAPATH%shareddata/Package_05",
"targetProject": "Inj_Package05",
"description": "Inject Package_05",
"sourceName": "dev01-Package_05",
"includeProjectMetaData": true,
"project": "inj_Package05"
}