From DataSource
- 1 Minute to read
- Print
- DarkLight
- PDF
From DataSource
- 1 Minute to read
- Print
- DarkLight
- PDF
Article summary
Did you find this summary helpful?
Thank you for your feedback
In Progress...
There are 3 mechanisms available for importing data directly from a DataSource:
- Single Table import (Script | Editor | Import Data | Data Source)
- Bulk Import (Script | Editor | Import Data | Bulk Import | Entire DataSource)
- Custom Importer (ProcessEx method)
The single table import dialog allows each table to be inspected individually, the bulk importer will create a definition for each table in the data source
Data Source Configuration
Before Data can be imported from a Data Source, the Data Source needs to be configured in Admin | Data Source Management.
Before Loading Data From a DataSource
Field Definitions
When importing from a DataSource, it's important to check field definitions before the import begins, otherwise load times can be very slow, or even fail.
In particular, for large tables, check:
In particular, for large tables, check:
- Very high cardinality fields (i.e., IDs, addresses, emails etc) are configured as CONTINUOUS
- CONTINUOUS strings must be FIXED WIDTH - i.e., provide a field width that is sufficient to load the data
- Use Import Data | Data Source to inspect individual table definitions and obtain string length information (if available for the Data Source)
Field Width for Continuous Strings
- Continuous strings must be a fixed width - no variable length
- Length is in bytes: "NAMESURNAME|CONTINUOUS|STRING|100",
- bytes = characters for ansii
- bytes != characters for unicode
See Script | Methods | RDBImporterEx for full details of how to import continuous strings from a Data Source
Samples
When loading data from a DataSource, do a SAMPLE LOAD and then run a Data Audit once the load has completed. This will allow the resulting data to be inspected and field definitions to be modified if necessary before loading the full data source.
Import Data | Data Source
Note: When inspecting large tables, it may take up to a minute for the sample data to load
Import Data | Bulk Loader | Entire DataSource
Was this article helpful?