QQ - How do I load multiple files into the same table?

Two options are available:

1. Load each file by name
2. Use Bulk import and load all files in a folder

Both options assume that all files have identical field definitions and data-types.

Option 1 - Load each file by name

Set up a CreateTableFromFile method using one of the following methods:

• Script Editor| Import Data | Delimited File  
• Script Editor | Methods | DataModel | CreateTableFromFile
• Project | Quick Load | Add Table From File

Select the first file to import.  

Configure the import using either the dialog/edit option (recommended) or by editing the JSON in the JSON panel. (Make sure field definitions are suitable for the finished table - if the initial file doesn't contain much data, DataSize may not be sufficient)

Select the second file to import, and make sure the definition matches that of the first file - including the table name!

In the JSON panel of the second file, right-click and choose "Convert To Load".   This will turn CreateTableFromFile into action: "Load" and will append data to the existing table.

{
  "action": "LOAD",
  "table": "TestTable",
  "filename": "%DATAPATH%Demo_Modelling/BikeData/BikeData/01aJourneyDataExtract10Jan16-23Jan16.csv",
  "sample": false,
  "preHeaderSkip": 0,
  "dateFormat": "DD-MM-YYYY",
  "fileFormat": "ASCII8",
  "skipFirstLine": true,
  "delimiter": "COMMA",
  "stripCharacter": "",
  "dateTimeFormat": "DD-MM-YYYY HH:MM",
  "project": "TEST_5",
  "method": "LoadTableFromFile",
  "sourceProject": "",
  "sourceFields": [
    "Rental Id",
    "Duration",
    "Bike Id",
    "End Date",
    "EndStation Id",
    "EndStation Name",
    "Start Date",
    "StartStation Id",
    "StartStation Name"
  ],
  "fields": [
    "Rental Id",
    "Duration",
    "Bike Id",
    "End Date",
    "EndStation Id",
    "EndStation Name",
    "Start Date",
    "StartStation Id",
    "StartStation Name"
  ]
}

Option 2 - Use Bulk import and load all files in a folder

Note: To use this method, all files must be of the same format and must be in a folder that contains only the files to be loaded into the table.

Set up a CreateTableFromFile method using one of the following methods:

• Script Editor| Import Data | Delimited File  
• Script Editor | Methods | DataModel | CreateTableFromFile
• Project | Quick Load | Add Table From File

Select the first file to import.  

Configure the import using either the dialog/edit option (recommended) or by editing the JSON in the JSON panel. (Make sure field definitions are suitable for the finished table - if the initial file doesn't contain much data, DataSize may not be sufficient)

Save the changes, and in the JSON panel, change the shortfilename to *

• "filename": "%DATAPATH%Demo_Modelling/BikeData/01aJourneyDataExtract10Jan16-23Jan16.csv",

becomes

• "filename": "%DATAPATH%Demo_Modelling/BikeData/*",

This will then load all files in the folder into the same table.