Moving Projects across Realms

DataJet Realms are designed to operate as completely independent silos of data and activity.  To move data or projects across them is possible with the following process.

Create Backup Files Of Projects in Old Realm

This step will create a backup file for each project that is going to be moved or copied across realms.

1. Use DataJet Desktop Client to Login to the source realm (e.g., dj-oldrealm)
2. Create Backup Files for all projects to be moved or copied
   1. Using Script Editor
   2. Use "Backup" method for each project that is to be moved
   3. This will create a zip file for each project
   4. Put zip files into the root of the datasources directory so they can be easily found later
   5. Use the %DATAPATH% system variable in the output path
      JSONJSON

      {
        "method": ";Backup",
        "path": "%DATAPATH%BACKUPS",
        "description": "Backup Project1",
        "targetProject": "Project1",
        "project": "Moverealm"
      }

3. Use Desktop | Admin | Remote Files to Download zip files from DataSources to desktop

Create a SYSTEM BACKUP of the Old Realm

This step will create an archive of system objects such as Stored Scripts, Functions, Models and Snippets.

1. Use DataJet Desktop Client to Login to the source realm (e.g., dj-oldrealm)
2. Create a single SYSTEM Backup File by calling Backup with "system":true and "targetproject": ""
   JSONJSON

   {
     "method": "Backup",
     "path": "%DATAPATH%Backups",
     "description": "Backup dj-oldrealm System",
     "targetProject": "",
     "system": true,
     "project": "Moverealm"
   }

Commission New Realm

This step will add a new realm to the system

1. Use DJRealmAdministrator to commission the new Realm if it does not already exist
   1. This will be in format xx-realmname where xx is the company code.  eg., dj-newrealm.    
   2. Obtain the realm name.
   3. NOTE: Realm names must be unique within an organisation.   
   4. Ensure that necessary users have been set up with admin permissions for dj-newrealm
2. Test access to new realm - verify users can login and projects can be created.
3. Drop any temporary test projects.

Shut Down Server

This step will temporarily shut down the engine server in preparation for a manual modification to the repos folder on the new realm.

1. Make sure all users are off the new realm.
2. Stop the DataJet Engine Server

Copy or Move Data From New Realm to Old Realm

This step will move or copy the contents of the DataJet repository from the old to the new realm

1. Find the repository location of the existing realm.
   1. The data for the existing realm (dj-oldrealm) will usually be in a sub-directory of the datajet install.  e.g., [mnt/oldrealm...]/datajet/repos/dj-oldrealm
   2. The main repository location, if not the default, can be found in the DataJet server configuration file.
2. Copy the [mnt/oldrealm...] /datajet/repos/dj-oldrealm folder to [mnt/newrealm...] /datajet/repos/dj-newrealm
   1. NOTE: Once this process has been through testing within an organisation, it may be desirable to move the repository rather than copy it
   2. NOTE: Copying ...repos/dj-oldrealm to ...repos/dj-newrealm will copy all projects in the old realm. To copy or move a subset of projects, instead copy or move .../repos/dj-oldrealm/ProjectToCopy to .../repos/dj-newrealm/ProjectToCopy.  Do this for each project to be moved/copied

Restart Server

This step will restart the DataJet server

1. Start the Engine Server
   1. From bin directory
   2. sh start.sh
2. Verify that Mongo is running
   1. systemctl status mongod
3. Verify system starts as expected
4. Verify users have access
   

Import Projects to new Realm 

This step will add the projects that were backed up in the old realm to the mongo database in the new realm.   Call Restore for each project that is to be added to the new realm.

1. Use the desktop client to login to the new realm (e.g., dj-newrealm)
2. Use Desktop | Admin | Remote Files to upload *.zip files to the datasources folder 
   1. NOTE: This step may not be required if dj-newrealm is able to see the datasources folder of dj-oldrealm
3. Using Script Editor, Restore each project into the new realm by calling the "Restore" method
   JSONJSON

   {
     "method": "Restore",
     "path": "%DATAPATH%/dj-oldrealm_CreateTest.zip",
     "targetProject": "CreateTest",
     "description": "Restore CREATETEST",
     "project": "CreateTest"
   }

   
   
4. Test that projects have been correctly imported and that underlying data is as it should be by running a full Data Audit  (Desktop | Analytics | Data Audit)

Import System Objects into New Realm

This step will import system objects from the old realm into the new realm.   The system backup file contains the details of all system objects for the old realm.  Call the SystemRestore method once for each project that needs system objects restored.

1. Use the desktop client to login to the new realm (e.g., dj-newrealm)
2. Using Script Editor, Restore system objects into each project in the new realm by calling the "SystemRestore" method
   JSONJSON

   {
     "method": "SystemRestore",
     "path": "%DATAPATH%Backups/dj-oldrealm_20230512_144354_2.zip",
     "projectName": "project1",
     "project": "Moverealm",
     "description": "Restore system objects to project1"
   }