Backup and Restore

For backing up data in the datasqill repository, there are various methods depending on the use case. Data to be backed up can include:

  • Transformation folders
  • Transformation sheets
  • Batch folders
  • Batches
  • Executed transformations (runs)
  • Settings

Full Backup

With a full backup, the entire repository is backed up.

The datasqill repository is an H2 database. Under this link you will find instructions for backing up and restoring an H2 database.

A backup script is located on the datasqill server under

h2 backup

It creates a backup file with timestamp that is placed in the "backup" directory:

$ h2 backup
BACKUP to 'backup/BACKUP_20220313_1827.zip';

It is recommended to create such backups regularly, for example by generating a daily backup with the help of a cron job. A cron job could look like this:

crontab -l
...
# Do a backup every day at 22:30
30 22 * * * . ./bin/h2.env && ./bin/h2 backup
...

So that the file system does not overflow, backups must be deleted regularly. Here a simple strategy via cron:

crontab -l
...
# Delete every Saturday at 22:40 backups which are older than 2 weeks
40 22 * * Sat find ./backup -maxdepth 1 -mtime +13 -type f -delete
...

To restore such a backup, the H2 database must first be shut down.

Then the zip file with the backup can be unpacked and copied into the H2 directory.

unzip backup/BACKUP_20220313_1827.zip
cp backup/datasqill.mv.db h2server/data

Then the H2 database is started again.

Backup of Transformations

Instead of the entire repository, individual transformation sheets can also be backed up and restored.

The export function in the datasqill GUI serves to backup a single sheet. An export can be triggered via the context menu in the navigation menu.

Export 1

Alternatively, the export button in the toolbar of an open sheet can be used.

Export 2

As a result, you get a datasqill export file, for example "sheet.dsr". This can be restored on a datasqill server using the "deploySheet" script. To do this, transfer it to the server and load it into the datasqill repository with the following command:

deploySheet.sh --option '{"validate":"true", "intoFolder":10135}' < sheet.dsr

The ID of the target folder must be specified (in the example 10135). This can be determined beforehand using the GUI; folders are displayed with their ID in the navigation tree.