Überblick Anhang Release Notes

Release Notes

datasqill 4.1.1

March 26, 2025

backend:
- SQDV now fully supported for h2, oracle and postgres
- statementsIfNotExists can now be used in dsdb files to execute database statements if a specifig object does not exist
GUI
- NullpointerException in actiondialog.ActionCTL fixed

datasqill 4.1.0

March 13, 2025

All:
- Initial implementation for using https
- Preparation for elemination of direct access to the repository database
- New proxy jdbc driver. Should be added to all repository connections: Use instead of
  
  SQTS_DB_0 jdbc:h2:...
  
  SQTS_DB_0 jdbc:dsproxy:h2:...
- Enhancement of postgres statistics to gather sdqv row counts
backend:
- Bug fix while handling decimals in mysql and sqlserver
- Detection if sqlserver is case sensitive or not
- New system property to disable the automatic scheduling: set de.softquadrat.datasqill.scheduler.enabled to false
- Dependenccies for Vertica, OpenEdge and Snowflake Databases added
- Insert into Oracle database: Convert to Number if target column is number
modules:
- DsModRemote and DsModTemplate: If the action is empty and there is only one source database object, datasqill is generating a query which selects from the source all columns which are in source and target
- DsModWebServiceClient: New connection parameter loopDelay introduced
- DsModWebServiceClient: DELETE method added
- DsModLoadFlatFile: added new attribute "Line separator"
- DsModTemplate: Snowflake in history modul added
- DsModWriteExcel: Date handling improved
- DsModTemplate: New Module sync from source to target to be used with trigger based history
GUI
- Connections
  - Default values for trigger are taken now by the coresponding connection type.
  - Statement sets can be added by GUI.
  - Most changes in connection tabels are revertible before save.
  - Testing the connections to the database and the rest server is now done in one action with one button.
- Documentation
  - In read only mode only the rendered markdown is shown in the documentation pane.
  - External links in markdown web view open the browser.
- Action and Object Editor
  - New objects or actions get the name '-' avoiding empty names.
  - Adjust window size of an action to the size of the display.
  - New context menu entry for objects added: "Copy All SQL Scripts".
  - Moved Documentation of action in extra docu tab the attributes on action editor have more place.
- Left Tree
  - Sheets can be cloned now. 'name of origninal sheet + - COPY' is used as pre selection of the new name.
  - Refactoring and improvement of navigation functionality
  - Improvment of drag and drop on left tree. Transfer of drag item is not allowed to different environments.
  - Move of folder to its children is not allowed.
- Runs
  - Popover of showing run statics on run items replaced by a tooltip.
- Batches
  - Filter of Batches now possible.
  - Improvement lazy loding of batch tree items.
  - Navigation to sheet from batch list.
  - Navigation to action from batch list.
- Bugs:
  - Canceling sheet export file chooser is not causing exceptions any more.
  - Wrong title of STDOUT/STDERR Dialog fixed.
  - Improvement and fixing of code related to escape key in several dialogs.
  - Multiple showing dialogs of the same exception avoided.
- General GUI Improvemnts
  - Application exit and restart can be enforced by user in case of unsaved changes.
  - "No" as default button on exit with unsaved changes."
  - Rework of startup communication and order for App and WorkbenchPreloader improved
  - The last visited directory by the file chooser is persisted in local preferences.
  - Long Errormessages are shown in a lazy loaded scrollpane.
  - Info-Tooltip for run elements is now lazy loaded.
  - Ctrl/Shortcut-W-HotKey can be used for closing tabs.
  - F1-Hotkey opens https://docs.datasqill.de/de.
  - Show '..'-Button when the column in run monitoring table is too wide or too long.

datasqill 4.0.0

February 13, 2024

All:
- upgrade to java 17
backend:
- datasqill server is now based on SpringBoot. TOMEE is no longer required and can deinstalled
- Error message can now be longer than 500 characters
- Log configuration moved from application.properties to datasqill.env
- deployment:
  - Fixed deletion of empty sheet during deployment
  - New option for the deployment to save the deployed Version no to the control table for versions
  - Use the newest version of deployed entity when updating the version table
  - Added functionality to use the deployment for own objects
modules:
- New module for call Salesforce Web Services (Sales cloud)
- DsModWriteExcel
  - limit the column size to 100 characters to avoid that the excel limit for column size is outside its bounds
- DsModSAP
  - added support for RFC_READ_TABLE without compression
- DsModTemplateSQL
  - History module supports now SQL Server
  - All templates of DsModTemplateSQL use now the quoted table and schema names
GUI:
- New Editor Monaco for action content and documentation
  - Syntax highlighting
  - Documentation in markdown
  - New Editor functions like mini map, zoom / font size, see more details on pages about the Monaco editor
- Documentation editor
  - Visualisation of rendered markdown
  - Live-Rendering of markdown during editing
  - Insert pictures into documentation

datasqill 3.5.8

February 5, 2023

modules: DsModRemote: bug fix for a close logic of dbWriter
GUI: bug fix for missing or wrong formatted Json config file as program argument

datasqill 3.5.7

January 16, 2023

modules: DsModWriteExcel: added new functionality to generate Excel without a template
GUI: bug fix for Windows path in logging properties file
GUI: additional logging properties introduced in local settings

datasqill 3.5.6

November 23, 2022

backend: for prepared statement use execute instead of executeUpdate
backend: bug fix of 3.5.5. Missing CAST added

datasqill 3.5.5

September 18, 2022

backend: added email notification(ERROR event), if a batch cannot be started
backend: preparation for h2 release 2.x
GUI: fixed the problem with enter key in text area

datasqill 3.5.4

July 31, 2022

backend: added action_id, action_name, module_name for default trigger variables
GUI: added copy Path to Sheet context menu
GUI: added copy path in action and copy fullname, copy select in object
GUI: NEW Batch overview Tab
GUI: changed Batch editor Tab layout
GUI: added edit information for Connection Tab

datasqill 3.5.3

March 09, 2022

GUI: Preparation for using a batch as a batch element

datasqill 3.5.2

Februar 10, 2022

backend / GUI: Preparation for h2 Version 2.1
backend / GUI: Preparation for java 17
modules: DsLoadFlatfile:
- archive processed files only
- added variable connection_id and target_connection_id for trigger
modules: DsModExecProcedure added variable connection_id and target_connection_id for trigger
modules: DsModWebServiceClient: added variable connection_id and target_connection_id for trigger
modules: DsModWriteExcel: added new functionality to replace / exchange a sheet in existing Excel
modules: DsModSendMail: corrected the senders domain address
modules: DsModRunShell: Added loop query to select parameters from database und run the script per returned row
scheduler: CPPS(Critical path based scheduling) algorithm added. Thanks to Sergey with his master thesis

backend: deploySheet.sh new option to deploy a sheet as new (duplicate the sheet and the content with get new IDs)

The used ids will be renumbered with numbers of the sequence of the target database.
--> Target must be the development database / environment, because the sequence for object-ids is needed
deploySheet.sh --option '{"validate":"true", "intoFolder":target-folder-id}' < sheet-file

GUI: Enable batches as batch element. Capturing not enabled
GUI: cancel confirmation dialogs: make them all ask the same question and act the same way
GUI: Settings/Connections completly reworked:
- new Statement set editor
- new Trigger editor for Database and Database Types
- edit items via popup window
GUI: Action editor added Trigger editor
GUI: In batch element editor set default priority to NULL
GUI: On sheet Tab added right mouse funtionality "Go to path". Moves the focus in the navigation tree to the sheet
GUI: Added right click on database object to find usage of this object

datasqill 3.5.1

September 30, 2021

module: Trigger implemented in DsModExecProcedure, DsModSendMail and DsModWebServiceClient
module: DsModWriteExcel: add a validation for target and template directory
backend: simplified upgrade of datasqill versions

datasqill 3.5

June 16, 2021

backend: major rework of file system structure:
- directory structure modified. ~/apps removed
- all jar-files are placed in ~/lib
- all jdbc files in ~/lib/jdbc
- central setting of environment settings in bin/datasqill.env
- files like datasqill-server/conf/server.xml datasqill-server/bin/setenv.sh etc. are no longer customized
backend: New tables for modul trigger. All database specific commands are now in this trigger tables (Konzepte > Trigger)
scheduler:
- added partition dependency
- added functionality to have dependency of virtual objects between sheets. Added an attribute in sqts_object_type to enable this
- object type 1 (database table / view) and object type 2 (global virtual object) set to enable intersheet dependency
datasqill-api: renamed to datasqill-api.jar and includes required libraries in correct version
module: Java Modul Interface simplified. All Java modules now use bin/runJavaModule.sh
module: Java Moduls fire module trigger. Currently implemented in DsModTemplate and DsModRemote
module: DsModSendMail
- added attribute "Mail Body Mime Type"
module: DsModTemplate
- History module can now handle "ignore" and "inplace" columns
- History module now works with Oracle

datasqill 3.4.7

February 12, 2021

backend: Create unique typeNames for Oracle by adding current time in milliseconds
backend: Improved Logging on modulelibrary
backend: Bug Fix for write to SQTS_REQUST_RESUL of long String(>4k)
backend: datasqillDBConnection:
- If supportsIdentifierCase is set to false in the connection all identifiers are used in lower case
- If the database does not support transactions, all transaction statements are ignored (commit, rollback, setAutoCommit)
- Error message if a key cannot be found with mappertool / getkey
backend: Singleton handling in ConnectionReader for testing purposes improved.
backend: Exception thrown instead of "false" return value on reading connection data by key.
backend: datasqillDBValidate use more often try with resource, do a rollback in exasol dependencies and use unique view names
backend: draft version for testing a connection are integrated into datasqill library
backend: mappertool:
- Command keyfile simplified and usage of GLOBAL_KEYFILE removed
- New order for specifying a keyfile. Environment Variable KEYFILE added for this purpose
module: DsModMetaData:
- Logging can now be set on module level for this module
- Improved exception handling for wrong DB
module: DsModWriteExcel:
- Bug Fix for suppressEmptySheet condition
- Bug Fix with templateRow (it was not possible to insert data into excel on first cell)
module: DsModTemplate:
- Make use of global variable to do some activity after writing into a table
- Bug Fix with time format of module_start-dates
- Logging improved in loops
module: DsModRemote: use several schemata from keyfile
module: DsModWebServiceClient:
- added set for custom RequestProperty
- Logging improved
module: First beta of history module for postgres and exasol

datasqill 3.4.6

December 06, 2020

backend: Resolve synonym columns in Oracle
backend: Detect nullable columns of target tables in Exasol
backend: Removed Column next_start_dt from tv_sqts_batch table
backend: Null pointer during deployment fixed
backend: Catch error when setting autocommit to false. Some lazy drivers do not support the setting of autocommit, because the driver has no autocommit implementation
backend: New logging format for all services
backend: Repository connection pool is now using the keyfile to retrieve user and password
backend: Moved Connection pool from tomee.xml to resources.xml (included in datasqill war file)
backend: The scheduluer process is closing every hour the repository connection and purges all idle connection of the connection pool
backend: The scheduluer does not use prevIsScheduled and prevIsMissing if the dependency is to the same batch
backend: Avoid Merge statement, because of memory leak in h2
modules: DsModExecProcedure, DsModRemote, DsModTemplateSQL: Make sure that all database connections are closed
module: DsModTemplateSQL:
- When comparing nullable columns with Exasol use (a = b OR A IS NULL AND b IS NULL) instead of DECODE(a, b, 1, 0) = 1. According to Exasol the Joins will be faster
- Prepared to generate several statements
- Use freemarker when validating Loop and Main Query to allow usage of variables
module: DsModLoadFlatFile:
- Logging can now be set on module level for this module
- Additional Attribute for moving the file after successful load to an archive directory
- Allow to switch between regular expressions and file wildcard syntax, when selecting files
- Make use of global Variable home_directory (set in vv_sqts_config or vv_sqts_config_global). If set all source paths are relative to home_directory
module: DsModWebServiceClient:
- Enabled to set https proxy host and port. In keyfile the variables https.proxyHost and https.proxyPort can be set. If both is given the https proxy will be used
- Logging can now be set on module level for this module
- New functionality to use basic authentication if a password is given. The password must be encoded
GUI: Changes to trans db connections via setting dialog get effective without restart

datasqill 3.4.5

September 09, 2020

backend: Bug Fix for batch dependency conditions
backend: Bug Fix for load all dependencies views in ActionDependency
backend: Use additional action attribute to get the newest change in action
module: Bug Fix for the library to get metadata of a query
module: Bug Fix for the library added setNull if the object is null. Required when writing in batches
module: DsModMetadata: Improvement when opening the source database
GUI: Bug Fix for array index exception in NumberTextField
GUI: TimeTextField in batch tab replaced by 2 NumberTextFields for hours and minutes
GUI: Improvement to calculation of timeOffsetProperty in Batch and handling of user input
GUI: new Icon

datasqill 3.4.4

July 24, 2020

backend: New Index to speed up insert of batch elements into the Queue
backend: New Index to speed up GUI when opening a batch run
backend: Bug fix: stay in loop if signal has been received
module: DsModRemote: Delete rows from target database using a selected rowset from source database
module: DsModRemote: Log amount of deleted rows
module: DsModRemote: Nullpointer exception fix with dummy databases
module: DsModTemplate: Use columns in insert order
module: DsModVirtual: Bugfix with path
module: New library function for database writer

datasqill 3.4.3

June 22, 2020

backend: Solved Problem with locking of batches by adding new table for next run date
GUI: Do no longer show several batch instances in one run view
GUI: Show logging after validate

datasqill 3.4.2

April 17, 2020

backend: validate now provides same variables as for execution
backend: improvements to install scripts
backend: validation during deployment now obeys sqdv_delete_flag
GUI: Sorting for monitoring of actions improved
GUI: Disabled viewing of all batches of a day / recent batches to avoid performance problems
module: DsModCompare: Use correct connection in case no reference query
module: DsModCheck: Commit disabled for hive
datasqill: New Member Alexander in datasqill Team

datasqill 3.4.1

April 7, 2020

backend: Fix for problem with deployment.
backend: Fix for problem when executing an Oracle procedure.
module: Fix in DsModWriteExcel for loop-query validation.
GUI: Fix for a security problem with workbench logging.
H2: improved resource cleanup; jar file changed

datasqill 3.4

March 27, 2020

GUI: Performance improvement for monitoring of runs
GUI: Channel output from "stdout", "stderr" and "log" added to validate action result.
GUI: Show path added to sheet tab as context menu item
GUI: New status "unknown" for objects in validation added
GUI: Double click on Recent Runs does no longer open all entries
GUI: Folder and sheet items in transformation sub tree now displayed with id
GUI: Bug fix for missing hot keys (Select All, Copy, Paste) on sheet after click on grid
GUI: Bug fix for reload sheet with no changes after save
GUI: Performance improvement on first load of Object dialog
module: DsModExecProcedure can now run Exasol scripts
module: DsModSendMail debugging information added
module: DsModTemplateSQL enabled for looping queries
backend: DatasqillMail debugging information added. Add in datasqill-server/conf/logging.properties a line containing "de.softquadrat.datasqill.mail.DatasqillMail.level = FINE" to enable debug output
backend: Allow several schema-variables per connection, i.e. schemaList={ds=DATASQILL,sq=SOFTQUADRAT}
backend: Fixed problem with open cursors in SheetBuilder
backend: Switched to use connection pool for services and scheduler. Tomee no longer uses new connection with getkey
backend: Problem "validate sheet locks other service activities" solved
services: Timeout for long webservice calls extended to one hour
scheduler: Made scheduler more robust and retry with reconnect to database

datasqill 3.3.2

February 17, 2020

modules: DsModTemplate use variables
modules: logging improved
backend: Implementation of variable manager enhanced
backend: deploySchema now via webservice
backend: First steps to remove nextstartdate from batch table
services: ValidateSheet now with given sheet content
validate: Use current schema in Exasol

datasqill 3.3.1

December 31, 2019

GUI: Added gitCommitDate and gitRevision
GUI: New logo added to splash and about dialogs
GUI: Save Folders with and without content
backend: DeploySheet reworked

datasqill 3.3.0

December 16, 2019

GUI: Active keepalive to database
GUI: Edit connections and schema
GUI: Tooltip now yellow with line breaks
modules: msserver dependencies and validate fixed. Position of error can no longer be delivered
modules: postgres dependencies enhanced
modules: First templates for insert, merge, update, time series
modules: central function to prepare a call to a database procedure
modules: Freemarker introduced to DsModTemplate
modules: Allow SAP access in loops
backend: Views reduced to minimum requirements of GUI. Tempspace and statistical versioning columns removed from queue view
services: Validate bug fix
services: New variable manager in experimental state
services: bug fix of calculation if frequency for batch interval was set to 0
scheduler: dont restart a batch if one job was killed manually

datasqill 3.2.2

September 09, 2019

GUI: Cut between Config and Transformation DB improved
GUI: Set user name and ID in database (used by trigger)
modules: DsModSendMail, NPE in case value for placeholder is NULL
modules: New Module to execute a stored procedure in the database. Currently released for sqlserver
modules: Release allocated CLOBs in Module DsModRemote. Required if the loop contains a lot rows
modules: further DSRT functions for Bulk write to oracle db
backend: Generalized runtimeProcedure to use all customized function in pg_dsrt_interface
backend: hive enabled
services: Enable Restart Batch if batch instance is in status interrupted
services: Validate reworked

datasqill 3.2.0

July 15, 2019

GUI: New locking for H2
GUI: Show Version in Title removed
modules: snowflake enabled
modules: DsModRunShell now with parameter
modules: mysql enabled
modules: mssql enabled
modules: DsModTemplate can handle quoted table names
backend: new deployment format for database objects in hjson
backend: Use everywhere new table sqts_batch_instance_dependency
backend: First releases for deploySchema and deploySheet
backend: performance improvement for monitoring batches
H2: Trigger for versioned datasqill tables
H2: start/stop scripts
H2: Full support as repository database

datasqill 3.1.0

May 17, 2019

GUI: Show Version in Title
GUI: Try one reconnect after broken connection
GUI: Memory leaks in SQL Area fixed
GUI: New menu with developer tools (Get Heap Dump, Show Memory Usage, Run Garbage Collection
GUI: About dialog enhanced
GUI: Bug fix for double click on tabs and splitpane
modules: New module app, which allows to generate in Database Mappings from SQL-Templates with first module for insert from select
backend: git backend Folder structure reworked
backend: New backend install
backend: Modules selectable per customer
server: Nano HTTP removed
server: Revision: Adjust tomme server port to datasqill port 17491
services: Capture system exit calls
services: Persist dependencies to table sqts_batch_instance_dependency
services: Process control of child processes improved (kill command)

datasqill 3.0

January 31, 2019

First complete Version with java FX and H2 experimental