Skip to content

DataWORQ Release Notes

Last updated: 19 Jul 2024

Download DataWORQ

The DataWORQ Official Release is updated on a monthly basis after testing and documentation is updated. Updates can be downloaded through the QueBIT software portal at https://quebit.com/software-access/.

Releases

6.4.2 - 7/19/2024

  • Bug Fixes

  • Fixed Import/Export Bug

6.4.1 - 5/21/2024

  • Improvements

  • Flow Node Enhancements

    • Dynamic Run Profile selection
    • Parameter overrides
    • Full parameter support

    • Flow Control Nodes added

    • Condition Node

    • Supports conditional execution of nodes.

    • Uses parameters or upstream node values.

    • Stop Execution Node

    • Allows you to stop flow execution.

    • Intended to be used with the condition node.
    • Support for the Long data type to allow for large numbers.
    • Bug Fixes

  • PA nodes now support all valid characters in object names.

  • Fixed DataFrame sort issue in predictive nodes.
  • Fixed sorting of files in dialogs.
  • Accuracy node now properly leverages the ROLE column.
  • Fixed cut functionality for flows in the content menu.
  • UI selection now shows up properly in the content menus.

6.3.3 - 2/5/2024

  • Improvements

  • Improved responsiveness in the following areas

    • moving nodes on the canvas
    • performing actions on setting screens such as reordering branches in the run profiles menu
    • adding nodes to the canvas
    • moving nodes around via arrange, move branch up, down etc...
    • status issues where changes aren't detected during canvas move operations

    • DataWORQ Native Installer

    • Installer now includes the requests Python package by default

    • All installer Windows services are now set to Automatic startup

    • Run Profiles

    • Changed the "Add" run profile button to "Duplicate"

    • New Run Profiles are initialized with the settings of the active Run Profile

    • All Run Profiles can be activated/deactivated at once via a checkbox
    • Chart nodes no longer show up as runnable nodes

    • Updated the Pigment REST API calls internal to DataWORQ to the latest version

    • Only the most recent form of Pigment API keys are now supported

    • Node Improvements

    • Changing columns upstream of the Filter node is now handled more gracefully

    • Added the ability to refresh column names in the Reorder node
    • The Aggregate node now supports using String types for min/max aggregate functions

    • Updated Planning Analytics TI Process

    • Improved the standard Planning Analytics TI process to kick off flows when polling a long-running job

    • Removed the pWait parameter
    • 202 is now a valid response code for checking job status via the REST API

    • Business Rules node enhancements

    • Added support for parameters to certain input fields

    • Improved error messaging when using unquoted strings as inputs

      • Output Column Name
      • Default Rule
      • Name
      • Business Purpose
      • Custom Expression
      • Rule Result
      • General

    • Node indicators can be enabled/disabled via Application Settings. They are now disabled by default

    • Added Parameter values for jobs to notification emails
    • SMTP connections now support empty strings for usernames/passwords

    • Error Messaging Improvements

    • when using a range that includes duplicate header names in the Excel source node

    • when previewing the entire sheet in the Excel source node
    • for the Business Rules node
    • for the Reorder node
    • in file browsers when navigating to an invalid directory
    • Bug Fixes

  • The virtual file browser is no longer showing folders as physical paths, and is now correctly showing folder names as virtual folder paths

  • Fixed an issue where Excel source nodes didn't accept source ranges that included multi-letter columns
  • Added the ability to vertically scroll when looking at job request history in the Cloud Connector settings screen
  • Email addresses for notifications now support addresses being entered with upper-case letters
  • Fixed an issue where license activation would sometimes fail due to an invalid SSL certificate
  • Added the ability to horizontally scroll when previewing the entire sheet in the Excel source node
  • The "parameters" property is no longer required when kicking of a job via the REST API
  • Fixed an issue where ensemble models weren't properly mapped to the correct name when used in the Forecast Single Strategy node
  • Fixed an issue where ensemble models weren't properly calculating their underlying models when used in the Forecast Single Strategy node
  • Fixed an issue where column names with hyphens weren't usable in the Forecast Strategy/Forecast Single Strategy node
  • Fixed an issue where blank file names threw an exception in the Flat File/Excel source nodes
  • Fixed an issue where pop-up notifications were causing settings tabs to flicker
  • Fixed an issue with ARIMA models in the Forecast Strategy node re: score frame periods

6.3.2 - 12/15/2023

  • Improvements

  • Performance

    • Prevented an issue where Script nodes refreshed on every key event in the editor
    • The DataWORQ URL is now case-insensitive
    • Centralized all supported file types
    • Bug Fixes

  • Fixed an issue where network drives weren't supported as mapped directories

  • Fixed an issue where context menu options for the flow and file management screens weren't working

    • Delete
    • Rename
    • New Folder
    • Fixed an issue with the delete hotkey in the Flow management screen

    • Fixed an issue where context menu options for canvas nodes weren't working

    • Add Node

    • Connect Nodes
    • Fixed an issue where File Watcher wasn't informing users of invalid watched directories
    • Importing a flow now creates a flow with the same flow ID as the exported flow
    • Importing a flow now refreshes the list of flows
    • Fixed an issue where importing flows with nodes using parameters weren't properly being mapped on import
    • Fixed an issue where Flat File/Excel source nodes couldn't be saved with a blank file name

6.3.1 - 11/28/2023

  • Bug Fixes

  • Updated the default SQL Server connection string to be consistent with the value in the JDBC connection string dropdown

  • Fixed an issue where IBM Planning Analytics cellsets weren't being cleaned up after creation
  • Fixed an issue where custom CAM namespaces weren't properly used in IBM Planning Analytics data connections
  • Fixed a UI issue when selecting flows to export in the Flow Management screen
  • Fixed a UI issue where the Flow/File Management screens weren't automatically refreshing on open

6.3.0- 11/06/2023

  • Containerization support

  • Added support for running DataWORQ in containers

  • Using Podman for container infrastructure

  • Created a Admin Tool (containerized web application) to help manage DataWORQ

    • Configuration of application properties
    • Check the status of running containers
    • Start/stop containers ad hoc
    • Import/export content store to/from containers
    • Made performance improvements around the refreshing of multiple UI components, including opening/closing node edit mode and script node previewing
    • Bug Fixes

  • Fixed an issue when using parameters in the Planning Analytics Process node

  • Fixed an issue where Forecast Strategy nodes weren't supporting some input column headers with spaces
  • Fixed an issue where flows couldn't be exported via the Flow Management screen

6.2.2- 10/25/2023

  • Amazon S3 Support

  • Added Amazon S3 support

  • New S3 Source Node
  • New S3 Export Node

  • File Management updates

  • Removed Temporary and Working directories from application settings

  • Remove the mappings entries in the mapping section of the directoryMappings.cfg file
  • Created a Backups folder that is empty at the default folder location

  • Import File Browser updates

  • Default for Target location is now / if no file was selected

  • Target location file browser can only browse folder locations
  • Added a message to select a file if export file is chosen with no file selection
  • Updated the file browser to use common file types (.txt, .csv, .xlsx, .dwexport)

  • Bug Fixes

  • Fixed an issue where the temporary folder used for forecast node models wasn't probably referenced

  • Fixed an issue due to extra quotes when attempting to import flows
  • Fixed an issue where files didn't have extensions properly applied after browsing in file browsers
  • Fixed issue with setting up parameters for Adaptive data connections

6.2.0- 9/22/2023

  • Adaptive Support

  • Added Workday Adaptive support

  • New Adaptive Source Node
  • New Adaptive Export Node

  • Script Node Improvements

  • Script Source and Column nodes now support multiple input nodes

  • Script Source and Column nodes now support defining output data types

  • Bug Fixes

  • Updated resource for Windows Integrated Login

  • Fixed bug with record limit in mysql
  • File Export node now adds the proper file extension

6.1.10- 8/02/2023

  • Improvements

  • Smartsheet support

    • Added Smartsheet data connection support
    • New Smartsheet Source node to pull data from a named Smartsheet sheet
    • New Smartsheet Export node to update an existing Smartsheet sheet
    • Added a new Pivot node to support dataframe pivot/unpivot functionality

    • JDBC support

    • Updated supported JDBC driver versions

    • Updated default connection strings for JDBC connections

    • IBM Planning Analytics support

    • Updated IBM PA Local/Cloud/PAaaS support for running flows

    • Updated SSL certificate validation for PA connections
    • R installations newer than 4.2.1 are now supported

    • Updated HTTP client connection pooling for:

    • Anaplan connections

    • Pigment connections

    • Bug Fixes

    • Pigment support

    • Fixed an issue re: Pigment connections on flow import

    • Added Pigment connections to supported data connection parameters
    • Fixed an issue with parameter resolution for data connections in nodes
    • Fixed an issue where "." wasn't supported in flow names

6.1.6- 7/17/2023

  • Improvements

  • Added new flow import/export functionality

    • Flow Export

    • Admins can export flows via the Flow Management screen

    • Exports can be local downloads or exported to a virtual path on the server
    • Exports are given a .dwexport extension

    • Flow Import

    • Admins can import flows via the Flow Management screen

    • Imports are selected via the virtual file browser

    • Importing flows involves a mapping process for nodes with:

      • Flat file references
      • Data connections
      • Flow references
        • Legacy Flow Import

    • Legacy flows (.flx files) can be imported as well

    • Follows a similar mapping exercise to the above process

    • Script node enhancements:

    • Improved node configuration re: output data types

    • Improved node configuration re: delimiters
    • Added support for multiple source data frames for a single script node
    • Improved script editor styling to allow for greater script visibility

    • Cloud connector enhancements:

    • Added a new tab to show jobs initiated by Cloud Connector requests

    • Added a new tab to show Cloud Connector request errors

    • JDBC driver version updates:

    • Microsoft SQL Server

    • MySQL
    • Oracle
    • PostgreSQL

    • Improved logging:

    • Node IDs now included in error logs

    • Errors when activating licenses
    • Upstream nodes when a node throws an exception
    • When using Forecast Strategy/Forecast Single Strategy nodes with negative actuals
    • When using the Planning Analytics Source node against views that don't exist
    • When using the Planning Analytics Export node against views/dimensions/elements that don't exist
    • Improved security admin setup when switching security modes and switching environments for the same security mode
    • Improved Planning Analytics Process node status checking
    • Updated the Flow Menu to remove the Data Connections option
    • Bug Fixes

  • Nodes

    • Improved exception handling when inferring Data Types for the Flat File Source node
    • Fixed an issue when Flow Output nodes had invalid flow IDs/run profiles saved to them
    • Fixed an issue when Planning Analytics Process nodes had no connection source saved to them
    • Fixed an issue when scoring data for an R model in the Forecast Single Strategy node
    • Fixed an issue where R models were being placed in the incorrect directory when running R models via the Forecast Strategy node

    • Parameters

    • Fixed an issue when saving a new string parameter with no value

    • Newly added string parameters now default to an empty string
    • Newly added numeric parameters now default to 0
    • Fixed an issue when parameters had spaces in their names
    • Fixed an issue when adjacent parameters in a string had an underscore between them

    • Cube Watcher

    • Fixed an issue where Cube Watcher would not connect on startup even if configured to

    • Fixed an issue with the Cube Watcher Connect on Startup label

    • Cloud Connector

    • Fixed an issue when configuring Cloud Connector for the first time

    • JDBC

    • Removed deprecated JDBC drivers

    • org.netezza.Driver

    • com.cloudera.impala.jdbc41.Driver

    • Updated JDBC drivers

    • com.mysql.jdbc.Driver

    • Fixed an issue when previewing with an invalid JDBC connection source
    • Fixed an issue with files/flows being randomly sorted in lists
    • Fixed an issue when enabling debug logging
    • Fixed an issue with copy/pasting flows in the Flow Management screen
    • Updated exception dialog styling
    • Fixed an issue where data connections using the default password weren't being saved

6.0.7- 6/02/2023

  • Bug Fixes

  • Updated usernames where UUIDs were displayed

  • Updated the error message when an incorrect API key header is used
  • Fixed an issue re: seeding flow paths from the Open Flow browser dialog
  • Removed the unused Log Output Directory property from scheduled jobs
  • Disabled non-functional enable/disable buttons on cards

6.0.5- 6/01/2023

  • Improvements

  • Added parameter support for Command Line nodes

  • Added support for seeding initial admins when changing security modes

  • Bug Fixes

  • Updated script editor styling

  • Fixed an issue re: logged usernames for some security modes
  • Fixed an issue re: rebuilding LP Optimization node models
  • Updated LP Optimization node GMPL template parameter definitions
  • Fixed an issue when saving new default flows

6.0.4- 5/24/2023

  • Improvements

  • Improved exception messaging when using the Forecast node

  • Bug Fixes

  • Always rebuild predictive models that shouldn't be serialized

    • Forecast
    • Forecast Single Strategy
    • Forecast Strategy

    • Prevented the ability to unnecessarily rebuild predictive models via Run Profiles

    • Forecast

    • Forecast Single Strategy
    • Forecast Strategy

    • Fixed an issue re: serializing predictive models

    • Text Analytics

    • Association
    • Classification
    • Regression
    • Cluster
    • Survival
    • Fixed an issue impacting selecting AutoKMedians as a model type when using the Cluster Node
    • Improved exception handling around opening Data Connections

6.0.3- 5/22/2023

  • Bug Fixes

  • Fixed an issue re: interpreting parameters with empty values for the PA process node

  • Fixed an issue around flow ordering in the browser dialog
  • Added support for trailing whitespace characters in file browsers
  • Added support for flow browser name case insensitivity

6.0.2- 5/9/2023

  • Bug Fixes

  • Resolved issue where flows were incorrectly being saved as duplicates

6.0.1- 5/4/2023

  • Improvements

  • New security mode support:

    • Open ID Connect
    • SAML2
    • LDAP
    • Active Directory

    • Application Managed Content:

    • Application managed content is centralized and available across all flows

    • Flows
    • SMTP Connection
    • Data Connections

    • Virtual File Management:

    • Ability to map physical directories to virtual in-app directories

    • Virtual file resolution support for nodes
    • File Watcher is managed via virtual mappings
    • Default settings are now managed virtually
    • Scripting nodes (R and Python) now have virutal support
    • Upload file support and download file support
    • General file and folder management support such as copy/paste/move/delete

    • New settings are supported:

    • Execution properties:

    • Python

    • R
    • GLPK
    • Command Line Node
    • Database properties
    • Security and password vault
    • Logging
    • Authentication properties
    • Directory mappings

    • Execution updates:

    • Job state is captured prior to execution. This ensures the state of all flow components is consistent with the flow state at the time the flow is executed.

    • Improved data connection resolution
    • Improved parameter resolution
    • Flow resolution
    • JavaScript processing upgraded to use GraalVM

    • Security & Encryption:

    • All user passwords are hashed with latest encryption technology

    • All connection credentials are securely encrypted

    • API improvements:

    • Added API key security

    • Updated endpoints for job execution to match modern resource conventions
    • URL API uses updated flow paths
    • API error messaging improved
    • Cloud Connector now uses updated API convention

    • Improved user management:

    • Admin and non-admin roles for users

    • Admins can manage user security, including setting roles and generating API keys
    • Improved support for user administration

    • Legacy feature support:

    • Exercises are temporarily disabled

    • UI updates:

    • Settings screens now instantly apply changes (save and cancel removed)

    • Styling improvements
    • Added new flow and file browsers
    • Bug Fixes

  • Resolved issue where node pane would open in uneditable state

5.4.2- 1/10/2023

  • DataWORQ rebranding

  • Rebranded Euclid Studio as DataWORQ

  • Updated any references in-app or in documentation to the new product name

  • Improvements

  • Optimized the performance of Croston's/Improved Croston's models for the Forecast Strategy node

  • Optimized Forecast Strategy node performance when querying metadata downstream
  • Improved Forecast Accuracy node error reporting
  • Improved error messaging for the Croston's predictive model type

  • Bug Fixes

  • Fixed an issue when checking for Forecast Strategy node R version compatibility

  • Fixed an issue with the way the Forecast Accuracy node was sorting by role column values
  • Fixed file dialog file extensions not being in sync with node file paths
  • Fixed an issue where the NN predictive model type wouldn't work with integer target data
  • Fixed an issue with ensemble predictive models erroring on averaging

5.3.15- 12/5/2022

  • Improvements

  • Performance improvements to Forecast Strategy node models:

    • Mean
    • Seasonal Decomposition

    • Added support for Excel files (.xls, .xlsx, .xlsm):

    • Excel File Source Node

    • Excel File Export Node

    • Added support for Pigment views:

    • Pigment Source Node

    • Pigment Export Node
    • Added the ability to set the frequency at which the Planning Analytics Process node checks for the completion of a process
    • Automatically retrying failed Planning Analytics Process node checks a set number of times before erroring
    • Bug Fixes

  • Fixed a bug re: Forecast Strategy models in R forecasting extra periods

  • Stopped the Forecast Strategy node from forecasting future periods when using partitions
  • Fixed an issue re: generating forecasts for the seasonal decomposition model
  • Fixed issue where .jdt file support wasn't working correctly in the Flat File Source node
  • Fixed an issue with dragging and dropping nodes with canvas scroll
  • Fixed an issue with parameter override data types and formatting

5.3.7- 10/25/2022

  • Improvements

  • Added detailed performance information to job logs

  • Updates to the predictive model types used by the Forecast Strategy/Forecast Single Strategy nodes, including:

    • Additions:

    • Improved Croston's

      • A one-step ahead forecast using Croston’s method, incorporating that prediction back into the training set.
        • Neural Network
        • Removals:

    • Naive

    • Seasonal Naive

    • Updates:

    • All ensemble models have been updated to represent the type of data they should be run on, including:

      • Good Signal
      • Poor Signal
      • Intermittent Data
      • On Forecast Strategy node R errors, now writing out the entire error to the R temp directory.
      • Truncating the Error output column to 255 characters for Forecast Strategy node R errors.

Bug Fixes

  • Fixed a bug preventing the Forecast Strategy/Forecast Single Strategy nodes from running forecasts in some cases
  • Fixed a bug with branch copy/duplictate functionality
  • Fixed a bug where the active run profile wasn't updated to use the most recent flow parameter values
  • Fix for R processes not being cleaned up after running/cancelling Forecast Strategy node runs
  • Updated ensemble model names in Forecast Strategy node
  • Forecast Strategy node Model Filter is no longer unselectable
  • Fix for running the Improved Croston's model alone in the Forecast Strategy nodes
  • Fix for running the Intermittent Data ensemble model in the Forecast Strategy nodes
  • Documentation updates

  • Fixes for when running multiple Forecast Strategy/Forecast Single Strategy nodes in parallel:

  • Fix for strategy names in Scoring/Forecast periods not being correctly assigned across similar combos

  • Updated R NA/null values to output as null

5.3.0- 10/05/2022

  • Improvements

  • Added multiple new nodes to enhance time series forecasting

    • Sequence (Row) - Fills in missing parts of a numeric sequence or time period.
    • Frequency (Calculation & Analysis) - Calculates the count or percentage of occurrences of a condition across rows.
    • Forecast Accuracy (Calculation & Analysis) - Calculates numerous accuracy metrics for numerous time series forecasts.
    • White Noise (Calculation & Analysis) - Calculates the amount of white noise present in a dataset.
    • Forecast Category (Calculation & Analysis) - Groups time series data into one of nine categories based on available records and their variation.
    • Forecast Single Strategy (Prediction) - Produce a single time series forecast across many datasets.
    • Forecast Strategy (Prediction) - Produce numerous time series forecast across many datasets.

    • New predictive model types supported, including:

    • Naive

    • Seasonal Naive
    • Mean
    • Arima
    • Exponential Smoothing
    • Seasonal Decomposition
    • Theta
    • Croston's
    • Linear Regression
    • Prophet
    • Various ensemble models composed of the above
    • Various updates to provide clarity re: partition and role columns
    • Update to Log4j config to not log performance metrics by default
    • Partition node added support for time series data and splitting by record count.
    • Added a timeout setting to all Script nodes
    • Update to Chart node to support scatter charts
    • Update to Cleanse node to support breaking out a Datetime column into multiple string columns
    • Updated Script nodes to support multiple data frame delimeter types
    • Planning Analytics Process Node now supports Euclid Studio parameters for TI Process parameter values.
    • UI Updates

  • A new node category called Calculation & Analysis was added.

  • Time Series node was moved to the Calculation & Analysis category.
  • Business Rule node was moved to the Calculation & Analysis category.
  • Chart node was moved to the Calculation & Analysis category.
  • Optimization node category color was changed.
  • Charting colors were updated in the Chart node.

  • Bug Fixes

  • Fixed font sizes not being saved in Script nodes

  • Fixed the canvas snapping to the left when selecting nodes off canvas
  • Fixed the Filter node not converting between String and Datetime data types
  • Fixed the Database export node not creating columns starting with numeric values
  • Fixed the node state indicator position when a node contains a comment
  • Updated the Chart node and Rest Command node to be previewable from the node context menu
  • Fixed node and branch copy and duplication on the canvas
  • Disabled interactive notes during node edit mode
  • Fixed database connections disappearing when navigating to a flow via the API
  • Fixed double clicking nodes sometimes causing a blank pane to open
  • Node context menu fix for disabled nodes
  • Add to Canvas style updates for disabled nodes

5.2.7- 8/22/2022

  • Improvements

  • Updates to script nodes styling

  • Updates to Job Scheduling styling
  • Updates to Cube Watch styling
  • Updates to File Watcher styling
  • License checks now happen on restart and once daily
  • Updates to license related logging

  • Bug Fixes

  • Fixed issue with licensing entitlements

5.2.6- 8/11/2022

  • Improvements

  • Job Activity

    • Lazy load all jobs and paginate them (25,50,100)
    • Only store the last 1000 executed jobs
    • Ability to disable UI auto refresh/toast notifications

5.2.5 - 8/4/2022

  • Improvements

  • Added support for Flow failure on PA Process failure

  • Improved PA Process logging
  • Added support for Flow failure on Anaplan Action failure
  • Improved Anaplan Action logging
  • Added data type support for cells in Cube Watch views
  • Added Cube Watch icon to top action bar when running
  • Added Error badges to Cube Watch icon and settings tabs
  • Added autocomplete and free form value input to Business Rules
  • Improved Business Rules editing performance
  • Various styling improvements
  • Various documentation updates

  • Bug Fixes

  • Fixed issue with saving Flow database connections

  • Fixed issue with overriding saved database connections
  • Fixed Cube Watch configuration error
  • Fixed script editor rendering issue
  • Fixed Business Rules initial groupings

5.2.4 - 7/9/2022

  • New Features

  • Planning Analytics

    • Planning Analytics Source Node
    • Planning Analytics Process Output Node
    • Planning Analytics Export Node
    • Planning Analytics Cube Watcher

    • Anaplan

    • Anaplan Transactional Source Node

    • Anaplan Bulk Source Node
    • Anaplan Process Output Node
    • Anaplan Transactional Export Node
    • Anaplan Bulk Export Node

    • Data Connections

    • Planning Analytics Local Connection

    • Planning Analytics Cloud Connection
    • Anaplan Connection
    • JDBC Connection
    • Fixes and Improvements

  • Added font size and expansion to all editor nodes

  • Canvas performance improvements
  • Database Connection improvements
  • Navigation improvements
  • Save and change rollback performance improvements

5.1.3.0 - 5/16/2022

  • Bug Fixes

  • Optimization Node styling

  • Error styling
  • Flow Output Node Parameter Support
  • Grid toggle is fixed
  • Flow navigation error
  • Node confirmation fix
  • Double click properly opens now

  • Improvements

  • Drag and drop auto link from canvas

  • License settings shortcut in footer

5.1.0.0 - 4/04/2022

  • New Features

  • New user interface

    • A complete overhaul re: the user expereience
    • These changes are detailed in the updated Reference Guide

    • In-app reference documentation

    • Accessed through the Help pane

    • In-app examples and exercises to get you started

    • Exercises can be updated with a click of a button as new content is released

    • Includes Node Examples and How-To Exercises
    • Drag and drop nodes from the Node Menu
    • Keyboard shortcuts
    • Auto arrange branches