DataWORQ Release Notes

Download DataWORQ

The DataWORQ Official Release is updated on a monthly basis after testing and documentation is updated. Updates can be downloaded through the QueBIT software portal at https://quebit.com/software-access/.

Releases

6.4.2 - 7/19/2024

• Bug Fixes

  • Fixed Import/Export Bug

6.4.1 - 5/21/2024

• Improvements

  • Flow Node Enhancements

    • Dynamic Run Profile selection

    • Parameter overrides

    • Full parameter support

  • Flow Control Nodes added

    • Condition Node

      • Supports conditional execution of nodes.

      • Uses parameters or upstream node values.

    • Stop Execution Node

      • Allows you to stop flow execution.

      • Intended to be used with the condition node.

  • Support for the Long data type to allow for large numbers.

• Bug Fixes

  • PA nodes now support all valid characters in object names.

  • Fixed DataFrame sort issue in predictive nodes.

  • Fixed sorting of files in dialogs.

  • Accuracy node now properly leverages the ROLE column.

  • Fixed cut functionality for flows in the content menu.

  • UI selection now shows up properly in the content menus.

6.3.3 - 2/5/2024

• Improvements

  • Improved responsiveness in the following areas

    • moving nodes on the canvas

    • performing actions on setting screens such as reordering branches in the run profiles menu

    • adding nodes to the canvas

    • moving nodes around via arrange, move branch up, down etc...

    • status issues where changes aren't detected during canvas move operations

  • DataWORQ Native Installer

    • Installer now includes the requests Python package by default

    • All installer Windows services are now set to Automatic startup

  • Run Profiles

    • Changed the "Add" run profile button to "Duplicate"

      • New Run Profiles are initialized with the settings of the active Run Profile

    • All Run Profiles can be activated/deactivated at once via a checkbox

    • Chart nodes no longer show up as runnable nodes

  • Updated the Pigment REST API calls internal to DataWORQ to the latest version

    • Only the most recent form of Pigment API keys are now supported

  • Node Improvements

    • Changing columns upstream of the Filter node is now handled more gracefully

    • Added the ability to refresh column names in the Reorder node

    • The Aggregate node now supports using String types for min/max aggregate functions

    • Updated Planning Analytics TI Process

      • Improved the standard Planning Analytics TI process to kick off flows when polling a long-running job

      • Removed the pWait parameter

      • 202 is now a valid response code for checking job status via the REST API

    • Business Rules node enhancements

      • Added support for parameters to certain input fields

      • Improved error messaging when using unquoted strings as inputs

        • Output Column Name

        • Default Rule

        • Name

        • Business Purpose

        • Custom Expression

        • Rule Result

  • General

    • Node indicators can be enabled/disabled via Application Settings. They are now disabled by default

    • Added Parameter values for jobs to notification emails

    • SMTP connections now support empty strings for usernames/passwords

  • Error Messaging Improvements

    • when using a range that includes duplicate header names in the Excel source node

    • when previewing the entire sheet in the Excel source node

    • for the Business Rules node

    • for the Reorder node

    • in file browsers when navigating to an invalid directory

• Bug Fixes

  • The virtual file browser is no longer showing folders as physical paths, and is now correctly showing folder names as virtual folder paths

  • Fixed an issue where Excel source nodes didn't accept source ranges that included multi-letter columns

  • Added the ability to vertically scroll when looking at job request history in the Cloud Connector settings screen

  • Email addresses for notifications now support addresses being entered with upper-case letters

  • Fixed an issue where license activation would sometimes fail due to an invalid SSL certificate

  • Added the ability to horizontally scroll when previewing the entire sheet in the Excel source node

  • The "parameters" property is no longer required when kicking of a job via the REST API

  • Fixed an issue where ensemble models weren't properly mapped to the correct name when used in the Forecast Single Strategy node

  • Fixed an issue where ensemble models weren't properly calculating their underlying models when used in the Forecast Single Strategy node

  • Fixed an issue where column names with hyphens weren't usable in the Forecast Strategy/Forecast Single Strategy node

  • Fixed an issue where blank file names threw an exception in the Flat File/Excel source nodes

  • Fixed an issue where pop-up notifications were causing settings tabs to flicker

  • Fixed an issue with ARIMA models in the Forecast Strategy node re: score frame periods

6.3.2 - 12/15/2023

• Improvements

  • Performance

    • Prevented an issue where Script nodes refreshed on every key event in the editor

  • The DataWORQ URL is now case-insensitive

  • Centralized all supported file types

• Bug Fixes

  • Fixed an issue where network drives weren't supported as mapped directories

  • Fixed an issue where context menu options for the flow and file management screens weren't working

    • Delete

    • Rename

    • New Folder

  • Fixed an issue with the delete hotkey in the Flow management screen

  • Fixed an issue where context menu options for canvas nodes weren't working

    • Add Node

    • Connect Nodes

  • Fixed an issue where File Watcher wasn't informing users of invalid watched directories

  • Importing a flow now creates a flow with the same flow ID as the exported flow

  • Importing a flow now refreshes the list of flows

  • Fixed an issue where importing flows with nodes using parameters weren't properly being mapped on import

  • Fixed an issue where Flat File/Excel source nodes couldn't be saved with a blank file name

6.3.1 - 11/28/2023

• Bug Fixes

  • Updated the default SQL Server connection string to be consistent with the value in the JDBC connection string dropdown

  • Fixed an issue where IBM Planning Analytics cellsets weren't being cleaned up after creation

  • Fixed an issue where custom CAM namespaces weren't properly used in IBM Planning Analytics data connections

  • Fixed a UI issue when selecting flows to export in the Flow Management screen

  • Fixed a UI issue where the Flow/File Management screens weren't automatically refreshing on open

6.3.0- 11/06/2023

•  Containerization support

  • Added support for running DataWORQ in containers

  • Using Podman for container infrastructure

  • Created a Admin Tool (containerized web application) to help manage DataWORQ

    • Configuration of application properties

    • Check the status of running containers

    • Start/stop containers ad hoc

    • Import/export content store to/from containers

  • Made performance improvements around the refreshing of multiple UI components, including opening/closing node edit mode and script node previewing

• Bug Fixes

  • Fixed an issue when using parameters in the Planning Analytics Process node

  • Fixed an issue where Forecast Strategy nodes weren't supporting some input column headers with spaces

  • Fixed an issue where flows couldn't be exported via the Flow Management screen

6.2.2- 10/25/2023

• Amazon S3 Support

  • Added Amazon S3 support

  • New S3 Source Node

  • New S3 Export Node

• File Management updates

  • Removed Temporary and Working directories from application settings

  • Remove the mappings entries in the mapping section of the directoryMappings.cfg file

  • Created a Backups folder that is empty at the default folder location

• Import File Browser updates

  • Default for Target location is now / if no file was selected

  • Target location file browser can only browse folder locations

  • Added a message to select a file if export file is chosen with no file selection

  • Updated the file browser to use common file types (.txt, .csv, .xlsx, .dwexport)

• Bug Fixes

  • Fixed an issue where the temporary folder used for forecast node models wasn't probably referenced

  • Fixed an issue due to extra quotes when attempting to import flows

  • Fixed an issue where files didn't have extensions properly applied after browsing in file browsers

  • Fixed issue with setting up parameters for Adaptive data connections

6.2.0- 9/22/2023

• Adaptive Support

  • Added Workday Adaptive support

  • New Adaptive Source Node

  • New Adaptive Export Node

• Script Node Improvements

  • Script Source and Column nodes now support multiple input nodes

  • Script Source and Column nodes now support defining output data types

• Bug Fixes

  • Updated resource for Windows Integrated Login

  • Fixed bug with record limit in mysql

  • File Export node now adds the proper file extension

6.1.10- 8/02/2023

• Improvements

  • Smartsheet support

    • Added Smartsheet data connection support

    • New Smartsheet Source node to pull data from a named Smartsheet sheet

    • New Smartsheet Export node to update an existing Smartsheet sheet

    • Added a new Pivot node to support dataframe pivot/unpivot functionality

  • JDBC support

    • Updated supported JDBC driver versions

    • Updated default connection strings for JDBC connections

  • IBM Planning Analytics support

    • Updated IBM PA Local/Cloud/PAaaS support for running flows

    • Updated SSL certificate validation for PA connections

  • R installations newer than 4.2.1 are now supported

  • Updated HTTP client connection pooling for:

    • Anaplan connections

    • Pigment connections

  • Bug Fixes

    • Pigment support

      • Fixed an issue re: Pigment connections on flow import

      • Added Pigment connections to supported data connection parameters

    • Fixed an issue with parameter resolution for data connections in nodes

    • Fixed an issue where "." wasn't supported in flow names

6.1.6- 7/17/2023

• Improvements

  • Added new flow import/export functionality

    • Flow Export

      • Admins can export flows via the Flow Management screen

      • Exports can be local downloads or exported to a virtual path on the server

      • Exports are given a .dwexport extension

    • Flow Import

      • Admins can import flows via the Flow Management screen

      • Imports are selected via the virtual file browser

      • Importing flows involves a mapping process for nodes with:

        • Flat file references

        • Data connections

        • Flow references

    • Legacy Flow Import

      • Legacy flows (.flx files) can be imported as well

      • Follows a similar mapping exercise to the above process

  • Script node enhancements:

    • Improved node configuration re: output data types

    • Improved node configuration re: delimiters

    • Added support for multiple source data frames for a single script node

    • Improved script editor styling to allow for greater script visibility

  • Cloud connector enhancements:

    • Added a new tab to show jobs initiated by Cloud Connector requests

    • Added a new tab to show Cloud Connector request errors

  • JDBC driver version updates:

    • Microsoft SQL Server

    • MySQL

    • Oracle

    • PostgreSQL

  • Improved logging:

    • Node IDs now included in error logs

    • Errors when activating licenses

    • Upstream nodes when a node throws an exception

    • When using Forecast Strategy/Forecast Single Strategy nodes with negative actuals

    • When using the Planning Analytics Source node against views that don't exist

    • When using the Planning Analytics Export node against views/dimensions/elements that don't exist

  • Improved security admin setup when switching security modes and switching environments for the same security mode

  • Improved Planning Analytics Process node status checking

  • Updated the Flow Menu to remove the Data Connections option

• Bug Fixes

  • Nodes

    • Improved exception handling when inferring Data Types for the Flat File Source node

    • Fixed an issue when Flow Output nodes had invalid flow IDs/run profiles saved to them

    • Fixed an issue when Planning Analytics Process nodes had no connection source saved to them

    • Fixed an issue when scoring data for an R model in the Forecast Single Strategy node

    • Fixed an issue where R models were being placed in the incorrect directory when running R models via the Forecast Strategy node

  • Parameters

    • Fixed an issue when saving a new string parameter with no value

    • Newly added string parameters now default to an empty string

    • Newly added numeric parameters now default to 0

    • Fixed an issue when parameters had spaces in their names

    • Fixed an issue when adjacent parameters in a string had an underscore between them

  • Cube Watcher

    • Fixed an issue where Cube Watcher would not connect on startup even if configured to

    • Fixed an issue with the Cube Watcher Connect on Startup label

  • Cloud Connector

    • Fixed an issue when configuring Cloud Connector for the first time

  • JDBC

    • Removed deprecated JDBC drivers

      • org.netezza.Driver

      • com.cloudera.impala.jdbc41.Driver

    • Updated JDBC drivers

      • com.mysql.jdbc.Driver

    • Fixed an issue when previewing with an invalid JDBC connection source

  • Fixed an issue with files/flows being randomly sorted in lists

  • Fixed an issue when enabling debug logging

  • Fixed an issue with copy/pasting flows in the Flow Management screen

  • Updated exception dialog styling

  • Fixed an issue where data connections using the default password weren't being saved

6.0.7- 6/02/2023

• Bug Fixes

  • Updated usernames where UUIDs were displayed

  • Updated the error message when an incorrect API key header is used

  • Fixed an issue re: seeding flow paths from the Open Flow browser dialog

  • Removed the unused Log Output Directory property from scheduled jobs

  • Disabled non-functional enable/disable buttons on cards

6.0.5- 6/01/2023

• Improvements

  • Added parameter support for Command Line nodes

  • Added support for seeding initial admins when changing security modes

• Bug Fixes

  • Updated script editor styling

  • Fixed an issue re: logged usernames for some security modes

  • Fixed an issue re: rebuilding LP Optimization node models

  • Updated LP Optimization node GMPL template parameter definitions

  • Fixed an issue when saving new default flows

6.0.4- 5/24/2023

• Improvements

  • Improved exception messaging when using the Forecast node

• Bug Fixes

  • Always rebuild predictive models that shouldn't be serialized

    • Forecast

    • Forecast Single Strategy

    • Forecast Strategy

  • Prevented the ability to unnecessarily rebuild predictive models via Run Profiles

    • Forecast

    • Forecast Single Strategy

    • Forecast Strategy

  • Fixed an issue re: serializing predictive models

    • Text Analytics

    • Association

    • Classification

    • Regression

    • Cluster

    • Survival

  • Fixed an issue impacting selecting AutoKMedians as a model type when using the Cluster Node

  • Improved exception handling around opening Data Connections

6.0.3- 5/22/2023

• Bug Fixes

  • Fixed an issue re: interpreting parameters with empty values for the PA process node

  • Fixed an issue around flow ordering in the browser dialog

  • Added support for trailing whitespace characters in file browsers

  • Added support for flow browser name case insensitivity

6.0.2- 5/9/2023

• Bug Fixes

  • Resolved issue where flows were incorrectly being saved as duplicates

6.0.1- 5/4/2023

• Improvements

  • New security mode support:

    • Open ID Connect

    • SAML2

    • LDAP

    • Active Directory

  • Application Managed Content:

    • Application managed content is centralized and available across all flows

    • Flows

    • SMTP Connection

    • Data Connections

  • Virtual File Management:

    • Ability to map physical directories to virtual in-app directories

    • Virtual file resolution support for nodes

    • File Watcher is managed via virtual mappings

    • Default settings are now managed virtually

    • Scripting nodes (R and Python) now have virutal support

    • Upload file support and download file support

    • General file and folder management support such as copy/paste/move/delete

  • New settings are supported:

    • Execution properties:

      • Python

      • R

      • GLPK

      • Command Line Node

    • Database properties

    • Security and password vault

    • Logging

    • Authentication properties

    • Directory mappings

  • Execution updates:

    • Job state is captured prior to execution. This ensures the state of all flow components is consistent with the flow state at the time the flow is executed.

    • Improved data connection resolution

    • Improved parameter resolution

    • Flow resolution

    • JavaScript processing upgraded to use GraalVM

  • Security & Encryption:

    • All user passwords are hashed with latest encryption technology

    • All connection credentials are securely encrypted

  • API improvements:

    • Added API key security

    • Updated endpoints for job execution to match modern resource conventions

    • URL API uses updated flow paths

    • API error messaging improved

    • Cloud Connector now uses updated API convention

  • Improved user management:

    • Admin and non-admin roles for users

    • Admins can manage user security, including setting roles and generating API keys

    • Improved support for user administration

  • Legacy feature support:

    • Exercises are temporarily disabled

  • UI updates:

    • Settings screens now instantly apply changes (save and cancel removed)

    • Styling improvements

    • Added new flow and file browsers

• Bug Fixes

  • Resolved issue where node pane would open in uneditable state

5.4.2- 1/10/2023

• DataWORQ rebranding

  • Rebranded Euclid Studio as DataWORQ

  • Updated any references in-app or in documentation to the new product name

• Improvements

  • Optimized the performance of Croston's/Improved Croston's models for the Forecast Strategy node

  • Optimized Forecast Strategy node performance when querying metadata downstream

  • Improved Forecast Accuracy node error reporting

  • Improved error messaging for the Croston's predictive model type

• Bug Fixes

  • Fixed an issue when checking for Forecast Strategy node R version compatibility

  • Fixed an issue with the way the Forecast Accuracy node was sorting by role column values

  • Fixed file dialog file extensions not being in sync with node file paths

  • Fixed an issue where the NN predictive model type wouldn't work with integer target data

  • Fixed an issue with ensemble predictive models erroring on averaging

5.3.15- 12/5/2022

• Improvements

  • Performance improvements to Forecast Strategy node models:

    • Mean

    • Seasonal Decomposition

  • Added support for Excel files (.xls, .xlsx, .xlsm):

    • Excel File Source Node

    • Excel File Export Node

  • Added support for Pigment views:

    • Pigment Source Node

    • Pigment Export Node

  • Added the ability to set the frequency at which the Planning Analytics Process node checks for the completion of a process

  • Automatically retrying failed Planning Analytics Process node checks a set number of times before erroring

• Bug Fixes

  • Fixed a bug re: Forecast Strategy models in R forecasting extra periods

  • Stopped the Forecast Strategy node from forecasting future periods when using partitions

  • Fixed an issue re: generating forecasts for the seasonal decomposition model

  • Fixed issue where .jdt file support wasn't working correctly in the Flat File Source node

  • Fixed an issue with dragging and dropping nodes with canvas scroll

  • Fixed an issue with parameter override data types and formatting

5.3.7- 10/25/2022

• Improvements

  • Added detailed performance information to job logs

  • Updates to the predictive model types used by the Forecast Strategy/Forecast Single Strategy nodes, including:

    • Additions:

      • Improved Croston's

        • A one-step ahead forecast using Croston’s method, incorporating that prediction back into the training set.

      • Neural Network

    • Removals:

      • Naive

      • Seasonal Naive

    • Updates:

      • All ensemble models have been updated to represent the type of data they should be run on, including:

        • Good Signal

        • Poor Signal

        • Intermittent Data

  • On Forecast Strategy node R errors, now writing out the entire error to the R temp directory.

  • Truncating the `Error` output column to 255 characters for Forecast Strategy node R errors.

Bug Fixes

• Fixed a bug preventing the Forecast Strategy/Forecast Single Strategy nodes from running forecasts in some cases

• Fixed a bug with branch copy/duplictate functionality

• Fixed a bug where the active run profile wasn't updated to use the most recent flow parameter values

• Fix for R processes not being cleaned up after running/cancelling Forecast Strategy node runs

• Updated ensemble model names in Forecast Strategy node

• Forecast Strategy node Model Filter is no longer unselectable

• Fix for running the Improved Croston's model alone in the Forecast Strategy nodes

• Fix for running the Intermittent Data ensemble model in the Forecast Strategy nodes

• Documentation updates

• Fixes for when running multiple Forecast Strategy/Forecast Single Strategy nodes in parallel:

  • Fix for strategy names in Scoring/Forecast periods not being correctly assigned across similar combos

  • Updated R NA/null values to output as null

5.3.0- 10/05/2022

• Improvements

  • Added multiple new nodes to enhance time series forecasting

    • Sequence (Row) - Fills in missing parts of a numeric sequence or time period.

    • Frequency (Calculation & Analysis) - Calculates the count or percentage of occurrences of a condition across rows.

    • Forecast Accuracy (Calculation & Analysis) - Calculates numerous accuracy metrics for numerous time series forecasts.

    • White Noise (Calculation & Analysis) - Calculates the amount of white noise present in a dataset.

    • Forecast Category (Calculation & Analysis) - Groups time series data into one of nine categories based on available records and their variation.

    • Forecast Single Strategy (Prediction) - Produce a single time series forecast across many datasets.

    • Forecast Strategy (Prediction) - Produce numerous time series forecast across many datasets.

    • New predictive model types supported, including:

      • Naive

      • Seasonal Naive

      • Mean

      • Arima

      • Exponential Smoothing

      • Seasonal Decomposition

      • Theta

      • Croston's

      • Linear Regression

      • Prophet

      • Various ensemble models composed of the above

  • Various updates to provide clarity re: partition and role columns

  • Update to Log4j config to not log performance metrics by default

  • Partition node added support for time series data and splitting by record count.

  • Added a timeout setting to all Script nodes

  • Update to Chart node to support scatter charts

  • Update to Cleanse node to support breaking out a Datetime column into multiple string columns

  • Updated Script nodes to support multiple data frame delimeter types

  • Planning Analytics Process Node now supports Euclid Studio parameters for TI Process parameter values.

• UI Updates

  • A new node category called Calculation & Analysis was added.

  • Time Series node was moved to the Calculation & Analysis category.

  • Business Rule node was moved to the Calculation & Analysis category.

  • Chart node was moved to the Calculation & Analysis category.

  • Optimization node category color was changed.

  • Charting colors were updated in the Chart node.

• Bug Fixes

  • Fixed font sizes not being saved in Script nodes

  • Fixed the canvas snapping to the left when selecting nodes off canvas

  • Fixed the Filter node not converting between String and Datetime data types

  • Fixed the Database export node not creating columns starting with numeric values

  • Fixed the node state indicator position when a node contains a comment

  • Updated the Chart node and Rest Command node to be previewable from the node context menu

  • Fixed node and branch copy and duplication on the canvas

  • Disabled interactive notes during node edit mode

  • Fixed database connections disappearing when navigating to a flow via the API

  • Fixed double clicking nodes sometimes causing a blank pane to open

  • Node context menu fix for disabled nodes

  • Add to Canvas style updates for disabled nodes

5.2.7- 8/22/2022

• Improvements

  • Updates to script nodes styling

  • Updates to Job Scheduling styling

  • Updates to Cube Watch styling

  • Updates to File Watcher styling

  • License checks now happen on restart and once daily

  • Updates to license related logging

• Bug Fixes

  • Fixed issue with licensing entitlements

5.2.6- 8/11/2022

• Improvements

  • Job Activity

    • Lazy load all jobs and paginate them (25,50,100)

    • Only store the last 1000 executed jobs

    • Ability to disable UI auto refresh/toast notifications

5.2.5 - 8/4/2022

• Improvements

  • Added support for Flow failure on PA Process failure

  • Improved PA Process logging

  • Added support for Flow failure on Anaplan Action failure

  • Improved Anaplan Action logging

  • Added data type support for cells in Cube Watch views

  • Added Cube Watch icon to top action bar when running

  • Added Error badges to Cube Watch icon and settings tabs

  • Added autocomplete and free form value input to Business Rules

  • Improved Business Rules editing performance

  • Various styling improvements

  • Various documentation updates

• Bug Fixes

  • Fixed issue with saving Flow database connections

  • Fixed issue with overriding saved database connections

  • Fixed Cube Watch configuration error

  • Fixed script editor rendering issue

  • Fixed Business Rules initial groupings

5.2.4 - 7/9/2022

• New Features

  • Planning Analytics

    • Planning Analytics Source Node

    • Planning Analytics Process Output Node

    • Planning Analytics Export Node

    • Planning Analytics Cube Watcher

  • Anaplan

    • Anaplan Transactional Source Node

    • Anaplan Bulk Source Node

    • Anaplan Process Output Node

    • Anaplan Transactional Export Node

    • Anaplan Bulk Export Node

  • Data Connections

    • Planning Analytics Local Connection

    • Planning Analytics Cloud Connection

    • Anaplan Connection

    • JDBC Connection

• Fixes and Improvements

  • Added font size and expansion to all editor nodes

  • Canvas performance improvements

  • Database Connection improvements

  • Navigation improvements

  • Save and change rollback performance improvements

5.1.3.0 - 5/16/2022

• Bug Fixes

  • Optimization Node styling

  • Error styling

  • Flow Output Node Parameter Support

  • Grid toggle is fixed

  • Flow navigation error

  • Node confirmation fix

  • Double click properly opens now

• Improvements

  • Drag and drop auto link from canvas

  • License settings shortcut in footer

5.1.0.0 - 4/04/2022

• New Features

  • New user interface

    • A complete overhaul re: the user expereience

    • These changes are detailed in the updated Reference Guide

  • In-app reference documentation

    • Accessed through the Help pane

  • In-app examples and exercises to get you started

    • Exercises can be updated with a click of a button as new content is released

    • Includes Node Examples and How-To Exercises

  • Drag and drop nodes from the Node Menu

  • Keyboard shortcuts

  • Auto arrange branches