Clone - DataWORQ Release Notes

Last updated: 04 Jan 2024

Download DataWORQ

The DataWORQ Official Release is updated on a monthly basis after testing and documentation is updated. Updates can be downloaded through the QueBIT software portal at https://quebit.com/software-access/.

Releases

6.3.2 - 12/15/2023

• Improvements
• Performance
  • Prevented an issue where Script nodes refreshed on every key event in the editor
• The DataWORQ URL is now case-insensitive
• Centralized all supported file types
• Bug Fixes
• Fixed an issue where network drives weren't supported as mapped directories
• Fixed an issue where context menu options for the flow and file management screens weren't working
  • Delete
  • Rename
  • New Folder
• Fixed an issue with the delete hotkey in the Flow management screen
• Fixed an issue where context menu options for canvas nodes weren't working
  • Add Node
  • Connect Nodes
• Fixed an issue where File Watcher wasn't informing users of invalid watched directories
• Importing a flow now creates a flow with the same flow ID as the exported flow

6.3.1 - 11/28/2023

• Bug Fixes
• Updated the default SQL Server connection string to be consistent with the value in the JDBC connection string dropdown
• Fixed an issue where IBM Planning Analytics cellsets weren't being cleaned up after creation
• Fixed an issue where custom CAM namespaces weren't properly used in IBM Planning Analytics data connections
• Fixed a UI issue when selecting flows to export in the Flow Management screen
• Fixed a UI issue where the Flow/File Management screens weren't automatically refreshing on open

6.3.0- 11/06/2023

• Containerization support
• Added support for running DataWORQ in containers
• Using Podman for container infrastructure
• Created a Admin Tool (containerized web application) to help manage DataWORQ
  • Configuration of application properties
  • Check the status of running containers
  • Start/stop containers ad hoc
  • Import/export content store to/from containers
• Made performance improvements around the refreshing of multiple UI components, including opening/closing node edit mode and script node previewing
• Bug Fixes
• Fixed an issue when using parameters in the Planning Analytics Process node
• Fixed an issue where Forecast Strategy nodes weren't supporting some input column headers with spaces
• Fixed an issue where flows couldn't be exported via the Flow Management screen

6.2.2- 10/25/2023

• Amazon S3 Support
• Added Amazon S3 support
• New S3 Source Node
• New S3 Export Node
• File Management updates
• Removed Temporary and Working directories from application settings
• Remove the mappings entries in the mapping section of the directoryMappings.cfg file
• Created a Backups folder that is empty at the default folder location
• Import File Browser updates
• Default for Target location is now / if no file was selected
• Target location file browser can only browse folder locations
• Added a message to select a file if export file is chosen with no file selection
• Updated the file browser to use common file types (.txt, .csv, .xlsx, .dwexport)
• Bug Fixes
• Fixed an issue where the temporary folder used for forecast node models wasn't probably referenced
• Fixed an issue due to extra quotes when attempting to import flows
• Fixed an issue where files didn't have extensions properly applied after browsing in file browsers
• Fixed issue with setting up parameters for Adaptive data connections

6.2.0- 9/22/2023

• Adaptive Support
• Added Workday Adaptive support
• New Adaptive Source Node
• New Adaptive Export Node
• Script Node Improvements
• Script Source and Column nodes now support multiple input nodes
• Script Source and Column nodes now support defining output data types
• Bug Fixes
• Updated resource for Windows Integrated Login
• Fixed bug with record limit in mysql
• File Export node now adds the proper file extension

6.1.10- 8/02/2023

• Improvements
• Smartsheet support
  • Added Smartsheet data connection support
  • New Smartsheet Source node to pull data from a named Smartsheet sheet
  • New Smartsheet Export node to update an existing Smartsheet sheet
  • Added a new Pivot node to support dataframe pivot/unpivot functionality
• JDBC support
  • Updated supported JDBC driver versions
  • Updated default connection strings for JDBC connections
• IBM Planning Analytics support
  • Updated IBM PA Local/Cloud/PAaaS support for running flows
  • Updated SSL certificate validation for PA connections
• R installations newer than 4.2.1 are now supported
• Updated HTTP client connection pooling for:
  • Anaplan connections
  • Pigment connections
• Bug Fixes
  • Pigment support
  • Fixed an issue re: Pigment connections on flow import
  • Added Pigment connections to supported data connection parameters
  • Fixed an issue with parameter resolution for data connections in nodes
  • Fixed an issue where "." wasn't supported in flow names

6.1.6- 7/17/2023

• Improvements
• Added new flow import/export functionality
  • Flow Export
  • Admins can export flows via the Flow Management screen
  • Exports can be local downloads or exported to a virtual path on the server
  • Exports are given a .dwexport extension
  • Flow Import
  • Admins can import flows via the Flow Management screen
  • Imports are selected via the virtual file browser
  • Importing flows involves a mapping process for nodes with:
    • Flat file references
    • Data connections
    • Flow references
  • Legacy Flow Import
  • Legacy flows (.flx files) can be imported as well
  • Follows a similar mapping exercise to the above process
• Script node enhancements:
  • Improved node configuration re: output data types
  • Improved node configuration re: delimiters
  • Added support for multiple source data frames for a single script node
  • Improved script editor styling to allow for greater script visibility
• Cloud connector enhancements:
  • Added a new tab to show jobs initiated by Cloud Connector requests
  • Added a new tab to show Cloud Connector request errors
• JDBC driver version updates:
  • Microsoft SQL Server
  • MySQL
  • Oracle
  • PostgreSQL
• Improved logging:
  • Node IDs now included in error logs
  • Errors when activating licenses
  • Upstream nodes when a node throws an exception
  • When using Forecast Strategy/Forecast Single Strategy nodes with negative actuals
  • When using the Planning Analytics Source node against views that don't exist
  • When using the Planning Analytics Export node against views/dimensions/elements that don't exist
• Improved security admin setup when switching security modes and switching environments for the same security mode
• Improved Planning Analytics Process node status checking
• Updated the Flow Menu to remove the Data Connections option
• Bug Fixes
• Nodes
  • Improved exception handling when inferring Data Types for the Flat File Source node
  • Fixed an issue when Flow Output nodes had invalid flow IDs/run profiles saved to them
  • Fixed an issue when Planning Analytics Process nodes had no connection source saved to them
  • Fixed an issue when scoring data for an R model in the Forecast Single Strategy node
  • Fixed an issue where R models were being placed in the incorrect directory when running R models via the Forecast Strategy node
• Parameters
  • Fixed an issue when saving a new string parameter with no value
  • Newly added string parameters now default to an empty string
  • Newly added numeric parameters now default to 0
  • Fixed an issue when parameters had spaces in their names
  • Fixed an issue when adjacent parameters in a string had an underscore between them
• Cube Watcher
  • Fixed an issue where Cube Watcher would not connect on startup even if configured to
  • Fixed an issue with the Cube Watcher Connect on Startup label
• Cloud Connector
  • Fixed an issue when configuring Cloud Connector for the first time
• JDBC
  • Removed deprecated JDBC drivers
  • org.netezza.Driver
  • com.cloudera.impala.jdbc41.Driver
  • Updated JDBC drivers
  • com.mysql.jdbc.Driver
  • Fixed an issue when previewing with an invalid JDBC connection source
• Fixed an issue with files/flows being randomly sorted in lists
• Fixed an issue when enabling debug logging
• Fixed an issue with copy/pasting flows in the Flow Management screen
• Updated exception dialog styling
• Fixed an issue where data connections using the default password weren't being saved

6.0.7- 6/02/2023

• Bug Fixes
• Updated usernames where UUIDs were displayed
• Updated the error message when an incorrect API key header is used
• Fixed an issue re: seeding flow paths from the Open Flow browser dialog
• Removed the unused Log Output Directory property from scheduled jobs
• Disabled non-functional enable/disable buttons on cards

6.0.5- 6/01/2023

• Improvements
• Added parameter support for Command Line nodes
• Added support for seeding initial admins when changing security modes
• Bug Fixes
• Updated script editor styling
• Fixed an issue re: logged usernames for some security modes
• Fixed an issue re: rebuilding LP Optimization node models
• Updated LP Optimization node GMPL template parameter definitions
• Fixed an issue when saving new default flows

6.0.4- 5/24/2023

• Improvements
• Improved exception messaging when using the Forecast node
• Bug Fixes
• Always rebuild predictive models that shouldn't be serialized
  • Forecast
  • Forecast Single Strategy
  • Forecast Strategy
• Prevented the ability to unnecessarily rebuild predictive models via Run Profiles
  • Forecast
  • Forecast Single Strategy
  • Forecast Strategy
• Fixed an issue re: serializing predictive models
  • Text Analytics
  • Association
  • Classification
  • Regression
  • Cluster
  • Survival
• Fixed an issue impacting selecting AutoKMedians as a model type when using the Cluster Node
• Improved exception handling around opening Data Connections

6.0.3- 5/22/2023

• Bug Fixes
• Fixed an issue re: interpreting parameters with empty values for the PA process node
• Fixed an issue around flow ordering in the browser dialog
• Added support for trailing whitespace characters in file browsers
• Added support for flow browser name case insensitivity

6.0.2- 5/9/2023

• Bug Fixes
• Resolved issue where flows were incorrectly being saved as duplicates

6.0.1- 5/4/2023

• Improvements
• New security mode support:
  • Open ID Connect
  • SAML2
  • LDAP
  • Active Directory
• Application Managed Content:
  • Application managed content is centralized and available across all flows
  • Flows
  • SMTP Connection
  • Data Connections
• Virtual File Management:
  • Ability to map physical directories to virtual in-app directories
  • Virtual file resolution support for nodes
  • File Watcher is managed via virtual mappings
  • Default settings are now managed virtually
  • Scripting nodes (R and Python) now have virutal support
  • Upload file support and download file support
  • General file and folder management support such as copy/paste/move/delete
• New settings are supported:
  • Execution properties:
  • Python
  • R
  • GLPK
  • Command Line Node
  • Database properties
  • Security and password vault
  • Logging
  • Authentication properties
  • Directory mappings
• Execution updates:
  • Job state is captured prior to execution. This ensures the state of all flow components is consistent with the flow state at the time the flow is executed.
  • Improved data connection resolution
  • Improved parameter resolution
  • Flow resolution
  • JavaScript processing upgraded to use GraalVM
• Security & Encryption:
  • All user passwords are hashed with latest encryption technology
  • All connection credentials are securely encrypted
• API improvements:
  • Added API key security
  • Updated endpoints for job execution to match modern resource conventions
  • URL API uses updated flow paths
  • API error messaging improved
  • Cloud Connector now uses updated API convention
• Improved user management:
  • Admin and non-admin roles for users
  • Admins can manage user security, including setting roles and generating API keys
  • Improved support for user administration
• Legacy feature support:
  • Exercises are temporarily disabled
• UI updates:
  • Settings screens now instantly apply changes (save and cancel removed)
  • Styling improvements
  • Added new flow and file browsers
• Bug Fixes
• Resolved issue where node pane would open in uneditable state

5.4.2- 1/10/2023

• DataWORQ rebranding
• Rebranded Euclid Studio as DataWORQ
• Updated any references in-app or in documentation to the new product name
• Improvements
• Optimized the performance of Croston's/Improved Croston's models for the Forecast Strategy node
• Optimized Forecast Strategy node performance when querying metadata downstream
• Improved Forecast Accuracy node error reporting
• Improved error messaging for the Croston's predictive model type
• Bug Fixes
• Fixed an issue when checking for Forecast Strategy node R version compatibility
• Fixed an issue with the way the Forecast Accuracy node was sorting by role column values
• Fixed file dialog file extensions not being in sync with node file paths
• Fixed an issue where the NN predictive model type wouldn't work with integer target data
• Fixed an issue with ensemble predictive models erroring on averaging

5.3.15- 12/5/2022

• Improvements
• Performance improvements to Forecast Strategy node models:
  • Mean
  • Seasonal Decomposition
• Added support for Excel files (.xls, .xlsx, .xlsm):
  • Excel File Source Node
  • Excel File Export Node
• Added support for Pigment views:
  • Pigment Source Node
  • Pigment Export Node
• Added the ability to set the frequency at which the Planning Analytics Process node checks for the completion of a process
• Automatically retrying failed Planning Analytics Process node checks a set number of times before erroring
• Bug Fixes
• Fixed a bug re: Forecast Strategy models in R forecasting extra periods
• Stopped the Forecast Strategy node from forecasting future periods when using partitions
• Fixed an issue re: generating forecasts for the seasonal decomposition model
• Fixed issue where .jdt file support wasn't working correctly in the Flat File Source node
• Fixed an issue with dragging and dropping nodes with canvas scroll
• Fixed an issue with parameter override data types and formatting

5.3.7- 10/25/2022

• Improvements
• Added detailed performance information to job logs
• Updates to the predictive model types used by the Forecast Strategy/Forecast Single Strategy nodes, including:
  • Additions:
  • Improved Croston's
    • A one-step ahead forecast using Croston’s method, incorporating that prediction back into the training set.
  • Neural Network
  • Removals:
  • Naive
  • Seasonal Naive
  • Updates:
  • All ensemble models have been updated to represent the type of data they should be run on, including:
    • Good Signal
    • Poor Signal
    • Intermittent Data
• On Forecast Strategy node R errors, now writing out the entire error to the R temp directory.
• Truncating the Error output column to 255 characters for Forecast Strategy node R errors.

Bug Fixes

• Fixed a bug preventing the Forecast Strategy/Forecast Single Strategy nodes from running forecasts in some cases
• Fixed a bug with branch copy/duplictate functionality
• Fixed a bug where the active run profile wasn't updated to use the most recent flow parameter values
• Fix for R processes not being cleaned up after running/cancelling Forecast Strategy node runs
• Updated ensemble model names in Forecast Strategy node
• Forecast Strategy node Model Filter is no longer unselectable
• Fix for running the Improved Croston's model alone in the Forecast Strategy nodes
• Fix for running the Intermittent Data ensemble model in the Forecast Strategy nodes
• Documentation updates
• Fixes for when running multiple Forecast Strategy/Forecast Single Strategy nodes in parallel:
• Fix for strategy names in Scoring/Forecast periods not being correctly assigned across similar combos
• Updated R NA/null values to output as null

5.3.0- 10/05/2022

• Improvements
• Added multiple new nodes to enhance time series forecasting
  • Sequence (Row) - Fills in missing parts of a numeric sequence or time period.
  • Frequency (Calculation & Analysis) - Calculates the count or percentage of occurrences of a condition across rows.
  • Forecast Accuracy (Calculation & Analysis) - Calculates numerous accuracy metrics for numerous time series forecasts.
  • White Noise (Calculation & Analysis) - Calculates the amount of white noise present in a dataset.
  • Forecast Category (Calculation & Analysis) - Groups time series data into one of nine categories based on available records and their variation.
  • Forecast Single Strategy (Prediction) - Produce a single time series forecast across many datasets.
  • Forecast Strategy (Prediction) - Produce numerous time series forecast across many datasets.
  • New predictive model types supported, including:
  • Naive
  • Seasonal Naive
  • Mean
  • Arima
  • Exponential Smoothing
  • Seasonal Decomposition
  • Theta
  • Croston's
  • Linear Regression
  • Prophet
  • Various ensemble models composed of the above
• Various updates to provide clarity re: partition and role columns
• Update to Log4j config to not log performance metrics by default
• Partition node added support for time series data and splitting by record count.
• Added a timeout setting to all Script nodes
• Update to Chart node to support scatter charts
• Update to Cleanse node to support breaking out a Datetime column into multiple string columns
• Updated Script nodes to support multiple data frame delimeter types
• Planning Analytics Process Node now supports Euclid Studio parameters for TI Process parameter values.
• UI Updates
• A new node category called Calculation & Analysis was added.
• Time Series node was moved to the Calculation & Analysis category.
• Business Rule node was moved to the Calculation & Analysis category.
• Chart node was moved to the Calculation & Analysis category.
• Optimization node category color was changed.
• Charting colors were updated in the Chart node.
• Bug Fixes
• Fixed font sizes not being saved in Script nodes
• Fixed the canvas snapping to the left when selecting nodes off canvas
• Fixed the Filter node not converting between String and Datetime data types
• Fixed the Database export node not creating columns starting with numeric values
• Fixed the node state indicator position when a node contains a comment
• Updated the Chart node and Rest Command node to be previewable from the node context menu
• Fixed node and branch copy and duplication on the canvas
• Disabled interactive notes during node edit mode
• Fixed database connections disappearing when navigating to a flow via the API
• Fixed double clicking nodes sometimes causing a blank pane to open
• Node context menu fix for disabled nodes
• Add to Canvas style updates for disabled nodes

5.2.7- 8/22/2022

• Improvements
• Updates to script nodes styling
• Updates to Job Scheduling styling
• Updates to Cube Watch styling
• Updates to File Watcher styling
• License checks now happen on restart and once daily
• Updates to license related logging
• Bug Fixes
• Fixed issue with licensing entitlements

5.2.6- 8/11/2022

• Improvements
• Job Activity
  • Lazy load all jobs and paginate them (25,50,100)
  • Only store the last 1000 executed jobs
  • Ability to disable UI auto refresh/toast notifications

5.2.5 - 8/4/2022

• Improvements
• Added support for Flow failure on PA Process failure
• Improved PA Process logging
• Added support for Flow failure on Anaplan Action failure
• Improved Anaplan Action logging
• Added data type support for cells in Cube Watch views
• Added Cube Watch icon to top action bar when running
• Added Error badges to Cube Watch icon and settings tabs
• Added autocomplete and free form value input to Business Rules
• Improved Business Rules editing performance
• Various styling improvements
• Various documentation updates
• Bug Fixes
• Fixed issue with saving Flow database connections
• Fixed issue with overriding saved database connections
• Fixed Cube Watch configuration error
• Fixed script editor rendering issue
• Fixed Business Rules initial groupings

5.2.4 - 7/9/2022

• New Features
• Planning Analytics
  • Planning Analytics Source Node
  • Planning Analytics Process Output Node
  • Planning Analytics Export Node
  • Planning Analytics Cube Watcher
• Anaplan
  • Anaplan Transactional Source Node
  • Anaplan Bulk Source Node
  • Anaplan Process Output Node
  • Anaplan Transactional Export Node
  • Anaplan Bulk Export Node
• Data Connections
  • Planning Analytics Local Connection
  • Planning Analytics Cloud Connection
  • Anaplan Connection
  • JDBC Connection
• Fixes and Improvements
• Added font size and expansion to all editor nodes
• Canvas performance improvements
• Database Connection improvements
• Navigation improvements
• Save and change rollback performance improvements

5.1.3.0 - 5/16/2022

• Bug Fixes
• Optimization Node styling
• Error styling
• Flow Output Node Parameter Support
• Grid toggle is fixed
• Flow navigation error
• Node confirmation fix
• Double click properly opens now
• Improvements
• Drag and drop auto link from canvas
• License settings shortcut in footer

5.1.0.0 - 4/04/2022

• New Features
• New user interface
  • A complete overhaul re: the user expereience
  • These changes are detailed in the updated Reference Guide
• In-app reference documentation
  • Accessed through the Help pane
• In-app examples and exercises to get you started
  • Exercises can be updated with a click of a button as new content is released
  • Includes Node Examples and How-To Exercises
• Drag and drop nodes from the Node Menu
• Keyboard shortcuts
• Auto arrange branches