Flat File

  • Updated on Jan 13, 2023
  • Published on Jan 28, 2022
  • 2 minute(s) read
Prev Next

Description

The Flat File Source Node is used to import data from flat files.


Configuration Options

Basic Configuration Options

Setting Description\Parameters
File Name Location of the file to import. This is the full path to the file, including the drive on which it lives. The path can either be entered manually here or selected using the File Browser.
Delimiter Specifies the delimiter that should be used to parse the selected file.

Options
  • Comma
  • Tab
  • Pipe
Quote Character Specifies the character used to wrap column data between each delimiter.

Options
  • Auto - data fields wrapped with default (double quote)
  • None - data fields not wrapped
  • Single - data fields wrapped with single quotes
  • Double - data fields wrapped with double quotes
First Row is Header Specifies if the first row in the file should be loaded as row header metadata. If this box is checked, the first row will be loaded as header metadata.
Use Data Type Template When selected, this option will load the files data types from a user-defined .jdt file. The .jdt and .csv files must have the same name.

Advanced Configuration Options

Click the Data Type Assignment button to view these options

Setting Description
Column Data Type This setting can be overridden.

Actions

Action Description
Infer Data Types Automatically infer the data types used for each column. These data types can be overridden in the advanced section.
Browse Opens the File Browser Dialog where a source file can be selected for the node to use.
Preview Once the node is configured, the source data can be previewed at any time. If not using a data type template, be sure to Infer Data Types.
Data Type Assignment Navigates to the advanced section where you can override data types.

File Browser Dialog

The Browse button opens the File Browser dialog where a source file can be selected for the node.

The Root Directory is the full path to the location of the source file. After entering a path, push the Refresh button to update the list of relevant files based on the File Type filter.

Selecting a file will populate the File Name input box. Click the Open button to return to the main dialog.


Misc. Node Requirements

  • This node supports comma, pipe, and tab-delimited files only.
  • Data must be formatted to conform to standard delimited file conventions. For example, each row should be terminated by a new line (\r\n).
  • Rows should not contain \r or \n characters outside of the newline.

Allowed Date Formats

When selecting the Date/Time data type, the following date formats are allowed:

  • yyyyMMdd
  • dd-MM-yyyy
  • yyyy-MM-dd
  • MM/dd/yyyy
  • yyyy/MM/dd
  • dd MMM yyyy
  • dd MMMM yyyy
  • yyyyMMdd HHmm
  • dd-MM-yyyy HH:mm
  • yyyy-MM-dd HH:mm
  • MM/dd/yyyy HH:mm
  • yyyy/MM/dd HH:mm
  • dd MMM yyyy HH:mm
  • dd MMMM yyyy HH:mm
  • yyyyMMddHHmmss
  • yyyyMMdd HHmmss
  • dd-MM-yyyy HH:mm:ss
  • yyyy-MM-dd HH:mm:ss
  • MM/dd/yyyy HH:mm:ss
  • yyyy/MM/dd HH:mm:ss
  • dd MMM yyyy HH:mm:ss
  • dd MMMM yyyy HH:mm:ss
  • yyyy-MM-dd HH:mm:ss.SSS
  • EEE MMM dd HH:mm:ss zzz yyyy

Node Use Case Example

See here for an example of this node in use.