S3

  • Updated on Oct 10, 2023
  • Published on Oct 6, 2023
  • 1 minute(s) read
Prev Next

Description

The S3 Source Node is used to import data from Amazon S3.


Configuration Options

Basic Configuration Options

Setting Description\Parameters
S3 Connection The S3 connection to query against. These connections can be configured in Flow Connections.
Bucket The bucket from which to pull data.
Folder The (optional) folder from which to pull data. If permissions are set to only allow access to objects in a specific folder, this field is required to be able to browse available files.
File The file from which to pull data. This is the full path of the bucket file.
Delimiter Specifies the delimiter that should be used to parse the selected file.

Options
  • Comma
  • Tab
  • Pipe
Quote Character Specifies the character used to wrap column data between each delimiter.

Options
  • Auto - data fields wrapped with default (double quote)
  • None - data fields not wrapped
  • Single - data fields wrapped with single quotes
  • Double - data fields wrapped with double quotes
First Row is Header Specifies if the first row in the file should be loaded as row header metadata. If this box is checked, the first row will be loaded as header metadata.

Advanced Configuration Options

Click the Data Type Assignment button to view these options

Setting Description
Column Data Type This setting can be overridden.

Actions

Action Description
Infer Data Types Automatically infer the data types used for each column. These data types can be overridden in the advanced section.
Preview Once the node is configured, the source data can be previewed at any time. Be sure to Infer Data Types.
Data Type Assignment Navigates to the advanced section where you can override data types.

Misc. Node Requirements

  • This node supports comma, pipe, and tab-delimited files only.
  • Data must be formatted to conform to standard delimited file conventions. For example, each row should be terminated by a new line (\r\n).
  • Rows should not contain \r or \n characters outside of the newline.