File Data Connector

Manually Upload files or ingest data from Cloud Storage Solutions

Overview

The File Data Connector allows you to manually upload files to the Aisera Platform or ingest files from Cloud Based file storage solutions.

Prepare

If you are manually uploading files to the Aisera Platform, it is recommended that you gather all files together in one folder to streamline the process of uploading files.

Supported File Types

The File Data source supports the following file types:

• .txt

• .csv

• .html

• .doc, .docx

• .pdf

• .json

• .xls, .xlsx

• .pptx

Supported Cloud Integrations

The following Cloud Integrations are supported for the File Data source:

• AWS S3

• Google Drive

• Google Cloud

• Box

• OneDrive

• Egnyte

• Azure Storage

• Snowflake

Integration Setup

If you are manually uploading filed you do not need to configure an integration. Proceed to the Data Source Setup. Otherwise, create an integration for the Cloud Integration you will be ingesting data from.

Data Source Setup

To create a File Data Source:

1. In the Aisera Admin UI navigate to Settings > Data Source

2. Click + New Data Source

3. In the menu, select File Data and click Next

4. Specify the files to be ingested onto the Aisera Platform

   1. Manual Upload

      1. Select the Uploaded Files radio button

      2. Click the Paperclip Icon to browse for files

   2. Cloud Files

      1. Select the Cloud Files radio button

      2. Select the Integration for the Cloud Integration you will be ingesting from

5. Fill in the General Data Source information, then click Next

   Field	Description
   Name	The name of the Data Source will appear as in the Admin UI
   Data Type	The type of data being uploaded to the Aisera Platform
   Schedule	How frequently the data source will request new information. This only affects the Data Source if the Data Source is configured to ingest Cloud Files
   File Format	Determines how data is ingested into the Aisera Platform. Can specify: CSV TSV JSON For all other file types, choose Raw
   Description	A short description of the data source

6. All remaining options are optional. See the section below for more information. Click Next until the window closes to complete the creation of the Data Source

Optional Configurations

Custom Script

This script runs on each individual record after it has been fetched and parsed from the source system, but before field mappings are applied. It receives a single JSON object representing one record and must return a modified JSON object. This allows you to add, edit, or remove fields on individual records before they are mapped into the Aisera Platform. For help creating, testing, and implementing these scripts, reach out to Aisera.

Transformation Script

This script runs on the entire raw string payload before it is parsed into individual records. It receives the raw body string and the connector's auth credentials, and must return a string. This allows you to reshape or restructure the raw payload before any record-level processing occurs. For help creating, testing, and implementing these scripts, reach out to Aisera.