s3-file

Imports a file from any S3-compatible endpoint.

Imports a file from any S3-compatible endpoint. S3="Simple Storage Service"

S3 is a cloud-based object storage service provided by AmazonWebServices, DigitalOcean, and other providers. Objects (e.g. files and the metadata about them) are stored in virtual containers called "buckets". The security of an object (here, a file) can be managed via Access Control Lists (ACLs). They can be accessed via a URL endpoint.

This Data-flo adaptor accesses files stored on S3, downloads them, and imports them into Data-flo as files, ready for further processing.

Arguments

Inputs:

url: The URL of the file to be imported, e.g. https://bucket-name.s3.example.com/file/path.

key: If the ACL (access control list) of the object is private, provide the key that when combined with the secret, gives permission for downloading. More about ACLs on Wikipedia

secret: If the ACL (access control list) of the object is private, provide the secret that when combined with the key, gives permission for downloading.

Output:

file: The imported file.

Example

Importing a CSV file from s3 endpoint, then converting it to a datatable

Possible use cases

• Pulling a Newick tree file into Data-flo to become input to export-to-microreact-project

• Importing a csv file so it can become input to the import-csv-file-to-datatable adaptor