• Connectors Learn about the available connectors

Data can be imported from one or more sources. Data is imported using connectors, which can read data from places such as files or databases.

Basic (Single Source)

Import data from one source.

read:
  - file:
      name: file.xlsx
      sheet_name: my sheet

Union

Combine data from multiple sources, stacked vertically, equivalent to a union in SQL.

Where headers differ, columns will be matched by name.

read:
  - union:
      sources:
    	  - file:
      	  	name: file1.xlsx
        - file:
      	  	name: file2.xlsx

column a	+	column b	→	column a b

Union is the default aggregation for multiple reads.

Concatenate

Combine data from multiple sources, stacked horizontally.

read:
  - concatenate:
      sources:
    	  - file:
      	  	name: file1.xlsx
        - file:
      	  	name: file2.xlsx

column a	+	column b	→	column column a b

Join

Joins can be done using data from any connector, or even custom functions. Want to join a csv to data from a database? Go for it!

Import data from two sources and merge the data, equivalent to a join in SQL.

This can be used recursively to join more than two tables.

read:
  - join:
    	how: inner
      left_on: column
      right_on: column
      sources:
    	  - file:
      	  	name: file1.xlsx
        - file:
      	  	name: file2.xlsx