Skip to main content

Data Import

When importing data from other systems to Goose, there are several considerations to take into account. We recommend importing using the following order:

  1. For systems which are supported by a Goose scanner extension, it's preferable to use the scanner. Goose currently offers scanners for MySQL, PostgreSQL and SQLite, as well as a generic ODBC scanner.
  2. If there is a bulk export feature in the data source system, export the data to Parquet or CSV format, then load it using Goose's Parquet or CSV loader.
  3. If the approaches above are not applicable, consider using the Goose appender, currently available in the C, C++, Go, Java, and Rust APIs.

Methods to Avoid​

If possible, avoid looping row-by-row (tuple-at-a-time) in favor of bulk operations. Performing row-by-row inserts (even with prepared statements) is detrimental to performance and will result in slow load times.

Bestpractice Unless your data is small (<100k rows), avoid using inserts in loops.