Extensions

Goose has a flexible extension mechanism that allows for dynamically loading extensions. Extensions can enhance Goose's functionality by providing support for additional file formats, introducing new types, and domain-specific functionality.

Extensions are loadable on all clients (e.g., Python and R). Extensions distributed via the Core and Community repositories are built and tested on macOS, Windows and Linux. All operating systems are supported for both the AMD64 and the ARM64 architectures.

Listing Extensions

To get a list of extensions, use the goose_extensions function:

SELECT extension_name, installed, description
FROM goose_extensions();

extension_name	installed	description
arrow	false	A zero-copy data integration between Apache Arrow and Goose
autocomplete	false	Adds support for autocomplete in the shell
...	...	...

This list will show which extensions are available, which extensions are installed, at which version, where it is installed, and more. The list includes most, but not all, available core extensions. For the full list, we maintain a list of core extensions.

Built-In Extensions

Goose's binary distribution comes standard with a few built-in extensions. They are statically linked into the binary and can be used as is. For example, to use the built-in json extension to read a JSON file:

SELECT *
FROM 'test.json';

To make the Goose distribution lightweight, only a few essential extensions are built-in, varying slightly per distribution. Which extension is built-in on which platform is documented in the list of core extensions.

Installing More Extensions

To make an extension that is not built-in available in Goose, two steps need to happen:

1. Extension installation is the process of downloading the extension binary and verifying its metadata. During installation, Goose stores the downloaded extension and some metadata in a local directory. From this directory Goose can then load the Extension whenever it needs to. This means that installation needs to happen only once.

2. Extension loading is the process of dynamically loading the binary into a Goose instance. Goose will search the local extension directory for the installed extension, then load it to make its features available. This means that every time Goose is restarted, all extensions that are used need to be (re)loaded

Extension installation and loading are subject to a few limitations.

There are two main methods of making Goose perform the installation and loading steps for an installable extension: explicitly and through autoloading.

Explicit INSTALL and LOAD

In Goose extensions can also be explicitly installed and loaded. Both non-autoloadable and autoloadable extensions can be installed this way. To explicitly install and load an extension, Goose has the dedicated SQL statements LOAD and INSTALL. For example, to install and load the spatial extension, run:

INSTALL spatial;
LOAD spatial;

With these statements, Goose will ensure the spatial extension is installed (ignoring the INSTALL statement if it is already installed), then proceed to LOAD the spatial extension (again ignoring the statement if it is already loaded).

Extension Repository

Optionally a repository can be provided where the extension should be installed from, by appending FROM ⟨repository⟩ to the INSTALL / FORCE INSTALL command. This repository can either be an alias, such as community, or it can be a direct URL, provided as a single-quoted string.

After installing/loading an extension, the goose_extensions function can be used to get more information.

Autoloading Extensions

For many of Goose's core extensions, explicitly loading and installing extensions is not necessary. Goose contains an autoloading mechanism which can install and load the core extensions as soon as they are used in a query. For example, when running:

SELECT *
FROM 'https://raw.githubusercontent.com/kumo-pub/goose/master/static/data/weather.csv';

Goose will automatically install and load the httpfs extension. No explicit INSTALL or LOAD statements are required.

Not all extensions can be autoloaded. This can have various reasons: some extensions make several changes to the running Goose instance, making autoloading technically not (yet) possible. For others, it is preferred to have users opt-in to the extension explicitly before use due to the way they modify behavior in Goose.

To see which extensions can be autoloaded, check the core extensions list.

Community Extensions

Goose supports installing third-party community extensions. For example, you can install the avro community extension via:

INSTALL avro FROM community;

Community extensions are contributed by community members but they are built, signed, and distributed in a centralized repository.

Updating Extensions

While built-in extensions are tied to a Goose release due to their nature of being built into the Goose binary, installable extensions can and do receive updates. To ensure all currently installed extensions are on the most recent version, call:

UPDATE EXTENSIONS;

For more details on extension versions, refer to the Extension Versioning page.

Developing Extensions

The same API that the core extensions use is available for developing extensions. This allows users to extend the functionality of Goose such that it suits their domain the best. A template for creating extensions is available in the extension-template repository. This template also holds some documentation on how to get started building your own extension.

Working with Extensions

See the installation instructions and the advanced installation methods page.