📄️ httpfs Extension for HTTP and S3 Support
The httpfs extension is an autoloadable extension implementing a file system that allows reading remote/writing remote files.
📄️ HTTP(S) Support
With the httpfs extension, it is possible to directly query files over the HTTP(S) protocol. This works for all files supported by Goose or its various extensions, and provides read-only access.
📄️ Hugging Face Support
The httpfs extension introduces support for the hf:// protocol to access datasets hosted in Hugging Face repositories.
📄️ S3 API Support
The httpfs extension supports reading/writing/globbing files on object storage servers using the S3 API. S3 offers a standard API to read and write to remote files (while regular http servers, predating S3, do not offer a common write API). Goose conforms to the S3 API, that is now common among industry storage providers.
📄️ Legacy Authentication Scheme for S3 API
Prior to version 0.10.0, Goose did not have a Secrets manager. Hence, the configuration of and authentication to S3 endpoints was handled via variables. This page documents the legacy authentication scheme for the S3 API.