Note
This repository contains experimental code that is not supported like other Airbyte projects, and is provided for reference purposes only. For assistance with this project, please use this repository's Issues tab to report any faults or feature requests.
A CLI tool (abtfexport) that fetches resources from the Airbyte API and converts them into Terraform configuration files for easier migration to Infrastructure as Code.
- Fetch Airbyte sources, destinations, and connections
- Support for Airbyte API Bearer token authentication
- Configuration via file, environment variables, or command-line flags
Download the latest release for your platform from the GitHub Releases page. Extract the archive and add the binary to your PATH, or run locally.
go build -o abtfexportgo install github.com/Airbyte-Solutions-Team/terraform-airbyte-exporter@latest# Export all Airbyte resources (sources, destinations, and connections) to Terraform
abtfexport --api-url https://api.airbyte.com --client-id "your-client-id" --client-secret "your-client-secret"
# Export from a specific workspace
abtfexport --workspace "workspace-id" --client-id "..." --client-secret "..."
# Export a specific connection only
abtfexport --connection-id "connection-id" --client-id "..." --client-secret "..."
# Split resources into separate files
abtfexport --split --client-id "..." --client-secret "..."Important: If you use Self-Managed Enterprise or an Open Source deployment, your URL will need to include /api/public at the end. For example, https://airbyte.contoso.com/api/public.
You can configure the tool using:
- Configuration file (
~/.abtfexport.yaml):
# Example configuration file for abtfexport
api:
url: "https://api.airbyte.com" # Can also be set via AIRBYTE_API_URL environment variable
client_id: "your_client_id" # Can also be set via AIRBYTE_API_CLIENT_ID environment variable
client_secret: "your_client_secret" # Can also be set via AIRBYTE_API_CLIENT_SECRET environment variable- Environment variables:
export AIRBYTE_API_URL="https://api.airbyte.com"
export AIRBYTE_API_CLIENT_ID="your-airbyte-client-id"
export AIRBYTE_API_CLIENT_SECRET="your-airbyte-client-secret"- Command-line flags (see
abtfexport --helpfor all options):
abtfexport --api-url https://api.airbyte.com --client-id "..." --client-secret "..."To use this tool, you'll need an Airbyte client ID and secret:
- Log into your Airbyte account
- Go to Settings → Account → Applications
- Create a new application
See the Airbyte API documentation for more details.
.
├── cmd/
│ ├── root.go # Root command and CLI configuration
│ └── airbyte.go # Export logic and Airbyte API integration
├── internal/
│ ├── airbyte/
│ │ └── types.go # Airbyte API response types
│ ├── api/
│ │ └── client.go # HTTP client for API calls
│ └── converter/
│ ├── terraform.go # JSON to Terraform HCL converter
│ └── cron.go # Schedule conversion utilities
├── main.go # Entry point with version info
├── go.mod
└── README.md
The Terraform converter in internal/converter/terraform.go can be extended to handle specific resource types or add custom conversion logic.
This project uses semantic versioning for releases (e.g., v1.0.0, v1.1.0, v2.0.0).
To create a new release:
- Ensure all changes are committed and pushed to the
mainbranch - Create and push a new tag:
git tag -a v0.1.0 -m "Release v0.1.0" git push origin v0.1.0 - GitHub Actions will automatically build binaries for all platforms and create a GitHub release with:
- Multi-platform binaries (macOS, Linux, Windows)
- SHA256 checksums
- Changelog generated from commits
To test the release process locally without publishing:
# Install goreleaser (macOS)
brew install goreleaser
# Test the configuration
goreleaser check
# Build locally without releasing
goreleaser build --snapshot --clean
# Check the built binaries in ./dist/MIT