Support Kafka IAM auth fields

[rmorehig]

What changed

• Added Kafka AWS IAM/OAUTHBEARER auth options to defineKafkaConnection.
• Emitted the matching .connection directives:
  • KAFKA_SASL_OAUTHBEARER_METHOD
  • KAFKA_SASL_OAUTHBEARER_AWS_REGION
  • KAFKA_SASL_OAUTHBEARER_AWS_ROLE_ARN
  • KAFKA_SASL_OAUTHBEARER_AWS_EXTERNAL_ID
• Updated migration parsing and TypeScript emission so existing .connection files round-trip into SDK definitions.
• Bumped @tinybirdco/sdk to 0.0.74.

Why

../analytics supports Kafka IAM authentication through those OAUTHBEARER AWS fields in connection datafiles. The SDK can now author and migrate those definitions.

Example

Authoring the connection in TypeScript:

import { defineKafkaConnection, secret } from "@tinybirdco/sdk";

export const awsMsk = defineKafkaConnection("aws_msk", {
  bootstrapServers: "b-1.msk.example.com:9098,b-2.msk.example.com:9098",
  securityProtocol: "SASL_SSL",
  saslMechanism: "OAUTHBEARER",
  saslOauthbearerMethod: "AWS",
  saslOauthbearerAwsRegion: "eu-west-1",
  saslOauthbearerAwsRoleArn: secret("KAFKA_AWS_ROLE_ARN"),
  saslOauthbearerAwsExternalId: secret("KAFKA_AWS_EXTERNAL_ID"),
});

Generated .connection datafile, which is also what migration parsing reads later:

TYPE kafka
KAFKA_BOOTSTRAP_SERVERS b-1.msk.example.com:9098,b-2.msk.example.com:9098
KAFKA_SECURITY_PROTOCOL SASL_SSL
KAFKA_SASL_MECHANISM OAUTHBEARER
KAFKA_SASL_OAUTHBEARER_METHOD AWS
KAFKA_SASL_OAUTHBEARER_AWS_REGION eu-west-1
KAFKA_SASL_OAUTHBEARER_AWS_ROLE_ARN {{ tb_secret("KAFKA_AWS_ROLE_ARN") }}
KAFKA_SASL_OAUTHBEARER_AWS_EXTERNAL_ID {{ tb_secret("KAFKA_AWS_EXTERNAL_ID") }}

When migrating that datafile back to TypeScript, the parser emits the same SDK options:

export const awsMsk = defineKafkaConnection("aws_msk", {
  bootstrapServers: "b-1.msk.example.com:9098,b-2.msk.example.com:9098",
  securityProtocol: "SASL_SSL",
  saslMechanism: "OAUTHBEARER",
  saslOauthbearerMethod: "AWS",
  saslOauthbearerAwsRegion: "eu-west-1",
  saslOauthbearerAwsRoleArn: secret("KAFKA_AWS_ROLE_ARN"),
  saslOauthbearerAwsExternalId: secret("KAFKA_AWS_EXTERNAL_ID"),
});

Validation

• pnpm vitest run src/schema/connection.test.ts src/generator/connection.test.ts src/migrate/parse-connection.test.ts src/cli/commands/migrate.test.ts
• pnpm typecheck
• pnpm test:run
• pnpm build