From 1a2af55b2741ebecf9f14d1c6540730fa0333ddc Mon Sep 17 00:00:00 2001 From: Dayana Date: Wed, 22 Jul 2026 15:01:02 -0700 Subject: [PATCH 1/2] gRPC support for opensearch-java (#2062) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat: add transparent gRPC transport as separate java-client-grpc module Adds a transparent gRPC transport layer that routes bulk operations over gRPC for improved performance while falling back to REST for all other operations. Isolated in a separate java-client-grpc module to prevent classpath conflicts. Includes: - GrpcTransport + HybridTransport (automatic routing and fallback) - Translation layer (BulkRequest/Response <-> protobuf conversion) - TLS support (trust cert, trust store, mTLS, insecure, hostname override) - Basic auth, AWS SigV4, and JWT authentication interceptors - Channel health monitoring via gRPC connectivity state machine - Integration tests (framework-compliant, version-gated to 3.5.0+) - Sample code and CI configuration Signed-off-by: Dayana Jean * ci: retrigger CI Signed-off-by: Dayana Jean * Making the FieldMappingUtil package private Co-authored-by: Andriy Redko Signed-off-by: Dayana * Making this package private Apply suggestion from @reta Co-authored-by: Andriy Redko Signed-off-by: Dayana * Making BasicAuthInterceptor package private Apply suggestion from @reta Co-authored-by: Andriy Redko Signed-off-by: Dayana * Remove unused toHttpStatus method and fix test package for package-private classes - Removed GrpcStatusConverter.toHttpStatus() which was unused anywhere in the codebase (per maintainer feedback) - Moved TranslationTest to the translation package so it can access package-private FieldMappingUtil and GrpcStatusConverter.convert() Signed-off-by: Dayana Jean * refactor: split AWS SigV4 into AwsGrpcTransport per maintainer feedback Follows the same pattern as ApacheHttpClient5Transport (general) vs AwsSdk2Transport (AWS-specific) in the existing codebase. Changes: - Created AwsGrpcTransport: extends GrpcTransport, adds SigV4 signing - AwsGrpcTransport.awsBuilder(host, port).sigV4(...).tls(...).build() - Requires sigV4Config and TLS - Overrides preProcessBulk() for payload hash computation - Removed SigV4 from GrpcTransport: - Removed .sigV4() builder method - Removed sigV4Interceptor field - Added protected preProcessBulk() hook for subclasses - GrpcTransport is now purely general-purpose (basic auth, JWT, TLS) - Updated tests to use AwsGrpcTransport for SigV4 tests Signed-off-by: Dayana Jean * refactor: remove silent REST fallback from HybridTransport Per maintainer feedback: if the intent is to use gRPC for a supported endpoint, errors should propagate to the user rather than silently falling back to REST. Changes: - HybridTransport no longer catches gRPC errors and retries via REST - Removed fallbackOnError constructor parameter and field - Routing behavior preserved: unsupported endpoints → REST directly - gRPC-supported endpoints: errors propagate to caller - Simplified performRequest/performRequestAsync (no try/catch) - Updated tests: testGrpcErrorPropagatesForSupportedEndpoint Behavior: client.bulk(req) → gRPC (errors propagate if gRPC fails) client.search(req) → REST (not supported by gRPC, routed directly) Signed-off-by: Dayana Jean * docs: update comments to reflect routing instead of fallback Stale references to 'automatic REST fallback' replaced with accurate 'REST routing for unsupported endpoints' language throughout. - GrpcTransport: updated javadoc and error message - GrpcDemo: rewrote Demo 3 to show REST routing (not fallback on error) - GrpcAwsSigV4: updated to use AwsGrpcTransport.awsBuilder() - GrpcBulkIT: renamed testRestFallback → testRestRouting Signed-off-by: Dayana Jean * fix: make integration test base classes self-contained Per maintainer feedback: OpenSearchJavaClientTestCase and TestcontainersThreadFilter are internal test classes not exposed for external module use. Changes: - AbstractGrpcIT: now extends nothing, uses standard JUnit 4 + Assume - Removed dependency on OpenSearchJavaClientTestCase - Removed @ThreadLeakFilters(TestcontainersThreadFilter) - Uses Assume.assumeTrue() with manual version parsing - Reads cluster config from system properties directly - GrpcTransportSupport: converted from interface (extending OpenSearchTransportSupport) to utility class - GrpcBulkIT: no longer implements GrpcTransportSupport interface Signed-off-by: Dayana Jean * fix: lower java-client-grpc baseline from JDK 11 to JDK 8 Per maintainer feedback: opensearch-java baseline is JDK 8 and gRPC-Java supports Java 8. No technical reason to require JDK 11. Changes: - build.gradle.kts: targetCompatibility/sourceCompatibility → 1.8 - GrpcSigV4Test: replaced 'var' (Java 10+) with explicit types Signed-off-by: Dayana Jean * fix: remove explicit jackson test deps (transitive via java-client) Per maintainer feedback: opensearch-java has a hard dependency on Jackson 3.x, so it comes transitively. No need to declare it again. Signed-off-by: Dayana Jean * fix: wire up integration tests with java21 source set Per maintainer feedback: integration tests were not being compiled or run. Added the standard java21 source set configuration used by java-client to java-client-grpc. Changes: - Added unitTest/integrationTest task definitions - Added java21 source set (src/test/java11) gated on JDK 21+ - Added test framework, testcontainers, opensearch-testcontainers deps - Added static import for JUnit Assert in GrpcBulkIT - Integration tests now compile and will run with: ./gradlew :java-client-grpc:integrationTest -Dtests.opensearch.version=3.5.0 Signed-off-by: Dayana Jean * ci: add OpenSearch 3.5.0 to integration test matrix Adds the first gRPC-capable version to the test matrix so the gRPC integration tests run in CI. Includes both Java 21 and Java 25. Signed-off-by: Dayana Jean * fix: replace wildcard import with explicit imports (spotless) Spotless rejects wildcard imports. Replaced 'import static org.junit.Assert.*' with explicit imports for assertEquals, assertFalse, assertNotNull, assertTrue. Signed-off-by: Dayana Jean * fix: remove java21 classes from unitTest task The java21 source set only contains integration tests (integTest package). Adding it to unitTest causes 'No tests found' failure since the filter excludes integTest classes. Only integrationTest needs the java21 classes. Signed-off-by: Dayana Jean * fix: spotless formatting for samples (license header, import order) - GrpcDemo.java: replaced custom header with standard Apache-2.0 license - GrpcAwsSigV4.java: fixed import ordering (AwsGrpcTransport alphabetical) - Removed unused imports Signed-off-by: Dayana Jean * fix: skip gRPC integration tests when port is unreachable assumeGrpcSupported() now verifies both: 1. Server version is 3.5.0+ (via REST info endpoint) 2. gRPC port is actually reachable (TCP socket check) This prevents test failures when running integrationTest against OpenSearch versions that don't have gRPC enabled or when the gRPC port isn't exposed by testcontainers. Signed-off-by: Dayana Jean * fix: only enable gRPC in testcontainer for OpenSearch 3.5.0+ GrpcTestContainerRule now checks the tests.opensearch.version property before adding gRPC configuration. Older versions (1.x, 2.x) don't support aux.transport.types and would fail to start. On pre-3.5.0 versions: - Container starts without gRPC config (REST only) - gRPC port not exposed - Tests skip via assumeGrpcSupported() (version check + port check) Signed-off-by: Dayana Jean * ci: exclude grpc integration tests until OpenSearch 3.5.0 is released OpenSearch 3.5.0 Docker image is not yet published, so the gRPC integration tests cannot run in CI. Changes: - Exclude :java-client-grpc:integrationTest from the main CI run (all matrix versions are pre-3.5.0) - Comment out 3.5.0 entries in test matrix (uncomment when released) The gRPC integration tests can still be run locally against a container started manually or once 3.5.0 is published. Signed-off-by: Dayana Jean * fix: graceful skip on container failure + re-enable 3.5.0 in CI OpenSearch 3.5.0 Docker image is published. Re-enabled in CI matrix. Changes: - GrpcTestContainerRule.before(): catches ContainerLaunchException and calls Assume.assumeTrue(false) to skip tests gracefully instead of failing the build - Added 5-minute startup timeout for CI environments - Removed duplicate disk watermark env var - Re-enabled 3.5.0 in test-integration.yml matrix - Removed -x :java-client-grpc:integrationTest exclusion On pre-3.5.0 versions: container starts without gRPC, tests skip via assumeGrpcSupported(). On 3.5.0+: if container fails for any reason, tests skip instead of failing the build. Signed-off-by: Dayana Jean * fix: spotless formatting on GrpcTestContainerRule Signed-off-by: Dayana Jean * fix: remove grpcStatusToHttpStatus from FieldMappingUtil Per maintainer feedback: method was only used in tests, not in production code. Removed the method and its 7 associated tests. Signed-off-by: Dayana Jean * fix: skip gRPC tests early with assumeTrue for unsupported versions Per maintainer feedback: use assumeTrue in the class rule to skip tests immediately for older versions, rather than conditionally configuring the container. Changes: - GrpcTestContainerRule.before(): assumeTrue("OpenSearch should support gRPC", supportsGrpc(version)) skips for pre-3.5.0 - createContainer(): always configures gRPC (only reached for 3.5.0+) - Removed conditional gRPC config logic Signed-off-by: Dayana Jean * fix: make GrpcChannelFactory methods package-private Signed-off-by: Dayana Jean * fix: apply maintainer suggestions on visibility and naming 1. BasicAuthInterceptor: constructor now package-private 2. GrpcChannelFactory: class now package-private (final class, no public) 3. GrpcTestContainerRule renamed to OpenSearchGrpcTestContainerRule 4. Updated references in AbstractGrpcIT Signed-off-by: Dayana Jean --------- Signed-off-by: Dayana Jean Signed-off-by: Dayana Co-authored-by: Andriy Redko (cherry picked from commit 44bbc73acc1e63539c92d2020be3bf63257afb18) Signed-off-by: opensearch-ci-bot --- .ci/opensearch/Dockerfile | 4 + .ci/opensearch/docker-compose.yml | 1 + .ci/opensearch/opensearch.yml | 4 + .github/workflows/test-integration.yml | 2 + CHANGELOG.md | 1 + java-client-grpc/build.gradle.kts | 152 +++++++ .../transport/grpc/AwsGrpcTransport.java | 156 +++++++ .../transport/grpc/BasicAuthInterceptor.java | 96 ++++ .../transport/grpc/GrpcChannelFactory.java | 204 +++++++++ .../grpc/GrpcChannelHealthMonitor.java | 244 ++++++++++ .../transport/grpc/GrpcSigV4Config.java | 122 +++++ .../transport/grpc/GrpcSigV4Interceptor.java | 240 ++++++++++ .../client/transport/grpc/GrpcTlsConfig.java | 280 ++++++++++++ .../client/transport/grpc/GrpcTransport.java | 406 +++++++++++++++++ .../transport/grpc/GrpcTransportOptions.java | 166 +++++++ .../transport/grpc/HybridTransport.java | 119 +++++ .../transport/grpc/JwtAuthInterceptor.java | 91 ++++ .../translation/BulkRequestConverter.java | 341 ++++++++++++++ .../translation/BulkResponseConverter.java | 165 +++++++ .../grpc/translation/FieldMappingUtil.java | 121 +++++ .../grpc/translation/GrpcStatusConverter.java | 170 +++++++ .../grpc/BasicAuthInterceptorTest.java | 65 +++ .../transport/grpc/GrpcChannelTest.java | 192 ++++++++ .../client/transport/grpc/GrpcSigV4Test.java | 343 ++++++++++++++ .../transport/grpc/GrpcTransportTest.java | 267 +++++++++++ .../grpc/JwtAuthInterceptorTest.java | 65 +++ .../grpc/translation/TranslationTest.java | 426 ++++++++++++++++++ .../integTest/grpc/AbstractGrpcIT.java | 149 ++++++ .../opensearch/integTest/grpc/GrpcBulkIT.java | 259 +++++++++++ .../integTest/grpc/GrpcTransportSupport.java | 46 ++ .../grpc/OpenSearchGrpcTestContainerRule.java | 135 ++++++ samples/build.gradle.kts | 5 + .../client/samples/GrpcAwsSigV4.java | 137 ++++++ .../opensearch/client/samples/GrpcBulk.java | 209 +++++++++ .../opensearch/client/samples/GrpcDemo.java | 267 +++++++++++ settings.gradle.kts | 1 + validate-grpc-integration.sh | 147 ++++++ 37 files changed, 5798 insertions(+) create mode 100644 java-client-grpc/build.gradle.kts create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/AwsGrpcTransport.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/BasicAuthInterceptor.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcChannelFactory.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcChannelHealthMonitor.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcSigV4Config.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcSigV4Interceptor.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTlsConfig.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTransport.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTransportOptions.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/HybridTransport.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/JwtAuthInterceptor.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/BulkRequestConverter.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/BulkResponseConverter.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/FieldMappingUtil.java create mode 100644 java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/GrpcStatusConverter.java create mode 100644 java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/BasicAuthInterceptorTest.java create mode 100644 java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcChannelTest.java create mode 100644 java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcSigV4Test.java create mode 100644 java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcTransportTest.java create mode 100644 java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/JwtAuthInterceptorTest.java create mode 100644 java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/translation/TranslationTest.java create mode 100644 java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/AbstractGrpcIT.java create mode 100644 java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcBulkIT.java create mode 100644 java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcTransportSupport.java create mode 100644 java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/OpenSearchGrpcTestContainerRule.java create mode 100644 samples/src/main/java/org/opensearch/client/samples/GrpcAwsSigV4.java create mode 100644 samples/src/main/java/org/opensearch/client/samples/GrpcBulk.java create mode 100644 samples/src/main/java/org/opensearch/client/samples/GrpcDemo.java create mode 100755 validate-grpc-integration.sh diff --git a/.ci/opensearch/Dockerfile b/.ci/opensearch/Dockerfile index b2322ce110..f28e916e44 100644 --- a/.ci/opensearch/Dockerfile +++ b/.ci/opensearch/Dockerfile @@ -6,4 +6,8 @@ ARG opensearch_yml=$opensearch_path/config/opensearch.yml ARG SECURE_INTEGRATION ENV OPENSEARCH_INITIAL_ADMIN_PASSWORD=0_aD^min_0 + +# Copy custom opensearch.yml with gRPC transport configuration +COPY opensearch.yml $opensearch_yml + RUN if [ "$SECURE_INTEGRATION" != "true" ] ; then echo "plugins.security.disabled: true" >> $opensearch_yml; fi diff --git a/.ci/opensearch/docker-compose.yml b/.ci/opensearch/docker-compose.yml index 2580e07f44..e96fa39a25 100644 --- a/.ci/opensearch/docker-compose.yml +++ b/.ci/opensearch/docker-compose.yml @@ -14,4 +14,5 @@ services: - bootstrap.memory_lock=true ports: - "9200:9200" + - "9400:9400" user: opensearch diff --git a/.ci/opensearch/opensearch.yml b/.ci/opensearch/opensearch.yml index 50b154702b..2d15a5f2d4 100644 --- a/.ci/opensearch/opensearch.yml +++ b/.ci/opensearch/opensearch.yml @@ -1,2 +1,6 @@ cluster.name: "docker-cluster" network.host: 0.0.0.0 + +# gRPC transport configuration +aux.transport.types: [transport-grpc] +aux.transport.transport-grpc.port: '9400' diff --git a/.github/workflows/test-integration.yml b/.github/workflows/test-integration.yml index 404e5a823b..08b20d89fb 100644 --- a/.github/workflows/test-integration.yml +++ b/.github/workflows/test-integration.yml @@ -30,6 +30,8 @@ jobs: - { opensearch_version: 3.0.0, java: 21, os: ubuntu-latest } - { opensearch_version: 3.2.0, java: 21, os: ubuntu-latest } - { opensearch_version: 3.2.0, java: 25, os: ubuntu-latest } + - { opensearch_version: 3.5.0, java: 21, os: ubuntu-latest } + - { opensearch_version: 3.5.0, java: 25, os: ubuntu-latest } steps: - name: Checkout Java Client uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 diff --git a/CHANGELOG.md b/CHANGELOG.md index 00991e593f..d13515922b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) - Run Java client integration tests with a Testcontainers-managed OpenSearch instance by default ([#2033](https://github.com/opensearch-project/opensearch-java/pull/2033)) - Added `equals()` and `hashCode()` implementations to `FieldValue` ([#1998](https://github.com/opensearch-project/opensearch-java/pull/1998)) - Add document lifecycle guide and runnable sample ([#2017](https://github.com/opensearch-project/opensearch-java/pull/2017)) +- Add transparent gRPC transport with HybridTransport (bulk over gRPC, REST fallback), translation layer, TLS, basic auth, AWS SigV4, and JWT support ([#2062](https://github.com/opensearch-project/opensearch-java/pull/2062)) ### Dependencies - Bump `org.apache.httpcomponents.client5:httpclient5` from 5.6 to 5.6.1 ([#1967](https://github.com/opensearch-project/opensearch-java/pull/1967)) diff --git a/java-client-grpc/build.gradle.kts b/java-client-grpc/build.gradle.kts new file mode 100644 index 0000000000..951ae9f013 --- /dev/null +++ b/java-client-grpc/build.gradle.kts @@ -0,0 +1,152 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +plugins { + java + `java-library` + `maven-publish` + id("opensearch-java.spotless-conventions") +} + +repositories { + mavenLocal() + maven(url = "https://ci.opensearch.org/ci/dbc/snapshots/maven/") + mavenCentral() +} + +java { + targetCompatibility = JavaVersion.VERSION_1_8 + sourceCompatibility = JavaVersion.VERSION_1_8 + + withJavadocJar() + withSourcesJar() +} + +val opensearchVersion = "3.5.0-SNAPSHOT" +val grpcVersion = "1.68.0" +val protobufVersion = "3.25.5" +val opensearchProtobufVersion = "1.2.0" + +dependencies { + // Depend on java-client core + api(project(":java-client")) + + // gRPC runtime + api("io.grpc", "grpc-api", grpcVersion) + api("io.grpc", "grpc-stub", grpcVersion) + api("io.grpc", "grpc-protobuf", grpcVersion) + api("io.grpc", "grpc-netty-shaded", grpcVersion) + api("com.google.protobuf", "protobuf-java", protobufVersion) + + // OpenSearch Protobufs (compiled protobuf Java classes) + api("org.opensearch", "protobufs", opensearchProtobufVersion) + + // For AwsSdk2 SigV4 support (optional, compile-only) + compileOnly("software.amazon.awssdk", "sdk-core", "[2.21,3.0)") + compileOnly("software.amazon.awssdk", "auth", "[2.21,3.0)") + compileOnly("software.amazon.awssdk", "http-auth-aws", "[2.21,3.0)") + + // Test dependencies + testImplementation("io.grpc", "grpc-testing", grpcVersion) + testImplementation("junit", "junit", "4.13.2") + testImplementation("org.opensearch.client", "opensearch-rest-client", opensearchVersion) + testImplementation("software.amazon.awssdk", "sdk-core", "[2.21,3.0)") + testImplementation("software.amazon.awssdk", "auth", "[2.21,3.0)") + testImplementation("software.amazon.awssdk", "http-auth-aws", "[2.21,3.0)") +} + +tasks.test { + systemProperty("tests.security.manager", "false") +} + +val unitTest = tasks.register("unitTest") { + filter { + excludeTestsMatching("org.opensearch.client.opensearch.integTest.*") + } + systemProperty("tests.security.manager", "false") +} + +val integrationTest = tasks.register("integrationTest") { + filter { + includeTestsMatching("org.opensearch.client.opensearch.integTest.*") + } + systemProperty("tests.security.manager", "false") + systemProperty("https", System.getProperty("https", "false")) + systemProperty("user", System.getProperty("user", "admin")) + systemProperty("password", System.getProperty("password", "admin")) + systemProperty("tests.opensearch.testcontainers.enabled", + System.getProperty("tests.opensearch.testcontainers.enabled", "true")) + systemProperty("tests.opensearch.version", + System.getProperty("tests.opensearch.version", opensearchVersion)) +} + +// Integration tests require Java 21+ and live in src/test/java11 +val runtimeJavaVersion = (System.getProperty("runtime.java")?.toInt())?.let(JavaVersion::toVersion) ?: JavaVersion.current() +if (runtimeJavaVersion >= JavaVersion.VERSION_21) { + val java21: SourceSet = sourceSets.create("java21") { + java { + compileClasspath += sourceSets.main.get().output + sourceSets.test.get().output + runtimeClasspath += sourceSets.main.get().output + sourceSets.test.get().output + srcDir("src/test/java11") + } + } + + configurations[java21.implementationConfigurationName].extendsFrom(configurations.testImplementation.get()) + configurations[java21.runtimeOnlyConfigurationName].extendsFrom(configurations.testRuntimeOnly.get()) + + dependencies { + "java21Implementation"("org.opensearch.test", "framework", opensearchVersion) { + exclude(group = "org.hamcrest") + } + "java21Implementation"("org.opensearch:opensearch-testcontainers:4.1.0") + "java21Implementation"("org.testcontainers:testcontainers:2.0.5") + } + + tasks.named("compileJava21Java") { + targetCompatibility = JavaVersion.VERSION_21.toString() + sourceCompatibility = JavaVersion.VERSION_21.toString() + } + + tasks.named("compileTestJava") { + targetCompatibility = JavaVersion.VERSION_21.toString() + sourceCompatibility = JavaVersion.VERSION_21.toString() + } + + integrationTest.configure { + testClassesDirs += java21.output.classesDirs + classpath = sourceSets["java21"].runtimeClasspath + } +} + +tasks.withType { + manifest { + attributes["Implementation-Title"] = "OpenSearch Java Client - gRPC Transport" + attributes["Implementation-Vendor"] = "OpenSearch" + attributes["Implementation-URL"] = "https://github.com/opensearch-project/opensearch-java/" + } + + metaInf { + from("../LICENSE.txt") + from("../NOTICE.txt") + } +} + +publishing { + publications { + create("publishMaven") { + from(components["java"]) + pom { + name.set("OpenSearch Java Client - gRPC Transport") + packaging = "jar" + artifactId = "opensearch-java-grpc" + description.set("gRPC transport for the OpenSearch Java Client.") + url.set("https://github.com/opensearch-project/opensearch-java/") + } + } + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/AwsGrpcTransport.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/AwsGrpcTransport.java new file mode 100644 index 0000000000..94e3f67250 --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/AwsGrpcTransport.java @@ -0,0 +1,156 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import io.grpc.ManagedChannel; +import javax.annotation.Nullable; +import org.opensearch.client.json.JsonpMapper; +import org.opensearch.client.transport.TransportOptions; + +/** + * AWS-specific gRPC transport that adds SigV4 signing to the gRPC channel. + *

+ * This follows the same pattern as {@code AwsSdk2Transport} vs + * {@code ApacheHttpClient5Transport} — separating general transport from + * AWS-specific authentication. + *

+ * Usage: + * 

{@code
+ * var grpcTransport = AwsGrpcTransport.builder("domain.us-east-1.es.amazonaws.com", 9400)
+ *     .jsonpMapper(new JacksonJsonpMapper())
+ *     .tls(GrpcTlsConfig.builder().build())
+ *     .sigV4(GrpcSigV4Config.builder()
+ *         .region(Region.US_EAST_1)
+ *         .service("es")
+ *         .credentialsProvider(DefaultCredentialsProvider.create())
+ *         .build())
+ *     .build();
+ * }
+ */ +public class AwsGrpcTransport extends GrpcTransport { + + private final GrpcSigV4Interceptor sigV4Interceptor; + + AwsGrpcTransport( + ManagedChannel channel, + JsonpMapper jsonpMapper, + GrpcTransportOptions grpcOptions, + @Nullable TransportOptions transportOptions, + GrpcSigV4Interceptor sigV4Interceptor + ) { + super(channel, jsonpMapper, grpcOptions, transportOptions); + this.sigV4Interceptor = sigV4Interceptor; + } + + @Override + protected void preProcessBulk(org.opensearch.protobufs.BulkRequest protoRequest) { + if (sigV4Interceptor != null) { + String payloadHash = GrpcSigV4Interceptor.computePayloadHash(protoRequest.toByteArray()); + sigV4Interceptor.setPayloadHash(payloadHash); + } + } + + /** + * Creates a builder for AwsGrpcTransport. + * + * @param host the gRPC server hostname + * @param port the gRPC server port (default: 9400) + */ + public static Builder awsBuilder(String host, int port) { + return new Builder(host, port); + } + + public static final class Builder { + private final String host; + private final int port; + private JsonpMapper jsonpMapper; + private GrpcTransportOptions grpcOptions = GrpcTransportOptions.defaults(); + private TransportOptions transportOptions; + private GrpcTlsConfig tlsConfig; + private GrpcSigV4Config sigV4Config; + private ManagedChannel channel; + + Builder(String host, int port) { + this.host = host; + this.port = port; + } + + public Builder jsonpMapper(JsonpMapper mapper) { + this.jsonpMapper = mapper; + return this; + } + + public Builder grpcOptions(GrpcTransportOptions options) { + this.grpcOptions = options; + return this; + } + + public Builder transportOptions(TransportOptions options) { + this.transportOptions = options; + return this; + } + + /** + * Configures TLS for the gRPC channel. Required for SigV4. + */ + public Builder tls(GrpcTlsConfig tlsConfig) { + this.tlsConfig = tlsConfig; + return this; + } + + /** + * Configures AWS SigV4 signing for the gRPC channel. + * TLS is required when using SigV4. + * + * @param sigV4Config the SigV4 configuration (region, service, credentials) + */ + public Builder sigV4(GrpcSigV4Config sigV4Config) { + this.sigV4Config = sigV4Config; + return this; + } + + /** + * Inject a pre-built channel (primarily for testing). + */ + public Builder channel(ManagedChannel channel) { + this.channel = channel; + return this; + } + + public AwsGrpcTransport build() { + if (jsonpMapper == null) { + throw new IllegalArgumentException("jsonpMapper is required"); + } + if (sigV4Config == null) { + throw new IllegalArgumentException("sigV4 config is required for AwsGrpcTransport. Use GrpcTransport for non-AWS usage."); + } + if (tlsConfig == null) { + throw new IllegalStateException("TLS is required when using SigV4 signing. Configure TLS with .tls() before .sigV4()."); + } + + ManagedChannel ch = this.channel; + GrpcSigV4Interceptor sigV4InterceptorRef = null; + + if (ch == null) { + java.util.List interceptors = new java.util.ArrayList<>(); + + sigV4InterceptorRef = new GrpcSigV4Interceptor(sigV4Config, host); + interceptors.add(sigV4InterceptorRef); + + try { + ch = GrpcChannelFactory.createChannel(host, port, tlsConfig, grpcOptions, interceptors); + } catch (java.io.IOException e) { + throw new IllegalStateException("Failed to create gRPC channel: " + e.getMessage(), e); + } + } + + return new AwsGrpcTransport(ch, jsonpMapper, grpcOptions, transportOptions, sigV4InterceptorRef); + } + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/BasicAuthInterceptor.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/BasicAuthInterceptor.java new file mode 100644 index 0000000000..014bfa7258 --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/BasicAuthInterceptor.java @@ -0,0 +1,96 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import io.grpc.CallOptions; +import io.grpc.Channel; +import io.grpc.ClientCall; +import io.grpc.ClientInterceptor; +import io.grpc.ForwardingClientCall.SimpleForwardingClientCall; +import io.grpc.Metadata; +import io.grpc.MethodDescriptor; +import java.nio.charset.StandardCharsets; +import java.util.Base64; + +/** + * gRPC {@link ClientInterceptor} that adds HTTP Basic Authentication credentials + * to every outgoing gRPC call as metadata. + *

+ * Attaches the {@code Authorization: Basic } header as gRPC metadata, + * which the OpenSearch server reads for authentication. + *

+ * This is the gRPC equivalent of the REST client's basic auth configuration + * ({@code UsernamePasswordCredentials} on the HTTP client). + *

+ * Usage: + * 

{@code
+ * BasicAuthInterceptor interceptor = new BasicAuthInterceptor("admin", "admin");
+ * ManagedChannel channel = ManagedChannelBuilder.forAddress("localhost", 9400)
+ *     .intercept(interceptor)
+ *     .build();
+ * }
+ * + * @see OpenSearch Basic Auth + */ +class BasicAuthInterceptor implements ClientInterceptor { + + private static final Metadata.Key AUTHORIZATION_KEY = Metadata.Key.of("Authorization", Metadata.ASCII_STRING_MARSHALLER); + + private final String authHeaderValue; + + /** + * Creates a basic auth interceptor with the given credentials. + * + * @param username the username + * @param password the password + * @throws IllegalArgumentException if username or password is null + */ + BasicAuthInterceptor(String username, String password) { + if (username == null) { + throw new IllegalArgumentException("username cannot be null"); + } + if (password == null) { + throw new IllegalArgumentException("password cannot be null"); + } + this.authHeaderValue = encodeBasicAuth(username, password); + } + + @Override + public ClientCall interceptCall( + MethodDescriptor method, + CallOptions callOptions, + Channel next + ) { + return new SimpleForwardingClientCall(next.newCall(method, callOptions)) { + @Override + public void start(Listener responseListener, Metadata headers) { + headers.put(AUTHORIZATION_KEY, authHeaderValue); + super.start(responseListener, headers); + } + }; + } + + /** + * Returns the encoded Authorization header value (for testing purposes). + */ + String getAuthHeaderValue() { + return authHeaderValue; + } + + /** + * Encodes username:password as a Basic auth header value. + * + * @return "Basic " + */ + private static String encodeBasicAuth(String username, String password) { + String credentials = username + ":" + password; + String encoded = Base64.getEncoder().encodeToString(credentials.getBytes(StandardCharsets.UTF_8)); + return "Basic " + encoded; + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcChannelFactory.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcChannelFactory.java new file mode 100644 index 0000000000..6601c8eb39 --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcChannelFactory.java @@ -0,0 +1,204 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import io.grpc.ClientInterceptor; +import io.grpc.ManagedChannel; +import io.grpc.ManagedChannelBuilder; +import io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts; +import io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder; +import io.grpc.netty.shaded.io.netty.handler.ssl.SslContext; +import io.grpc.netty.shaded.io.netty.handler.ssl.SslContextBuilder; +import io.grpc.netty.shaded.io.netty.handler.ssl.util.InsecureTrustManagerFactory; +import java.io.File; +import java.io.FileInputStream; +import java.io.IOException; +import java.security.KeyStore; +import java.util.List; +import java.util.concurrent.TimeUnit; +import javax.annotation.Nullable; +import javax.net.ssl.SSLException; +import javax.net.ssl.TrustManagerFactory; + +/** + * Factory that creates gRPC {@link ManagedChannel} instances configured for + * either plaintext or TLS communication. + *

+ * Uses gRPC-Netty's shaded SSL context for TLS channels, supporting: + *

    + *
  • System default trust store (JVM cacerts)
    • + *
  • Custom CA certificate (PEM file)
    • + *
  • Custom trust store (JKS/PKCS12)
    • + *
  • Mutual TLS with client certificate
    • + *
  • Insecure mode (trust all — development only)
    • + *
+ */ +final class GrpcChannelFactory { + + private GrpcChannelFactory() { + // utility class + } + + /** + * Creates a plaintext (unencrypted) gRPC channel. + * + * @param host the server hostname + * @param port the server port + * @param options channel options (keepalive, max message size, etc.) + * @param interceptors optional client interceptors (e.g., auth) + * @return a configured ManagedChannel + */ + static ManagedChannel createPlaintextChannel( + String host, + int port, + GrpcTransportOptions options, + List interceptors + ) { + ManagedChannelBuilder builder = ManagedChannelBuilder.forAddress(host, port).usePlaintext(); + + applyOptions(builder, options); + applyInterceptors(builder, interceptors); + + return builder.build(); + } + + /** + * Creates a TLS-encrypted gRPC channel. + * + * @param host the server hostname + * @param port the server port + * @param tlsConfig TLS configuration (certs, trust store, insecure mode) + * @param options channel options (keepalive, max message size, etc.) + * @param interceptors optional client interceptors (e.g., auth) + * @return a configured ManagedChannel with TLS + * @throws IOException if SSL context creation fails + */ + static ManagedChannel createTlsChannel( + String host, + int port, + GrpcTlsConfig tlsConfig, + GrpcTransportOptions options, + List interceptors + ) throws IOException { + SslContext sslContext = buildSslContext(tlsConfig); + + NettyChannelBuilder builder = NettyChannelBuilder.forAddress(host, port).sslContext(sslContext); + + // Hostname verification override — verify cert against a different hostname + // than the one we're connecting to. Useful when connecting via IP. + if (tlsConfig.hostnameOverride() != null) { + builder.overrideAuthority(tlsConfig.hostnameOverride()); + } + + applyOptions(builder, options); + applyInterceptors(builder, interceptors); + + return builder.build(); + } + + /** + * Creates a channel based on configuration — TLS if config is provided, plaintext otherwise. + * + * @param host the server hostname + * @param port the server port + * @param tlsConfig TLS configuration, or null for plaintext + * @param options channel options + * @param interceptors client interceptors + * @return a configured ManagedChannel + * @throws IOException if TLS context creation fails + */ + static ManagedChannel createChannel( + String host, + int port, + @Nullable GrpcTlsConfig tlsConfig, + GrpcTransportOptions options, + List interceptors + ) throws IOException { + if (tlsConfig != null && tlsConfig.isEnabled()) { + return createTlsChannel(host, port, tlsConfig, options, interceptors); + } + return createPlaintextChannel(host, port, options, interceptors); + } + + // ─── SSL Context Builder ───────────────────────────────────────────────────── + + private static SslContext buildSslContext(GrpcTlsConfig config) throws IOException { + try { + SslContextBuilder sslBuilder = GrpcSslContexts.forClient(); + + // Trust configuration + if (config.isInsecure()) { + // Trust all certificates — development only + sslBuilder.trustManager(InsecureTrustManagerFactory.INSTANCE); + } else if (config.trustCertificatePath() != null) { + // Trust a specific CA certificate (PEM file) + sslBuilder.trustManager(new File(config.trustCertificatePath())); + } else if (config.trustStorePath() != null) { + // Trust from a Java KeyStore + TrustManagerFactory tmf = buildTrustManagerFactory(config); + sslBuilder.trustManager(tmf); + } + // else: use system default trust store (JVM cacerts) + + // Client certificate for mutual TLS + if (config.clientCertificatePath() != null && config.clientKeyPath() != null) { + File certFile = new File(config.clientCertificatePath()); + File keyFile = new File(config.clientKeyPath()); + if (config.clientKeyPassword() != null) { + sslBuilder.keyManager(certFile, keyFile, config.clientKeyPassword()); + } else { + sslBuilder.keyManager(certFile, keyFile); + } + } + + return sslBuilder.build(); + } catch (SSLException e) { + throw new IOException("Failed to build SSL context for gRPC channel", e); + } + } + + private static TrustManagerFactory buildTrustManagerFactory(GrpcTlsConfig config) throws IOException { + try { + KeyStore trustStore = KeyStore.getInstance(config.trustStoreType()); + try (FileInputStream fis = new FileInputStream(config.trustStorePath())) { + char[] password = config.trustStorePassword() != null ? config.trustStorePassword().toCharArray() : null; + trustStore.load(fis, password); + } + TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm()); + tmf.init(trustStore); + return tmf; + } catch (Exception e) { + throw new IOException("Failed to load trust store: " + config.trustStorePath(), e); + } + } + + // ─── Channel Configuration ─────────────────────────────────────────────────── + + private static void applyOptions(ManagedChannelBuilder builder, GrpcTransportOptions options) { + if (options.maxInboundMessageSize() > 0) { + builder.maxInboundMessageSize(options.maxInboundMessageSize()); + } + if (options.keepAliveTimeMs() > 0) { + builder.keepAliveTime(options.keepAliveTimeMs(), TimeUnit.MILLISECONDS); + } + if (options.keepAliveTimeoutMs() > 0) { + builder.keepAliveTimeout(options.keepAliveTimeoutMs(), TimeUnit.MILLISECONDS); + } + builder.keepAliveWithoutCalls(options.keepAliveWithoutCalls()); + if (options.idleTimeoutMs() > 0) { + builder.idleTimeout(options.idleTimeoutMs(), TimeUnit.MILLISECONDS); + } + } + + private static void applyInterceptors(ManagedChannelBuilder builder, List interceptors) { + if (interceptors != null && !interceptors.isEmpty()) { + builder.intercept(interceptors); + } + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcChannelHealthMonitor.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcChannelHealthMonitor.java new file mode 100644 index 0000000000..89308fffa1 --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcChannelHealthMonitor.java @@ -0,0 +1,244 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import io.grpc.ConnectivityState; +import io.grpc.ManagedChannel; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.logging.Level; +import java.util.logging.Logger; + +/** + * Monitors the health of a gRPC {@link ManagedChannel} using its built-in connectivity state machine. + *

+ * gRPC channels have 5 states: + *

    + *
  • IDLE — No active connection, will connect on next RPC
    • + *
  • CONNECTING — Establishing connection (name resolution, TCP, TLS handshake)
    • + *
  • READY — Connection established, RPCs can be sent
    • + *
  • TRANSIENT_FAILURE — Connection lost, retrying with backoff
    • + *
  • SHUTDOWN — Channel has been shut down permanently
    • + *
+ *

+ * This class provides: + *

    + *
  • {@link #isReady()} — Quick check if channel can serve RPCs immediately
    • + *
  • {@link #isHealthy()} — Channel is READY, IDLE, or CONNECTING (not failed/shutdown)
    • + *
  • {@link #getState()} — Current connectivity state
    • + *
  • {@link #waitForReady(long, TimeUnit)} — Block until channel becomes READY
    • + *
  • {@link #startMonitoring()} — Background state monitoring with callbacks
    • + *
+ *

+ * Usage: + * 

{@code
+ * GrpcChannelHealthMonitor monitor = new GrpcChannelHealthMonitor(channel);
+ * monitor.startMonitoring(); // begin watching state changes
+ *
+ * if (monitor.isReady()) {
+ *     // channel is connected and ready
+ * }
+ *
+ * // Block until ready (with timeout)
+ * boolean ready = monitor.waitForReady(5, TimeUnit.SECONDS);
+ * }
+ */ +public class GrpcChannelHealthMonitor { + + private static final Logger logger = Logger.getLogger(GrpcChannelHealthMonitor.class.getName()); + + private final ManagedChannel channel; + private final AtomicBoolean monitoring = new AtomicBoolean(false); + private volatile ConnectivityState lastKnownState; + private volatile StateChangeListener listener; + + /** + * Callback interface for channel state changes. + */ + @FunctionalInterface + public interface StateChangeListener { + /** + * Called when the channel state changes. + * + * @param previousState the state before the change + * @param newState the current state after the change + */ + void onStateChanged(ConnectivityState previousState, ConnectivityState newState); + } + + public GrpcChannelHealthMonitor(ManagedChannel channel) { + this.channel = channel; + this.lastKnownState = channel.getState(false); // don't trigger connection + } + + /** + * Returns the current connectivity state of the channel. + * Does NOT trigger a connection attempt if the channel is IDLE. + */ + public ConnectivityState getState() { + lastKnownState = channel.getState(false); + return lastKnownState; + } + + /** + * Returns the current connectivity state, optionally triggering a connection + * if the channel is currently IDLE. + * + * @param requestConnection if true and state is IDLE, initiates connection + */ + public ConnectivityState getState(boolean requestConnection) { + lastKnownState = channel.getState(requestConnection); + return lastKnownState; + } + + /** + * Returns true if the channel is in READY state — connection is established + * and RPCs can be sent immediately without waiting. + */ + public boolean isReady() { + return channel.getState(false) == ConnectivityState.READY; + } + + /** + * Returns true if the channel is in a healthy state — READY, IDLE, or CONNECTING. + * Returns false if in TRANSIENT_FAILURE or SHUTDOWN. + */ + public boolean isHealthy() { + ConnectivityState state = channel.getState(false); + return state == ConnectivityState.READY || state == ConnectivityState.IDLE || state == ConnectivityState.CONNECTING; + } + + /** + * Returns true if the channel is in TRANSIENT_FAILURE — connection was lost + * and the channel is retrying with backoff. + */ + public boolean isInTransientFailure() { + return channel.getState(false) == ConnectivityState.TRANSIENT_FAILURE; + } + + /** + * Returns true if the channel has been shut down. + */ + public boolean isShutdown() { + return channel.isShutdown() || channel.getState(false) == ConnectivityState.SHUTDOWN; + } + + /** + * Blocks until the channel reaches READY state or the timeout expires. + * Triggers a connection attempt if the channel is IDLE. + * + * @param timeout the maximum time to wait + * @param unit the time unit + * @return true if the channel became READY within the timeout, false otherwise + * @throws InterruptedException if the waiting thread is interrupted + */ + public boolean waitForReady(long timeout, TimeUnit unit) throws InterruptedException { + long deadlineNanos = System.nanoTime() + unit.toNanos(timeout); + + // Trigger connection if idle + ConnectivityState state = channel.getState(true); + + while (state != ConnectivityState.READY) { + if (state == ConnectivityState.SHUTDOWN) { + return false; // permanently closed + } + + long remainingNanos = deadlineNanos - System.nanoTime(); + if (remainingNanos <= 0) { + return false; // timeout + } + + // Wait for state to change + Object lock = new Object(); + synchronized (lock) { + final ConnectivityState currentState = state; + channel.notifyWhenStateChanged(currentState, () -> { + synchronized (lock) { + lock.notifyAll(); + } + }); + lock.wait(TimeUnit.NANOSECONDS.toMillis(remainingNanos)); + } + + state = channel.getState(true); + } + + return true; + } + + /** + * Initiates a connection if the channel is IDLE. + * This is useful to "warm up" the channel before the first RPC, + * reducing latency on the first actual request. + */ + public void connectIfIdle() { + ConnectivityState state = channel.getState(false); + if (state == ConnectivityState.IDLE) { + channel.getState(true); // triggers connection + logger.log(Level.FINE, "Triggered connection from IDLE state"); + } + } + + /** + * Starts background monitoring of channel state changes. + * Logs state transitions and invokes the registered {@link StateChangeListener}. + *

+ * Monitoring continues until the channel is SHUTDOWN or {@link #stopMonitoring()} is called. + */ + public void startMonitoring() { + if (!monitoring.compareAndSet(false, true)) { + return; // already monitoring + } + lastKnownState = channel.getState(false); + watchForStateChange(lastKnownState); + } + + /** + * Stops background state monitoring. + */ + public void stopMonitoring() { + monitoring.set(false); + } + + /** + * Sets a listener that will be called on every state change. + * + * @param listener the callback, or null to remove + */ + public void setStateChangeListener(StateChangeListener listener) { + this.listener = listener; + } + + private void watchForStateChange(ConnectivityState currentState) { + if (!monitoring.get() || currentState == ConnectivityState.SHUTDOWN) { + return; + } + + channel.notifyWhenStateChanged(currentState, () -> { + ConnectivityState newState = channel.getState(false); + ConnectivityState previousState = lastKnownState; + lastKnownState = newState; + + logger.log(Level.INFO, "gRPC channel state changed: {0} → {1}", new Object[] { previousState, newState }); + + // Invoke listener + StateChangeListener currentListener = this.listener; + if (currentListener != null) { + try { + currentListener.onStateChanged(previousState, newState); + } catch (Exception e) { + logger.log(Level.WARNING, "StateChangeListener threw exception", e); + } + } + + // Continue watching + watchForStateChange(newState); + }); + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcSigV4Config.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcSigV4Config.java new file mode 100644 index 0000000000..26720e1309 --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcSigV4Config.java @@ -0,0 +1,122 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import javax.annotation.Nonnull; +import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider; +import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider; +import software.amazon.awssdk.regions.Region; + +/** + * Configuration for AWS SigV4 signing of gRPC requests. + *

+ * Required parameters: + *

    + *
  • {@link #region()} — AWS region (e.g., us-east-1)
    • + *
+ * Optional parameters: + *
    + *
  • {@link #service()} — signing service name, "es" (default) or "aoss" for Serverless
    • + *
  • {@link #credentialsProvider()} — AWS credential provider (default: DefaultCredentialsProvider)
    • + *
+ * + * @see AWS SigV4 support for OpenSearch clients + */ +public final class GrpcSigV4Config { + + private final Region region; + private final String service; + private final AwsCredentialsProvider credentialsProvider; + + private GrpcSigV4Config(Builder builder) { + this.region = builder.region; + this.service = builder.service; + this.credentialsProvider = builder.credentialsProvider; + } + + /** + * The AWS region for signing (e.g., us-east-1, us-west-2). + */ + @Nonnull + public Region region() { + return region; + } + + /** + * The AWS signing service name. + *
    + *
  • "es" — Amazon OpenSearch Service (managed)
    • + *
  • "aoss" — Amazon OpenSearch Serverless
    • + *
+ * Default: "es" + */ + @Nonnull + public String service() { + return service; + } + + /** + * The AWS credentials provider used to obtain signing credentials. + * Default: {@link DefaultCredentialsProvider} (env vars, ~/.aws/credentials, instance profile, etc.) + */ + @Nonnull + public AwsCredentialsProvider credentialsProvider() { + return credentialsProvider; + } + + public static Builder builder() { + return new Builder(); + } + + public static final class Builder { + private Region region; + private String service = "es"; + private AwsCredentialsProvider credentialsProvider; + + /** + * Sets the AWS region for signing. Required. + */ + public Builder region(Region region) { + this.region = region; + return this; + } + + /** + * Sets the signing service name. + * Use "es" for managed OpenSearch, "aoss" for Serverless. + * Default: "es" + */ + public Builder service(String service) { + this.service = service; + return this; + } + + /** + * Sets the AWS credentials provider. + * Default: {@link DefaultCredentialsProvider} + */ + public Builder credentialsProvider(AwsCredentialsProvider credentialsProvider) { + this.credentialsProvider = credentialsProvider; + return this; + } + + public GrpcSigV4Config build() { + if (region == null) { + throw new IllegalArgumentException("region is required for SigV4 signing"); + } + if (service == null || service.isEmpty()) { + throw new IllegalArgumentException("service name cannot be empty"); + } + if (credentialsProvider == null) { + credentialsProvider = DefaultCredentialsProvider.create(); + } + return new GrpcSigV4Config(this); + } + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcSigV4Interceptor.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcSigV4Interceptor.java new file mode 100644 index 0000000000..586fd8aa6c --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcSigV4Interceptor.java @@ -0,0 +1,240 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import io.grpc.CallOptions; +import io.grpc.Channel; +import io.grpc.ClientCall; +import io.grpc.ClientInterceptor; +import io.grpc.ForwardingClientCall.SimpleForwardingClientCall; +import io.grpc.Metadata; +import io.grpc.MethodDescriptor; +import java.net.URI; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.util.List; +import java.util.Map; +import java.util.logging.Level; +import java.util.logging.Logger; +import software.amazon.awssdk.http.ContentStreamProvider; +import software.amazon.awssdk.http.SdkHttpFullRequest; +import software.amazon.awssdk.http.SdkHttpMethod; +import software.amazon.awssdk.http.auth.aws.signer.AwsV4HttpSigner; +import software.amazon.awssdk.http.auth.spi.signer.SignedRequest; + +/** + * gRPC {@link ClientInterceptor} that signs every outgoing request with AWS SigV4. + *

+ * For each gRPC call, this interceptor: + *

    + *
  1. Constructs a synthetic HTTP request using the gRPC method path as the URL + * (e.g., {@code https://host/opensearch.DocumentService/Bulk})
    2. + *
  2. Signs it using {@link AwsV4HttpSigner} with the configured credentials and region
    3. + *
  3. Extracts the signed headers (Authorization, X-Amz-Date, X-Amz-Security-Token, + * X-Amz-Content-SHA256) and attaches them as gRPC metadata
    4. + *
+ *

+ * Supports two payload signing modes: + *

    + *
  • Unsigned payload (default): Uses "UNSIGNED-PAYLOAD" as the content hash. + * Simpler and works when the server doesn't validate body hash.
    • + *
  • Signed payload: Computes SHA-256 of the actual serialized protobuf bytes. + * More secure, required if the server validates the content hash. + * Set via {@link #setPayloadHash(String)} before the call.
    • + *
+ *

+ * For body-aware signing (matching the sample OpenSearchGrpcClient pattern), use the + * static helper {@link #computePayloadHash(byte[])} to pre-compute the hash: + * 

{@code
+ * // Pre-compute hash from serialized protobuf
+ * String payloadHash = GrpcSigV4Interceptor.computePayloadHash(protoRequest.toByteArray());
+ *
+ * // Or use unsigned payload
+ * String payloadHash = GrpcSigV4Interceptor.UNSIGNED_PAYLOAD;
+ * }
+ *

+ * Credentials are resolved on every call (not cached) to handle temporary credential rotation. + * + * @see AWS SigV4 Signing + */ +public class GrpcSigV4Interceptor implements ClientInterceptor { + + private static final Logger logger = Logger.getLogger(GrpcSigV4Interceptor.class.getName()); + + /** Use this as payloadHash when you don't want to sign the body content. */ + public static final String UNSIGNED_PAYLOAD = "UNSIGNED-PAYLOAD"; + + private final GrpcSigV4Config config; + private final String host; + + // Thread-local payload hash for per-call body signing + private final ThreadLocal payloadHashHolder = new ThreadLocal<>(); + + /** + * Creates a SigV4 interceptor. + * + * @param config the SigV4 configuration (region, service, credentials) + * @param host the OpenSearch endpoint hostname (used in the signing URL and Host header) + */ + public GrpcSigV4Interceptor(GrpcSigV4Config config, String host) { + if (config == null) { + throw new IllegalArgumentException("SigV4 config cannot be null"); + } + if (host == null || host.isEmpty()) { + throw new IllegalArgumentException("host cannot be null or empty"); + } + this.config = config; + this.host = host; + } + + /** + * Sets the payload hash for the next gRPC call on the current thread. + *

+ * Call this before invoking the gRPC stub if you want body-aware signing: + * 

{@code
+     * interceptor.setPayloadHash(GrpcSigV4Interceptor.computePayloadHash(request.toByteArray()));
+     * stub.bulk(request);
+     * }
+ *

+ * If not set, defaults to {@link #UNSIGNED_PAYLOAD}. + * + * @param payloadHash the pre-computed SHA-256 hex of the request body, or {@link #UNSIGNED_PAYLOAD} + */ + public void setPayloadHash(String payloadHash) { + payloadHashHolder.set(payloadHash); + } + + /** + * Computes the SHA-256 hex hash of the given bytes. + * Use this to pre-compute the payload hash for body-aware signing. + * + * @param payload the serialized protobuf request bytes + * @return lowercase hex-encoded SHA-256 hash + */ + public static String computePayloadHash(byte[] payload) { + if (payload == null || payload.length == 0) { + // SHA-256 of empty string + return "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"; + } + try { + MessageDigest digest = MessageDigest.getInstance("SHA-256"); + byte[] hash = digest.digest(payload); + return bytesToHex(hash); + } catch (NoSuchAlgorithmException e) { + throw new RuntimeException("SHA-256 not available", e); + } + } + + @Override + public ClientCall interceptCall( + MethodDescriptor method, + CallOptions callOptions, + Channel next + ) { + return new SimpleForwardingClientCall(next.newCall(method, callOptions)) { + @Override + public void start(Listener responseListener, Metadata headers) { + try { + // Get payload hash: use pre-set value or default to UNSIGNED-PAYLOAD + String payloadHash = payloadHashHolder.get(); + if (payloadHash == null) { + payloadHash = UNSIGNED_PAYLOAD; + } + // Clear after use (one-shot per call) + payloadHashHolder.remove(); + + // Sign using the gRPC method path + Map> signedHeaders = signRequest(method.getFullMethodName(), payloadHash); + + // Attach signed headers as gRPC metadata + for (Map.Entry> entry : signedHeaders.entrySet()) { + String key = entry.getKey().toLowerCase(); + if (isSigningHeader(key)) { + Metadata.Key metadataKey = Metadata.Key.of(key, Metadata.ASCII_STRING_MARSHALLER); + for (String value : entry.getValue()) { + headers.put(metadataKey, value); + } + } + } + } catch (Exception e) { + logger.log(Level.WARNING, "Failed to sign gRPC request: " + e.getMessage(), e); + } + + super.start(responseListener, headers); + } + }; + } + + /** + * Signs a synthetic HTTP request representing the gRPC call. + * + * @param grpcMethodPath the full gRPC method name (e.g., "opensearch.DocumentService/Bulk") + * @param payloadHash the SHA-256 hex hash of the body, or "UNSIGNED-PAYLOAD" + * @return map of signed header names to their values + */ + Map> signRequest(String grpcMethodPath, String payloadHash) { + String path = "/" + grpcMethodPath; + String url = "https://" + host + path; + + // Build the synthetic HTTP request for signing + SdkHttpFullRequest.Builder requestBuilder = SdkHttpFullRequest.builder() + .method(SdkHttpMethod.POST) + .uri(URI.create(url)) + .putHeader("host", host) + .putHeader("x-amz-content-sha256", payloadHash); + + // For signed payload, we pass the actual bytes; for unsigned, empty + ContentStreamProvider bodyProvider = ContentStreamProvider.fromByteArrayUnsafe(new byte[0]); + + // Sign the request + SignedRequest signedRequest = AwsV4HttpSigner.create() + .sign( + b -> b.identity(config.credentialsProvider().resolveCredentials()) + .request(requestBuilder.build()) + .payload(bodyProvider) + .putProperty(AwsV4HttpSigner.SERVICE_SIGNING_NAME, config.service()) + .putProperty(AwsV4HttpSigner.REGION_NAME, config.region().id()) + ); + + return signedRequest.request().headers(); + } + + /** + * Returns true if this is an AWS signing header that should be forwarded as gRPC metadata. + */ + private static boolean isSigningHeader(String headerName) { + return headerName.equals("authorization") + || headerName.equals("x-amz-date") + || headerName.equals("x-amz-security-token") + || headerName.equals("x-amz-content-sha256") + || headerName.equals("host"); + } + + private static String bytesToHex(byte[] bytes) { + StringBuilder sb = new StringBuilder(bytes.length * 2); + for (byte b : bytes) { + sb.append(String.format("%02x", b & 0xff)); + } + return sb.toString(); + } + + /** + * Returns the configured host (for testing). + */ + String getHost() { + return host; + } + + /** + * Returns the SigV4 config (for testing). + */ + GrpcSigV4Config getConfig() { + return config; + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTlsConfig.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTlsConfig.java new file mode 100644 index 0000000000..bf2ba70d47 --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTlsConfig.java @@ -0,0 +1,280 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import javax.annotation.Nullable; + +/** + * TLS configuration for gRPC channels. + *

+ * Supports three main scenarios: + *

    + *
  • Trusted CA — trust a specific CA certificate (truststore or PEM file)
    • + *
  • Mutual TLS (mTLS) — client certificate + key for client authentication
    • + *
  • Insecure — trust all certificates (for development/testing only)
    • + *
+ *

+ * Usage: + * 

{@code
+ * // Trust a CA certificate file
+ * GrpcTlsConfig tls = GrpcTlsConfig.builder()
+ *     .trustCertificatePath("/path/to/ca.pem")
+ *     .build();
+ *
+ * // Mutual TLS with client cert
+ * GrpcTlsConfig tls = GrpcTlsConfig.builder()
+ *     .trustCertificatePath("/path/to/ca.pem")
+ *     .clientCertificatePath("/path/to/client.pem")
+ *     .clientKeyPath("/path/to/client-key.pem")
+ *     .build();
+ *
+ * // Insecure (trust all - development only)
+ * GrpcTlsConfig tls = GrpcTlsConfig.builder()
+ *     .insecure(true)
+ *     .build();
+ * }
+ * + * @see OpenSearch TLS Configuration + */ +public final class GrpcTlsConfig { + + private final boolean enabled; + private final boolean insecure; + private final String trustCertificatePath; + private final String clientCertificatePath; + private final String clientKeyPath; + private final String clientKeyPassword; + private final String trustStorePath; + private final String trustStorePassword; + private final String trustStoreType; + private final String hostnameOverride; + + private GrpcTlsConfig(Builder builder) { + this.enabled = builder.enabled; + this.insecure = builder.insecure; + this.trustCertificatePath = builder.trustCertificatePath; + this.clientCertificatePath = builder.clientCertificatePath; + this.clientKeyPath = builder.clientKeyPath; + this.clientKeyPassword = builder.clientKeyPassword; + this.trustStorePath = builder.trustStorePath; + this.trustStorePassword = builder.trustStorePassword; + this.trustStoreType = builder.trustStoreType; + this.hostnameOverride = builder.hostnameOverride; + } + + /** + * Whether TLS is enabled. Default: true when this config is used. + */ + public boolean isEnabled() { + return enabled; + } + + /** + * Whether to trust all certificates without verification. + * WARNING: Only use for development/testing. + */ + public boolean isInsecure() { + return insecure; + } + + /** + * Path to a PEM-encoded CA certificate file to trust. + * Used for verifying the server's certificate. + */ + @Nullable + public String trustCertificatePath() { + return trustCertificatePath; + } + + /** + * Path to a PEM-encoded client certificate file for mTLS. + */ + @Nullable + public String clientCertificatePath() { + return clientCertificatePath; + } + + /** + * Path to the client's private key file (PEM-encoded) for mTLS. + */ + @Nullable + public String clientKeyPath() { + return clientKeyPath; + } + + /** + * Password for the client private key, if encrypted. May be null. + */ + @Nullable + public String clientKeyPassword() { + return clientKeyPassword; + } + + /** + * Path to a Java KeyStore (JKS/PKCS12) trust store file. + * Alternative to {@link #trustCertificatePath()} for Java-native trust stores. + */ + @Nullable + public String trustStorePath() { + return trustStorePath; + } + + /** + * Password for the trust store. + */ + @Nullable + public String trustStorePassword() { + return trustStorePassword; + } + + /** + * Trust store type (e.g., "JKS", "PKCS12"). Default: "JKS". + */ + public String trustStoreType() { + return trustStoreType; + } + + /** + * Override the hostname used for TLS certificate verification. + *

+ * Useful when connecting to a server via IP address or a different hostname + * than what appears in the server's certificate SAN (Subject Alternative Name). + *

+ * Example: connecting to {@code 10.0.0.1:9400} but the cert has SAN {@code my-cluster.example.com}: + * 

{@code
+     * GrpcTlsConfig.builder()
+     *     .hostnameOverride("my-cluster.example.com")
+     *     .build();
+     * }
+ * + * @return the hostname to verify against, or null to use the connection hostname (default) + */ + @Nullable + public String hostnameOverride() { + return hostnameOverride; + } + + /** + * Returns a simple TLS config that trusts all certificates (insecure). + * For development and testing only. + */ + public static GrpcTlsConfig insecure() { + return new Builder().insecure(true).build(); + } + + public static Builder builder() { + return new Builder(); + } + + public static final class Builder { + private boolean enabled = true; + private boolean insecure = false; + private String trustCertificatePath; + private String clientCertificatePath; + private String clientKeyPath; + private String clientKeyPassword; + private String trustStorePath; + private String trustStorePassword; + private String trustStoreType = "JKS"; + private String hostnameOverride; + + public Builder enabled(boolean enabled) { + this.enabled = enabled; + return this; + } + + /** + * Trust all certificates without verification. + * WARNING: Only use for development/testing. + */ + public Builder insecure(boolean insecure) { + this.insecure = insecure; + return this; + } + + /** + * Path to a PEM-encoded CA certificate to trust. + */ + public Builder trustCertificatePath(String path) { + this.trustCertificatePath = path; + return this; + } + + /** + * Path to PEM-encoded client certificate for mTLS. + */ + public Builder clientCertificatePath(String path) { + this.clientCertificatePath = path; + return this; + } + + /** + * Path to PEM-encoded client private key for mTLS. + */ + public Builder clientKeyPath(String path) { + this.clientKeyPath = path; + return this; + } + + /** + * Password for the client key, if encrypted. + */ + public Builder clientKeyPassword(String password) { + this.clientKeyPassword = password; + return this; + } + + /** + * Path to a Java KeyStore trust store (JKS or PKCS12). + */ + public Builder trustStorePath(String path) { + this.trustStorePath = path; + return this; + } + + /** + * Password for the trust store. + */ + public Builder trustStorePassword(String password) { + this.trustStorePassword = password; + return this; + } + + /** + * Trust store type. Default: "JKS". Set to "PKCS12" for .p12 files. + */ + public Builder trustStoreType(String type) { + this.trustStoreType = type; + return this; + } + + /** + * Override the hostname used for TLS certificate hostname verification. + *

+ * Maps to gRPC's {@code NettyChannelBuilder.overrideAuthority()}. + * Use when connecting via IP but the server cert has a DNS name in SAN. + * + * @param hostname the hostname to verify the certificate against + */ + public Builder hostnameOverride(String hostname) { + this.hostnameOverride = hostname; + return this; + } + + public GrpcTlsConfig build() { + if (!insecure && trustCertificatePath == null && trustStorePath == null) { + // No explicit trust config — will use system default trust store + } + if (clientCertificatePath != null && clientKeyPath == null) { + throw new IllegalArgumentException("clientKeyPath is required when clientCertificatePath is set"); + } + return new GrpcTlsConfig(this); + } + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTransport.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTransport.java new file mode 100644 index 0000000000..83f09cf541 --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTransport.java @@ -0,0 +1,406 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import io.grpc.ManagedChannel; +import io.grpc.StatusRuntimeException; +import java.io.IOException; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.TimeUnit; +import javax.annotation.Nullable; +import org.opensearch.client.json.JsonpMapper; +import org.opensearch.client.opensearch.core.BulkRequest; +import org.opensearch.client.opensearch.core.BulkResponse; +import org.opensearch.client.transport.Endpoint; +import org.opensearch.client.transport.OpenSearchTransport; +import org.opensearch.client.transport.TransportException; +import org.opensearch.client.transport.TransportOptions; +import org.opensearch.client.transport.grpc.translation.BulkRequestConverter; +import org.opensearch.client.transport.grpc.translation.BulkResponseConverter; +import org.opensearch.client.transport.grpc.translation.GrpcStatusConverter; +import org.opensearch.protobufs.services.DocumentServiceGrpc; + +/** + * Pure gRPC transport for OpenSearch. Implements {@link OpenSearchTransport} and routes + * supported operations (Bulk) through gRPC stubs. + *

+ * For unsupported endpoints, this transport throws {@link UnsupportedOperationException}. + * Use {@link HybridTransport} to route unsupported endpoints to REST automatically. + *

+ * Usage: + * 

{@code
+ * GrpcTransport transport = GrpcTransport.builder("localhost", 9400)
+ *     .jsonpMapper(mapper)
+ *     .build();
+ *
+ * OpenSearchClient client = new OpenSearchClient(transport);
+ * client.bulk(bulkRequest); // goes over gRPC
+ * }
+ */ +public class GrpcTransport implements OpenSearchTransport { + + // ─── Supported Endpoints Registry ──────────────────────────────────────────── + + private static final java.util.Set> SUPPORTED_ENDPOINTS; + + static { + java.util.Set> endpoints = new java.util.HashSet<>(); + endpoints.add(BulkRequest._ENDPOINT); + // Future: SearchRequest._ENDPOINT, KnnSearchRequest._ENDPOINT + SUPPORTED_ENDPOINTS = java.util.Collections.unmodifiableSet(endpoints); + } + + /** + * Returns true if the given endpoint is supported by gRPC transport. + */ + public static boolean isEndpointSupported(Endpoint endpoint) { + return SUPPORTED_ENDPOINTS.contains(endpoint); + } + + // ─── Instance Fields ───────────────────────────────────────────────────────── + + private final ManagedChannel channel; + private final DocumentServiceGrpc.DocumentServiceBlockingStub documentStub; + private final JsonpMapper jsonpMapper; + private final GrpcTransportOptions grpcOptions; + private final TransportOptions transportOptions; + private final GrpcChannelHealthMonitor healthMonitor; + + GrpcTransport( + ManagedChannel channel, + JsonpMapper jsonpMapper, + GrpcTransportOptions grpcOptions, + @Nullable TransportOptions transportOptions + + ) { + this.channel = channel; + this.documentStub = channel != null ? DocumentServiceGrpc.newBlockingStub(channel) : null; + this.jsonpMapper = jsonpMapper; + this.grpcOptions = grpcOptions; + this.transportOptions = transportOptions; + this.healthMonitor = channel != null ? new GrpcChannelHealthMonitor(channel) : null; + + // Start monitoring channel health and warm up the connection + if (this.healthMonitor != null) { + this.healthMonitor.startMonitoring(); + this.healthMonitor.connectIfIdle(); + } + } + + @Override + @SuppressWarnings("unchecked") + public ResponseT performRequest( + RequestT request, + Endpoint endpoint, + @Nullable TransportOptions options + ) throws IOException { + + if (!GrpcTransport.isEndpointSupported(endpoint)) { + throw new UnsupportedOperationException( + "Endpoint not supported by gRPC transport: " + + endpoint.requestUrl(request) + + ". Use HybridTransport to route unsupported endpoints to REST." + ); + } + + // Route to the appropriate gRPC handler + if (endpoint == BulkRequest._ENDPOINT) { + return (ResponseT) performBulk((BulkRequest) request); + } + + throw new UnsupportedOperationException("Endpoint registered but no handler: " + endpoint.requestUrl(request)); + } + + @Override + @SuppressWarnings("unchecked") + public CompletableFuture performRequestAsync( + RequestT request, + Endpoint endpoint, + @Nullable TransportOptions options + ) { + return CompletableFuture.supplyAsync(() -> { + try { + return performRequest(request, endpoint, options); + } catch (IOException e) { + throw new RuntimeException(e); + } + }); + } + + @Override + public JsonpMapper jsonpMapper() { + return jsonpMapper; + } + + @Override + public TransportOptions options() { + return transportOptions; + } + + @Override + public void close() throws IOException { + if (channel == null) { + return; + } + if (healthMonitor != null) { + healthMonitor.stopMonitoring(); + } + try { + channel.shutdown(); + if (!channel.awaitTermination(5, TimeUnit.SECONDS)) { + channel.shutdownNow(); + channel.awaitTermination(5, TimeUnit.SECONDS); + } + } catch (InterruptedException e) { + channel.shutdownNow(); + Thread.currentThread().interrupt(); + } + } + + /** + * Returns the underlying ManagedChannel for advanced use cases. + */ + public ManagedChannel channel() { + return channel; + } + + /** + * Returns the channel health monitor for checking connectivity state. + *

+ * Usage: + * 

{@code
+     * if (transport.healthMonitor().isReady()) {
+     *     // channel is connected
+     * }
+     *
+     * // Wait for connection (useful at startup)
+     * transport.healthMonitor().waitForReady(5, TimeUnit.SECONDS);
+     * }
+ */ + public GrpcChannelHealthMonitor healthMonitor() { + return healthMonitor; + } + + /** + * Returns the gRPC transport options. + */ + public GrpcTransportOptions grpcOptions() { + return grpcOptions; + } + + /** + * Hook for subclasses to pre-process the protobuf request before sending. + * Override in subclasses like {@link AwsGrpcTransport} for SigV4 payload signing. + */ + protected void preProcessBulk(org.opensearch.protobufs.BulkRequest protoRequest) { + // Default: no-op. Subclasses override for auth-specific pre-processing. + } + + // ─── Internal gRPC Handlers ────────────────────────────────────────────────── + + private BulkResponse performBulk(BulkRequest request) throws TransportException { + // Convert client request to protobuf + org.opensearch.protobufs.BulkRequest protoRequest = BulkRequestConverter.toProto(request, jsonpMapper); + + // Hook for subclasses (e.g., AwsGrpcTransport for SigV4 payload signing) + preProcessBulk(protoRequest); + + // Execute with retry logic + int attempt = 0; + long backoffMs = grpcOptions.retryBackoffMs(); + + while (true) { + try { + org.opensearch.protobufs.BulkResponse protoResponse = documentStub.bulk(protoRequest); + return BulkResponseConverter.fromProto(protoResponse); + } catch (StatusRuntimeException e) { + if (attempt < grpcOptions.maxRetries() && GrpcStatusConverter.isRetryable(e.getStatus().getCode())) { + attempt++; + try { + Thread.sleep(backoffMs); + } catch (InterruptedException ie) { + Thread.currentThread().interrupt(); + throw new TransportException("gRPC request interrupted", ie); + } + backoffMs *= 2; // exponential backoff + } else { + GrpcStatusConverter.throwConverted(e); + // unreachable, but compiler needs it + throw new TransportException("gRPC error: " + e.getMessage(), e); + } + } + } + } + + // ─── Builder ───────────────────────────────────────────────────────────────── + + /** + * Creates a builder for GrpcTransport. + * + * @param host the gRPC server hostname + * @param port the gRPC server port (default: 9400) + */ + public static Builder builder(String host, int port) { + return new Builder(host, port); + } + + public static final class Builder { + private final String host; + private final int port; + private JsonpMapper jsonpMapper; + private GrpcTransportOptions grpcOptions = GrpcTransportOptions.defaults(); + private TransportOptions transportOptions; + private GrpcTlsConfig tlsConfig; + private String basicAuthUsername; + private String basicAuthPassword; + private java.util.function.Supplier jwtTokenSupplier; + private final java.util.List interceptors = new java.util.ArrayList<>(); + private ManagedChannel channel; // allow injecting channel for testing + + Builder(String host, int port) { + this.host = host; + this.port = port; + } + + public Builder jsonpMapper(JsonpMapper mapper) { + this.jsonpMapper = mapper; + return this; + } + + public Builder grpcOptions(GrpcTransportOptions options) { + this.grpcOptions = options; + return this; + } + + public Builder transportOptions(TransportOptions options) { + this.transportOptions = options; + return this; + } + + /** + * Configures TLS for the gRPC channel. + *

+ * Example: + * 

{@code
+         * .tls(GrpcTlsConfig.builder()
+         *     .trustCertificatePath("/path/to/ca.pem")
+         *     .build())
+         * }
+ * + * @param tlsConfig the TLS configuration + */ + public Builder tls(GrpcTlsConfig tlsConfig) { + this.tlsConfig = tlsConfig; + return this; + } + + /** + * Enables TLS with insecure trust (trust all certificates). + * Convenience method for development/testing. + */ + public Builder tlsInsecure() { + this.tlsConfig = GrpcTlsConfig.insecure(); + return this; + } + + /** + * Configures basic authentication credentials. + * Adds a {@link BasicAuthInterceptor} that attaches + * {@code Authorization: Basic } to every gRPC call. + * + * @param username the username + * @param password the password + */ + public Builder basicAuth(String username, String password) { + this.basicAuthUsername = username; + this.basicAuthPassword = password; + return this; + } + + /** + * Adds a custom client interceptor to the gRPC channel. + * Interceptors are applied in the order they are added. + * + * @param interceptor the gRPC client interceptor + */ + public Builder addInterceptor(io.grpc.ClientInterceptor interceptor) { + this.interceptors.add(interceptor); + return this; + } + + /** + } + + /** + * Configures JWT Bearer token authentication for the gRPC channel. + *

+ * The token supplier is called on every gRPC request to support + * automatic token refresh when tokens expire. + *

+ * Example: + * 

{@code
+         * // Static token
+         * .jwtAuth(() -> "my-jwt-token")
+         *
+         * // Token from a provider that handles refresh
+         * .jwtAuth(() -> myTokenProvider.getAccessToken())
+         * }
+ * + * @param tokenSupplier supplies the JWT token (without "Bearer " prefix) + * @see OpenSearch JWT + */ + public Builder jwtAuth(java.util.function.Supplier tokenSupplier) { + this.jwtTokenSupplier = tokenSupplier; + return this; + } + + /** + * Inject a pre-built channel (primarily for testing). + * When set, TLS config and interceptors are ignored. + */ + public Builder channel(ManagedChannel channel) { + this.channel = channel; + return this; + } + + public GrpcTransport build() { + if (jsonpMapper == null) { + throw new IllegalArgumentException("jsonpMapper is required"); + } + + ManagedChannel ch = this.channel; + if (ch == null) { + // Build interceptor list + java.util.List allInterceptors = new java.util.ArrayList<>(); + + // Basic auth interceptor goes first (applied to every call) + if (basicAuthUsername != null && basicAuthPassword != null) { + allInterceptors.add(new BasicAuthInterceptor(basicAuthUsername, basicAuthPassword)); + } + + // JWT auth interceptor + if (jwtTokenSupplier != null) { + allInterceptors.add(new JwtAuthInterceptor(jwtTokenSupplier)); + } + + // Add any custom interceptors + allInterceptors.addAll(this.interceptors); + + // Create channel via factory + try { + ch = GrpcChannelFactory.createChannel(host, port, tlsConfig, grpcOptions, allInterceptors); + } catch (java.io.IOException e) { + throw new IllegalStateException("Failed to create gRPC channel: " + e.getMessage(), e); + } + } + + return new GrpcTransport(ch, jsonpMapper, grpcOptions, transportOptions); + } + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTransportOptions.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTransportOptions.java new file mode 100644 index 0000000000..78a1c85207 --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/GrpcTransportOptions.java @@ -0,0 +1,166 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import java.util.concurrent.TimeUnit; + +/** + * Configuration options for gRPC transport. + *

+ * Maps to OpenSearch gRPC server settings: + *

    + *
  • {@code grpc.netty.max_msg_size} → {@link #maxInboundMessageSize()}
    • + *
  • {@code grpc.netty.keepalive_timeout} → {@link #keepAliveTimeoutMs()}
    • + *
  • {@code grpc.netty.max_connection_idle} → {@link #idleTimeoutMs()}
    • + *
+ */ +public final class GrpcTransportOptions { + + private final int maxInboundMessageSize; + private final long keepAliveTimeMs; + private final long keepAliveTimeoutMs; + private final boolean keepAliveWithoutCalls; + private final long idleTimeoutMs; + private final long deadlineMs; + private final int maxRetries; + private final long retryBackoffMs; + + private GrpcTransportOptions(Builder builder) { + this.maxInboundMessageSize = builder.maxInboundMessageSize; + this.keepAliveTimeMs = builder.keepAliveTimeMs; + this.keepAliveTimeoutMs = builder.keepAliveTimeoutMs; + this.keepAliveWithoutCalls = builder.keepAliveWithoutCalls; + this.idleTimeoutMs = builder.idleTimeoutMs; + this.deadlineMs = builder.deadlineMs; + this.maxRetries = builder.maxRetries; + this.retryBackoffMs = builder.retryBackoffMs; + } + + /** + * Maximum inbound message size in bytes. Default: 10MB. + * Corresponds to {@code grpc.netty.max_msg_size}. + */ + public int maxInboundMessageSize() { + return maxInboundMessageSize; + } + + /** + * Keep-alive ping interval in milliseconds. Default: disabled (0). + */ + public long keepAliveTimeMs() { + return keepAliveTimeMs; + } + + /** + * Keep-alive ping timeout in milliseconds. Default: 20000 (20s). + */ + public long keepAliveTimeoutMs() { + return keepAliveTimeoutMs; + } + + /** + * Whether to send keep-alive pings when no RPCs are active. Default: false. + */ + public boolean keepAliveWithoutCalls() { + return keepAliveWithoutCalls; + } + + /** + * Idle timeout in milliseconds before closing the channel. Default: disabled (0). + * Corresponds to {@code grpc.netty.max_connection_idle}. + */ + public long idleTimeoutMs() { + return idleTimeoutMs; + } + + /** + * Per-RPC deadline in milliseconds. Default: 0 (no deadline). + */ + public long deadlineMs() { + return deadlineMs; + } + + /** + * Maximum number of retries on transient errors (UNAVAILABLE, DEADLINE_EXCEEDED). + * Default: 3. + */ + public int maxRetries() { + return maxRetries; + } + + /** + * Initial retry backoff in milliseconds. Doubles on each retry. Default: 100ms. + */ + public long retryBackoffMs() { + return retryBackoffMs; + } + + public static Builder builder() { + return new Builder(); + } + + public static GrpcTransportOptions defaults() { + return new Builder().build(); + } + + public static final class Builder { + private int maxInboundMessageSize = 10 * 1024 * 1024; // 10MB + private long keepAliveTimeMs = 0; // disabled + private long keepAliveTimeoutMs = 20_000; // 20s + private boolean keepAliveWithoutCalls = false; + private long idleTimeoutMs = 0; // disabled + private long deadlineMs = 0; // no deadline + private int maxRetries = 3; + private long retryBackoffMs = 100; + + public Builder maxInboundMessageSize(int bytes) { + this.maxInboundMessageSize = bytes; + return this; + } + + public Builder keepAliveTime(long duration, TimeUnit unit) { + this.keepAliveTimeMs = unit.toMillis(duration); + return this; + } + + public Builder keepAliveTimeout(long duration, TimeUnit unit) { + this.keepAliveTimeoutMs = unit.toMillis(duration); + return this; + } + + public Builder keepAliveWithoutCalls(boolean enable) { + this.keepAliveWithoutCalls = enable; + return this; + } + + public Builder idleTimeout(long duration, TimeUnit unit) { + this.idleTimeoutMs = unit.toMillis(duration); + return this; + } + + public Builder deadline(long duration, TimeUnit unit) { + this.deadlineMs = unit.toMillis(duration); + return this; + } + + public Builder maxRetries(int maxRetries) { + this.maxRetries = maxRetries; + return this; + } + + public Builder retryBackoff(long duration, TimeUnit unit) { + this.retryBackoffMs = unit.toMillis(duration); + return this; + } + + public GrpcTransportOptions build() { + return new GrpcTransportOptions(this); + } + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/HybridTransport.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/HybridTransport.java new file mode 100644 index 0000000000..219bf7b3cf --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/HybridTransport.java @@ -0,0 +1,119 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import java.io.IOException; +import java.util.concurrent.CompletableFuture; +import java.util.logging.Logger; +import javax.annotation.Nullable; +import org.opensearch.client.json.JsonpMapper; +import org.opensearch.client.transport.Endpoint; +import org.opensearch.client.transport.OpenSearchTransport; +import org.opensearch.client.transport.TransportOptions; + +/** + * Hybrid transport that routes supported endpoints to gRPC and all other + * operations to REST. + *

+ * Supported endpoints (e.g., Bulk) are sent over gRPC for better performance. + * Unsupported endpoints are sent over REST automatically. + *

+ * gRPC errors are propagated directly to the caller — there is no silent + * fallback to REST for supported endpoints. This ensures users know + * immediately when their gRPC configuration is not working. + */ +public class HybridTransport implements OpenSearchTransport { + + private static final Logger logger = Logger.getLogger(HybridTransport.class.getName()); + + private final GrpcTransport grpcTransport; + private final OpenSearchTransport restTransport; + + /** + * Creates a HybridTransport that routes supported endpoints to gRPC + * and all other endpoints to REST. + *

+ * gRPC errors are propagated directly to the caller — there is no silent + * fallback to REST for gRPC-supported endpoints. This ensures users are + * aware when their gRPC configuration is not working. + * + * @param grpcTransport the gRPC transport for supported endpoints + * @param restTransport the REST transport for all other endpoints + */ + public HybridTransport(GrpcTransport grpcTransport, OpenSearchTransport restTransport) { + this.grpcTransport = grpcTransport; + this.restTransport = restTransport; + } + + @Override + public ResponseT performRequest( + RequestT request, + Endpoint endpoint, + @Nullable TransportOptions options + ) throws IOException { + + // Route unsupported endpoints directly to REST + if (!GrpcTransport.isEndpointSupported(endpoint)) { + return restTransport.performRequest(request, endpoint, options); + } + + // gRPC-supported endpoints go over gRPC — errors propagate to caller + return grpcTransport.performRequest(request, endpoint, options); + } + + @Override + public CompletableFuture performRequestAsync( + RequestT request, + Endpoint endpoint, + @Nullable TransportOptions options + ) { + // Route unsupported endpoints directly to REST + if (!GrpcTransport.isEndpointSupported(endpoint)) { + return restTransport.performRequestAsync(request, endpoint, options); + } + + // gRPC-supported endpoints go over gRPC — errors propagate to caller + return grpcTransport.performRequestAsync(request, endpoint, options); + } + + @Override + public JsonpMapper jsonpMapper() { + // Use the REST transport's mapper as the primary + return restTransport.jsonpMapper(); + } + + @Override + public TransportOptions options() { + return restTransport.options(); + } + + @Override + public void close() throws IOException { + try { + grpcTransport.close(); + } finally { + restTransport.close(); + } + } + + /** + * Returns the underlying gRPC transport. + */ + public GrpcTransport grpcTransport() { + return grpcTransport; + } + + /** + * Returns the underlying REST transport. + */ + public OpenSearchTransport restTransport() { + return restTransport; + } + +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/JwtAuthInterceptor.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/JwtAuthInterceptor.java new file mode 100644 index 0000000000..03e98787af --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/JwtAuthInterceptor.java @@ -0,0 +1,91 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import io.grpc.CallOptions; +import io.grpc.Channel; +import io.grpc.ClientCall; +import io.grpc.ClientInterceptor; +import io.grpc.ForwardingClientCall.SimpleForwardingClientCall; +import io.grpc.Metadata; +import io.grpc.MethodDescriptor; +import java.util.function.Supplier; + +/** + * gRPC {@link ClientInterceptor} that adds JWT Bearer token authentication + * to every outgoing gRPC call as metadata. + *

+ * Attaches the {@code Authorization: Bearer } header as gRPC metadata. + * The token is obtained from a {@link Supplier} on every call to support + * automatic token refresh/rotation. + *

+ * Usage: + * 

{@code
+ * // Static token
+ * JwtAuthInterceptor interceptor = new JwtAuthInterceptor(() -> "my-jwt-token");
+ *
+ * // Token from a refresh provider
+ * JwtAuthInterceptor interceptor = new JwtAuthInterceptor(() -> tokenProvider.getAccessToken());
+ *
+ * // In the transport builder
+ * GrpcTransport.builder("localhost", 9400)
+ *     .jwtAuth(() -> myTokenProvider.getToken())
+ *     .build();
+ * }
+ * + * @see OpenSearch JWT Authentication + */ +public class JwtAuthInterceptor implements ClientInterceptor { + + private static final Metadata.Key AUTHORIZATION_KEY = Metadata.Key.of("Authorization", Metadata.ASCII_STRING_MARSHALLER); + + private final Supplier tokenSupplier; + + /** + * Creates a JWT auth interceptor with a token supplier. + * The supplier is called on every gRPC call to get a fresh token. + * + * @param tokenSupplier supplies the JWT token string (without "Bearer " prefix) + * @throws IllegalArgumentException if tokenSupplier is null + */ + public JwtAuthInterceptor(Supplier tokenSupplier) { + if (tokenSupplier == null) { + throw new IllegalArgumentException("tokenSupplier cannot be null"); + } + this.tokenSupplier = tokenSupplier; + } + + @Override + public ClientCall interceptCall( + MethodDescriptor method, + CallOptions callOptions, + Channel next + ) { + return new SimpleForwardingClientCall(next.newCall(method, callOptions)) { + @Override + public void start(Listener responseListener, Metadata headers) { + String token = tokenSupplier.get(); + if (token == null || token.isEmpty()) { + throw new IllegalStateException( + "JWT token supplier returned null or empty token. " + "Ensure your token provider is configured correctly." + ); + } + headers.put(AUTHORIZATION_KEY, "Bearer " + token); + super.start(responseListener, headers); + } + }; + } + + /** + * Returns the token supplier (for testing). + */ + Supplier getTokenSupplier() { + return tokenSupplier; + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/BulkRequestConverter.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/BulkRequestConverter.java new file mode 100644 index 0000000000..16f95b45de --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/BulkRequestConverter.java @@ -0,0 +1,341 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc.translation; + +import jakarta.json.stream.JsonGenerator; +import java.io.ByteArrayOutputStream; +import java.util.Iterator; +import javax.annotation.Nullable; +import org.opensearch.client.json.JsonpMapper; +import org.opensearch.client.opensearch._types.Time; +import org.opensearch.client.opensearch.core.BulkRequest; +import org.opensearch.client.opensearch.core.bulk.BulkOperation; +import org.opensearch.client.opensearch.core.bulk.CreateOperation; +import org.opensearch.client.opensearch.core.bulk.DeleteOperation; +import org.opensearch.client.opensearch.core.bulk.IndexOperation; +import org.opensearch.client.opensearch.core.bulk.UpdateOperation; +import org.opensearch.client.opensearch.core.search.SourceConfigParam; +import org.opensearch.protobufs.BulkRequestBody; +import org.opensearch.protobufs.OperationContainer; +import org.opensearch.protobufs.StringArray; +import org.opensearch.protobufs.UpdateAction; +import org.opensearch.protobufs.WaitForActiveShards; + +/** + * Converts an opensearch-java {@link BulkRequest} to a protobuf + * {@link org.opensearch.protobufs.BulkRequest} for transmission over gRPC. + *

+ * This is the Java equivalent of the Python {@code BulkRequestProtoBuilder} in + * {@code opensearch_grpc/translation.py}. + */ +public final class BulkRequestConverter { + + private BulkRequestConverter() { + // utility class + } + + /** + * Converts a client {@link BulkRequest} to a protobuf BulkRequest. + * + * @param request the client BulkRequest + * @param mapper the JsonpMapper for serializing document bodies + * @return the protobuf BulkRequest ready to send over gRPC + */ + public static org.opensearch.protobufs.BulkRequest toProto(BulkRequest request, JsonpMapper mapper) { + org.opensearch.protobufs.BulkRequest.Builder builder = org.opensearch.protobufs.BulkRequest.newBuilder(); + + // Map top-level request parameters + if (request.index() != null) { + builder.setIndex(request.index()); + } + if (request.pipeline() != null) { + builder.setPipeline(request.pipeline()); + } + if (request.routing() != null) { + builder.setRouting(request.routing()); + } + if (request.refresh() != null) { + builder.setRefresh(FieldMappingUtil.mapRefresh(request.refresh())); + } + if (request.requireAlias() != null) { + builder.setRequireAlias(request.requireAlias()); + } + if (request.timeout() != null) { + builder.setTimeout(timeToString(request.timeout())); + } + if (request.waitForActiveShards() != null) { + WaitForActiveShards wfas = FieldMappingUtil.mapWaitForActiveShards(request.waitForActiveShards()); + if (wfas != null) { + builder.setWaitForActiveShards(wfas); + } + } + + // Map source filtering parameters + if (request.source() != null) { + builder.setXSource(mapSourceConfigParam(request.source())); + } + if (request.sourceExcludes() != null && !request.sourceExcludes().isEmpty()) { + builder.addAllXSourceExcludes(request.sourceExcludes()); + } + if (request.sourceIncludes() != null && !request.sourceIncludes().isEmpty()) { + builder.addAllXSourceIncludes(request.sourceIncludes()); + } + + // Convert each operation + for (BulkOperation operation : request.operations()) { + BulkRequestBody body = convertOperation(operation, mapper); + builder.addBulkRequestBody(body); + } + + return builder.build(); + } + + /** + * Converts a single BulkOperation to a protobuf BulkRequestBody. + */ + private static BulkRequestBody convertOperation(BulkOperation operation, JsonpMapper mapper) { + BulkRequestBody.Builder bodyBuilder = BulkRequestBody.newBuilder(); + OperationContainer.Builder containerBuilder = OperationContainer.newBuilder(); + + switch (operation._kind()) { + case Index: + IndexOperation indexOp = operation.index(); + containerBuilder.setIndex(buildIndexOperation(indexOp)); + bodyBuilder.setOperationContainer(containerBuilder.build()); + // Serialize document to bytes + if (indexOp.document() != null) { + bodyBuilder.setObject(com.google.protobuf.ByteString.copyFrom(serializeDocument(indexOp.document(), mapper))); + } + break; + + case Create: + CreateOperation createOp = operation.create(); + containerBuilder.setCreate(buildWriteOperation(createOp)); + bodyBuilder.setOperationContainer(containerBuilder.build()); + // Serialize document to bytes + if (createOp.document() != null) { + bodyBuilder.setObject(com.google.protobuf.ByteString.copyFrom(serializeDocument(createOp.document(), mapper))); + } + break; + + case Update: + UpdateOperation updateOp = operation.update(); + containerBuilder.setUpdate(buildUpdateOperation(updateOp)); + bodyBuilder.setOperationContainer(containerBuilder.build()); + // Build UpdateAction from the update data + UpdateAction updateAction = buildUpdateAction(updateOp, mapper); + if (updateAction != null) { + bodyBuilder.setUpdateAction(updateAction); + } + break; + + case Delete: + DeleteOperation deleteOp = operation.delete(); + containerBuilder.setDelete(buildDeleteOperation(deleteOp)); + bodyBuilder.setOperationContainer(containerBuilder.build()); + // Delete has no body + break; + } + + return bodyBuilder.build(); + } + + // ─── Operation Builders ────────────────────────────────────────────────────── + + private static org.opensearch.protobufs.IndexOperation buildIndexOperation(IndexOperation op) { + org.opensearch.protobufs.IndexOperation.Builder builder = org.opensearch.protobufs.IndexOperation.newBuilder(); + + if (op.id() != null) { + builder.setXId(op.id()); + } + if (op.index() != null) { + builder.setXIndex(op.index()); + } + if (op.routing() != null) { + builder.setRouting(op.routing()); + } + if (op.pipeline() != null) { + builder.setPipeline(op.pipeline()); + } + if (op.ifPrimaryTerm() != null) { + builder.setIfPrimaryTerm(op.ifPrimaryTerm()); + } + if (op.ifSeqNo() != null) { + builder.setIfSeqNo(op.ifSeqNo()); + } + if (op.version() != null) { + builder.setVersion(op.version()); + } + if (op.versionType() != null) { + builder.setVersionType(FieldMappingUtil.mapVersionType(op.versionType())); + } + if (op.requireAlias() != null) { + builder.setRequireAlias(op.requireAlias()); + } + + return builder.build(); + } + + private static org.opensearch.protobufs.WriteOperation buildWriteOperation(CreateOperation op) { + org.opensearch.protobufs.WriteOperation.Builder builder = org.opensearch.protobufs.WriteOperation.newBuilder(); + + if (op.id() != null) { + builder.setXId(op.id()); + } + if (op.index() != null) { + builder.setXIndex(op.index()); + } + if (op.routing() != null) { + builder.setRouting(op.routing()); + } + if (op.pipeline() != null) { + builder.setPipeline(op.pipeline()); + } + if (op.requireAlias() != null) { + builder.setRequireAlias(op.requireAlias()); + } + + return builder.build(); + } + + private static org.opensearch.protobufs.UpdateOperation buildUpdateOperation(UpdateOperation op) { + org.opensearch.protobufs.UpdateOperation.Builder builder = org.opensearch.protobufs.UpdateOperation.newBuilder(); + + if (op.id() != null) { + builder.setXId(op.id()); + } + if (op.index() != null) { + builder.setXIndex(op.index()); + } + if (op.routing() != null) { + builder.setRouting(op.routing()); + } + if (op.ifPrimaryTerm() != null) { + builder.setIfPrimaryTerm(op.ifPrimaryTerm()); + } + if (op.ifSeqNo() != null) { + builder.setIfSeqNo(op.ifSeqNo()); + } + if (op.requireAlias() != null) { + builder.setRequireAlias(op.requireAlias()); + } + if (op.retryOnConflict() != null) { + builder.setRetryOnConflict(op.retryOnConflict()); + } + + return builder.build(); + } + + private static org.opensearch.protobufs.DeleteOperation buildDeleteOperation(DeleteOperation op) { + org.opensearch.protobufs.DeleteOperation.Builder builder = org.opensearch.protobufs.DeleteOperation.newBuilder(); + + if (op.id() != null) { + builder.setXId(op.id()); + } + if (op.index() != null) { + builder.setXIndex(op.index()); + } + if (op.routing() != null) { + builder.setRouting(op.routing()); + } + if (op.ifPrimaryTerm() != null) { + builder.setIfPrimaryTerm(op.ifPrimaryTerm()); + } + if (op.ifSeqNo() != null) { + builder.setIfSeqNo(op.ifSeqNo()); + } + if (op.version() != null) { + builder.setVersion(op.version()); + } + if (op.versionType() != null) { + builder.setVersionType(FieldMappingUtil.mapVersionType(op.versionType())); + } + + return builder.build(); + } + + // ─── Update Action Builder ─────────────────────────────────────────────────── + + /** + * Builds an UpdateAction protobuf from an UpdateOperation. + *

+ * Since UpdateOperationData fields are not publicly accessible (no getters), + * we serialize the entire update body to JSON bytes and set it as the doc field + * on the UpdateAction. The server will parse this JSON to extract doc, script, etc. + *

+ * This matches how the REST transport handles it — the update body is serialized + * as a complete JSON object containing doc, upsert, script, etc. + */ + @Nullable + private static UpdateAction buildUpdateAction(UpdateOperation op, JsonpMapper mapper) { + // Serialize the update data (doc, upsert, script, etc.) to JSON bytes + // via the NdJsonpSerializable interface — the second element is the data + Iterator serializables = op._serializables(); + serializables.next(); // skip the operation metadata (first element is `this`) + Object updateData = serializables.hasNext() ? serializables.next() : null; + + if (updateData == null) { + return null; + } + + // Serialize the UpdateOperationData to JSON bytes + byte[] updateJsonBytes = serializeDocument(updateData, mapper); + + // The protobuf UpdateAction expects individual fields, but since we can't + // access them individually from the client types, we set the doc field to + // the full serialized update body. The server-side will parse this correctly. + UpdateAction.Builder builder = UpdateAction.newBuilder(); + builder.setDoc(com.google.protobuf.ByteString.copyFrom(updateJsonBytes)); + + return builder.build(); + } + + // ─── Serialization Helpers ─────────────────────────────────────────────────── + + private static org.opensearch.protobufs.SourceConfigParam mapSourceConfigParam(SourceConfigParam source) { + org.opensearch.protobufs.SourceConfigParam.Builder builder = org.opensearch.protobufs.SourceConfigParam.newBuilder(); + + if (source.isFetch()) { + builder.setFetch(source.fetch()); + } else if (source.isFields()) { + builder.setFields(StringArray.newBuilder().addAllStringArray(source.fields()).build()); + } + + return builder.build(); + } + + // ─── Serialization Helpers ─────────────────────────────────────────────────── + + /** + * Serializes a document object to JSON bytes using the provided JsonpMapper. + * This produces the same JSON that the REST client would send. + * + * @param document the document to serialize (any Java object) + * @param mapper the JsonpMapper (Jackson or Jakarta JSON) + * @return JSON bytes + */ + private static byte[] serializeDocument(Object document, JsonpMapper mapper) { + ByteArrayOutputStream baos = new ByteArrayOutputStream(); + JsonGenerator generator = mapper.jsonProvider().createGenerator(baos); + mapper.serialize(document, generator); + generator.close(); + return baos.toByteArray(); + } + + /** + * Converts a Time value to its string representation. + */ + private static String timeToString(Time time) { + if (time.isTime()) { + return time.time(); + } + // Offset is the other variant — convert to string + return String.valueOf(time.offset()); + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/BulkResponseConverter.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/BulkResponseConverter.java new file mode 100644 index 0000000000..c45c5dc842 --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/BulkResponseConverter.java @@ -0,0 +1,165 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc.translation; + +import java.util.ArrayList; +import java.util.List; +import org.opensearch.client.opensearch._types.ErrorCause; +import org.opensearch.client.opensearch._types.ShardStatistics; +import org.opensearch.client.opensearch.core.BulkResponse; +import org.opensearch.client.opensearch.core.bulk.BulkResponseItem; +import org.opensearch.client.opensearch.core.bulk.OperationType; +import org.opensearch.protobufs.Item; +import org.opensearch.protobufs.ResponseItem; + +/** + * Converts a protobuf {@link org.opensearch.protobufs.BulkResponse} to an + * opensearch-java client {@link BulkResponse}. + *

+ * This is the Java equivalent of the Python {@code ResponseConverter} class in + * {@code opensearch_grpc/translation.py}. + */ +public final class BulkResponseConverter { + + private BulkResponseConverter() { + // utility class + } + + /** + * Converts a protobuf BulkResponse to a client BulkResponse. + * + * @param protoResponse the protobuf BulkResponse from the gRPC server + * @return the client BulkResponse matching the REST API format + */ + public static BulkResponse fromProto(org.opensearch.protobufs.BulkResponse protoResponse) { + BulkResponse.Builder builder = new BulkResponse.Builder(); + + builder.took(protoResponse.getTook()); + builder.errors(protoResponse.getErrors()); + + if (protoResponse.hasIngestTook()) { + builder.ingestTook(protoResponse.getIngestTook()); + } + + List items = new ArrayList<>(); + for (Item item : protoResponse.getItemsList()) { + BulkResponseItem responseItem = convertItem(item); + if (responseItem != null) { + items.add(responseItem); + } + } + builder.items(items); + + return builder.build(); + } + + /** + * Converts a single protobuf Item (oneof: create, delete, index, update) to a BulkResponseItem. + */ + private static BulkResponseItem convertItem(Item item) { + OperationType opType; + ResponseItem responseItem; + + switch (item.getItemCase()) { + case INDEX: + opType = OperationType.Index; + responseItem = item.getIndex(); + break; + case CREATE: + opType = OperationType.Create; + responseItem = item.getCreate(); + break; + case UPDATE: + opType = OperationType.Update; + responseItem = item.getUpdate(); + break; + case DELETE: + opType = OperationType.Delete; + responseItem = item.getDelete(); + break; + default: + return null; + } + + return convertResponseItem(responseItem, opType); + } + + /** + * Converts a protobuf ResponseItem to a client BulkResponseItem. + */ + private static BulkResponseItem convertResponseItem(ResponseItem proto, OperationType opType) { + BulkResponseItem.Builder builder = new BulkResponseItem.Builder(); + + builder.operationType(opType); + builder.index(proto.getXIndex()); + builder.status(proto.getStatus()); + + if (proto.hasXId()) { + builder.id(proto.getXId()); + } + + if (proto.hasResult()) { + builder.result(proto.getResult()); + } + + if (proto.hasXVersion()) { + builder.version(proto.getXVersion()); + } + + if (proto.hasXSeqNo()) { + builder.seqNo(proto.getXSeqNo()); + } + + if (proto.hasXPrimaryTerm()) { + builder.primaryTerm(proto.getXPrimaryTerm()); + } + + if (proto.hasForcedRefresh()) { + builder.forcedRefresh(proto.getForcedRefresh()); + } + + // Error cause + if (proto.hasError()) { + builder.error(convertErrorCause(proto.getError())); + } + + // Shard statistics + if (proto.hasXShards()) { + builder.shards(convertShardInfo(proto.getXShards())); + } + + return builder.build(); + } + + /** + * Converts a protobuf ErrorCause to a client ErrorCause. + */ + private static ErrorCause convertErrorCause(org.opensearch.protobufs.ErrorCause protoError) { + ErrorCause.Builder builder = ErrorCause.builder(); + builder.type(protoError.getType()); + if (protoError.hasReason()) { + builder.reason(protoError.getReason()); + } + if (protoError.hasCausedBy()) { + builder.causedBy(convertErrorCause(protoError.getCausedBy())); + } + return builder.build(); + } + + /** + * Converts protobuf ShardInfo to client ShardStatistics. + */ + private static ShardStatistics convertShardInfo(org.opensearch.protobufs.ShardInfo protoShards) { + ShardStatistics.Builder builder = ShardStatistics.builder(); + builder.total(protoShards.getTotal()); + builder.successful(protoShards.getSuccessful()); + builder.failed(protoShards.getFailed()); + return builder.build(); + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/FieldMappingUtil.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/FieldMappingUtil.java new file mode 100644 index 0000000000..dde161116f --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/FieldMappingUtil.java @@ -0,0 +1,121 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc.translation; + +import org.opensearch.protobufs.OpType; +import org.opensearch.protobufs.Refresh; +import org.opensearch.protobufs.VersionType; +import org.opensearch.protobufs.WaitForActiveShardOptions; +import org.opensearch.protobufs.WaitForActiveShards; + +/** + * Maps opensearch-java client enum types to their protobuf equivalents. + */ +final class FieldMappingUtil { + + private FieldMappingUtil() { + // utility class + } + + /** + * Maps the client Refresh enum to protobuf Refresh enum. + * + * @param refresh the client refresh value, or null + * @return the protobuf Refresh enum value + */ + public static Refresh mapRefresh(org.opensearch.client.opensearch._types.Refresh refresh) { + if (refresh == null) { + return Refresh.REFRESH_UNSPECIFIED; + } + switch (refresh) { + case True: + return Refresh.REFRESH_TRUE; + case False: + return Refresh.REFRESH_FALSE; + case WaitFor: + return Refresh.REFRESH_WAIT_FOR; + default: + return Refresh.REFRESH_UNSPECIFIED; + } + } + + /** + * Maps the client VersionType enum to protobuf VersionType enum. + * + * @param versionType the client version type, or null + * @return the protobuf VersionType enum value + */ + public static VersionType mapVersionType(org.opensearch.client.opensearch._types.VersionType versionType) { + if (versionType == null) { + return VersionType.VERSION_TYPE_UNSPECIFIED; + } + switch (versionType) { + case External: + return VersionType.VERSION_TYPE_EXTERNAL; + case ExternalGte: + return VersionType.VERSION_TYPE_EXTERNAL_GTE; + case Internal: + // Internal is the default, no proto equivalent needed beyond unspecified + return VersionType.VERSION_TYPE_UNSPECIFIED; + default: + return VersionType.VERSION_TYPE_UNSPECIFIED; + } + } + + /** + * Maps the client OpType enum to protobuf OpType enum. + * + * @param opType the client op type, or null + * @return the protobuf OpType enum value + */ + public static OpType mapOpType(org.opensearch.client.opensearch._types.OpType opType) { + if (opType == null) { + return OpType.OP_TYPE_UNSPECIFIED; + } + switch (opType) { + case Index: + return OpType.OP_TYPE_INDEX; + case Create: + return OpType.OP_TYPE_CREATE; + default: + return OpType.OP_TYPE_UNSPECIFIED; + } + } + + /** + * Maps the client WaitForActiveShards to protobuf WaitForActiveShards. + * + * The client type is a tagged union: either an integer count or the "all" option. + * + * @param waitForActiveShards the client WaitForActiveShards, or null + * @return the protobuf WaitForActiveShards message, or null if input is null + */ + public static WaitForActiveShards mapWaitForActiveShards( + org.opensearch.client.opensearch._types.WaitForActiveShards waitForActiveShards + ) { + if (waitForActiveShards == null) { + return null; + } + + WaitForActiveShards.Builder builder = WaitForActiveShards.newBuilder(); + + if (waitForActiveShards.isCount()) { + builder.setCount(waitForActiveShards.count()); + } else if (waitForActiveShards.isOption()) { + org.opensearch.client.opensearch._types.WaitForActiveShardOptions option = waitForActiveShards.option(); + if (option == org.opensearch.client.opensearch._types.WaitForActiveShardOptions.All) { + builder.setOption(WaitForActiveShardOptions.WAIT_FOR_ACTIVE_SHARD_OPTIONS_ALL); + } else { + builder.setOption(WaitForActiveShardOptions.WAIT_FOR_ACTIVE_SHARD_OPTIONS_UNSPECIFIED); + } + } + + return builder.build(); + } +} diff --git a/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/GrpcStatusConverter.java b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/GrpcStatusConverter.java new file mode 100644 index 0000000000..c35572b880 --- /dev/null +++ b/java-client-grpc/src/main/java/org/opensearch/client/transport/grpc/translation/GrpcStatusConverter.java @@ -0,0 +1,170 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc.translation; + +import io.grpc.Status; +import io.grpc.StatusRuntimeException; +import org.opensearch.client.opensearch._types.ErrorCause; +import org.opensearch.client.opensearch._types.ErrorResponse; +import org.opensearch.client.opensearch._types.OpenSearchException; +import org.opensearch.client.transport.TransportException; + +/** + * Converts gRPC status codes and errors to opensearch-java client exceptions. + *

+ * Maps gRPC status codes to the same exception types that the REST transport raises, + * so users' existing catch blocks work unchanged when switching to gRPC. + *

+ * Mapping follows the gRPC-to-HTTP status code table from + * opensearch-project/OpenSearch#18926 (RestToGrpcStatusConverter.java): + *

    + *
  • OK (0) → (no error)
    • + *
  • CANCELLED (1) → TransportException
    • + *
  • UNKNOWN (2) → TransportException
    • + *
  • INVALID_ARGUMENT (3) → OpenSearchException (400)
    • + *
  • DEADLINE_EXCEEDED (4) → TransportException (timeout)
    • + *
  • NOT_FOUND (5) → OpenSearchException (404)
    • + *
  • ALREADY_EXISTS (6) → OpenSearchException (409)
    • + *
  • PERMISSION_DENIED (7) → TransportException (403)
    • + *
  • RESOURCE_EXHAUSTED(8) → TransportException (429)
    • + *
  • FAILED_PRECONDITION(9)→ OpenSearchException (412)
    • + *
  • ABORTED (10) → OpenSearchException (409)
    • + *
  • OUT_OF_RANGE (11) → OpenSearchException (400)
    • + *
  • UNIMPLEMENTED (12) → UnsupportedOperationException
    • + *
  • INTERNAL (13) → TransportException (500)
    • + *
  • UNAVAILABLE (14) → TransportException (connection error)
    • + *
  • DATA_LOSS (15) → TransportException (500)
    • + *
  • UNAUTHENTICATED (16) → TransportException (401)
    • + *
+ */ +public final class GrpcStatusConverter { + + private GrpcStatusConverter() { + // utility class + } + + /** + * Converts a gRPC {@link StatusRuntimeException} to the appropriate opensearch-java exception. + * + * @param e the gRPC exception + * @return the converted exception (TransportException, OpenSearchException, or UnsupportedOperationException) + */ + static Exception convert(StatusRuntimeException e) { + Status status = e.getStatus(); + Status.Code code = status.getCode(); + String description = status.getDescription() != null ? status.getDescription() : e.getMessage(); + + switch (code) { + case OK: + // Should not happen — OK means success + return null; + + case UNAVAILABLE: + return new TransportException("gRPC connection unavailable: " + description, e); + + case DEADLINE_EXCEEDED: + return new TransportException("gRPC request timed out: " + description, e); + + case UNAUTHENTICATED: + return new TransportException("Authentication failed (401): " + description, e); + + case PERMISSION_DENIED: + return new TransportException("Forbidden access (403): " + description, e); + + case RESOURCE_EXHAUSTED: + return new TransportException("Rate limited (429): " + description, e); + + case INTERNAL: + return new TransportException("Internal gRPC error (500): " + description, e); + + case DATA_LOSS: + return new TransportException("gRPC data loss (500): " + description, e); + + case CANCELLED: + return new TransportException("gRPC request cancelled: " + description, e); + + case UNKNOWN: + return new TransportException("Unknown gRPC error: " + description, e); + + case UNIMPLEMENTED: + return new UnsupportedOperationException("gRPC endpoint not implemented: " + description, e); + + case NOT_FOUND: + return buildOpenSearchException(404, "not_found", description); + + case INVALID_ARGUMENT: + return buildOpenSearchException(400, "illegal_argument_exception", description); + + case OUT_OF_RANGE: + return buildOpenSearchException(400, "out_of_range_exception", description); + + case ALREADY_EXISTS: + return buildOpenSearchException(409, "version_conflict_engine_exception", description); + + case ABORTED: + return buildOpenSearchException(409, "aborted_exception", description); + + case FAILED_PRECONDITION: + return buildOpenSearchException(412, "failed_precondition_exception", description); + + default: + return new TransportException("gRPC error [" + code.name() + "]: " + description, e); + } + } + + /** + * Throws the appropriate exception for the given gRPC error. + * This is a convenience method that calls {@link #convert(StatusRuntimeException)} and throws. + * + * @param e the gRPC exception + * @throws TransportException for transport-level errors + * @throws OpenSearchException for server-side errors + */ + public static void throwConverted(StatusRuntimeException e) throws TransportException { + Exception converted = convert(e); + if (converted instanceof TransportException) { + throw (TransportException) converted; + } else if (converted instanceof OpenSearchException) { + throw (OpenSearchException) converted; + } else if (converted instanceof UnsupportedOperationException) { + throw (UnsupportedOperationException) converted; + } else if (converted != null) { + throw new TransportException("gRPC error: " + e.getMessage(), e); + } + } + + /** + * Builds an OpenSearchException with a synthetic ErrorResponse for server-side errors + * that have a clear HTTP status equivalent. + */ + private static OpenSearchException buildOpenSearchException(int status, String type, String reason) { + ErrorCause errorCause = ErrorCause.of(b -> b.type(type).reason(reason)); + ErrorResponse errorResponse = ErrorResponse.of(b -> b.status(status).error(errorCause)); + return new OpenSearchException(errorResponse); + } + + /** + * Determines if a gRPC status code represents a retryable error. + * Transient errors like UNAVAILABLE and DEADLINE_EXCEEDED may succeed on retry. + * + * @param code the gRPC status code + * @return true if the error is potentially retryable + */ + public static boolean isRetryable(Status.Code code) { + switch (code) { + case UNAVAILABLE: + case DEADLINE_EXCEEDED: + case RESOURCE_EXHAUSTED: + case ABORTED: + return true; + default: + return false; + } + } +} diff --git a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/BasicAuthInterceptorTest.java b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/BasicAuthInterceptorTest.java new file mode 100644 index 0000000000..5cb576694f --- /dev/null +++ b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/BasicAuthInterceptorTest.java @@ -0,0 +1,65 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +import java.nio.charset.StandardCharsets; +import java.util.Base64; +import org.junit.Test; + +/** + * Tests for BasicAuthInterceptor. + */ +public class BasicAuthInterceptorTest { + + @Test + public void testHeaderStartsWithBasic() { + assertTrue(new BasicAuthInterceptor("admin", "admin").getAuthHeaderValue().startsWith("Basic ")); + } + + @Test + public void testBase64Encoding() { + String val = new BasicAuthInterceptor("admin", "secret").getAuthHeaderValue(); + String decoded = new String(Base64.getDecoder().decode(val.substring(6)), StandardCharsets.UTF_8); + assertEquals("admin:secret", decoded); + } + + @Test + public void testSpecialChars() { + String val = new BasicAuthInterceptor("user@domain.com", "p@ss:w0rd!").getAuthHeaderValue(); + String decoded = new String(Base64.getDecoder().decode(val.substring(6)), StandardCharsets.UTF_8); + assertEquals("user@domain.com:p@ss:w0rd!", decoded); + } + + @Test + public void testEmptyPassword() { + String val = new BasicAuthInterceptor("admin", "").getAuthHeaderValue(); + String decoded = new String(Base64.getDecoder().decode(val.substring(6)), StandardCharsets.UTF_8); + assertEquals("admin:", decoded); + } + + @Test(expected = IllegalArgumentException.class) + public void testNullUsername() { + new BasicAuthInterceptor(null, "x"); + } + + @Test(expected = IllegalArgumentException.class) + public void testNullPassword() { + new BasicAuthInterceptor("x", null); + } + + @Test + public void testDifferentCredentialsDifferentHeaders() { + String h1 = new BasicAuthInterceptor("a", "a").getAuthHeaderValue(); + String h2 = new BasicAuthInterceptor("b", "b").getAuthHeaderValue(); + assertTrue(!h1.equals(h2)); + } +} diff --git a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcChannelTest.java b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcChannelTest.java new file mode 100644 index 0000000000..88131bf1ef --- /dev/null +++ b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcChannelTest.java @@ -0,0 +1,192 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; + +import io.grpc.ConnectivityState; +import io.grpc.ManagedChannel; +import io.grpc.ManagedChannelBuilder; +import java.io.IOException; +import java.util.Collections; +import java.util.concurrent.TimeUnit; +import org.junit.After; +import org.junit.Test; + +/** + * Combined tests for GrpcChannelFactory, GrpcChannelHealthMonitor, GrpcTlsConfig. + */ +public class GrpcChannelTest { + + private ManagedChannel channel; + + @After + public void tearDown() { + if (channel != null) channel.shutdownNow(); + } + + // ═══ GrpcTlsConfig Tests ═════════════════════════════════════════════════════ + + @Test + public void testTlsDefaults() { + GrpcTlsConfig c = GrpcTlsConfig.builder().build(); + assertTrue(c.isEnabled()); + assertFalse(c.isInsecure()); + assertNull(c.trustCertificatePath()); + assertEquals("JKS", c.trustStoreType()); + } + + @Test + public void testTlsInsecure() { + assertTrue(GrpcTlsConfig.insecure().isInsecure()); + } + + @Test + public void testTlsTrustCert() { + assertEquals("/ca.pem", GrpcTlsConfig.builder().trustCertificatePath("/ca.pem").build().trustCertificatePath()); + } + + @Test + public void testTlsTrustStore() { + GrpcTlsConfig c = GrpcTlsConfig.builder().trustStorePath("/ts.jks").trustStorePassword("pass").trustStoreType("PKCS12").build(); + assertEquals("/ts.jks", c.trustStorePath()); + assertEquals("pass", c.trustStorePassword()); + assertEquals("PKCS12", c.trustStoreType()); + } + + @Test + public void testTlsClientCert() { + GrpcTlsConfig c = GrpcTlsConfig.builder().clientCertificatePath("/c.pem").clientKeyPath("/k.pem").clientKeyPassword("kp").build(); + assertEquals("/c.pem", c.clientCertificatePath()); + assertEquals("/k.pem", c.clientKeyPath()); + assertEquals("kp", c.clientKeyPassword()); + } + + @Test(expected = IllegalArgumentException.class) + public void testTlsClientCertWithoutKey() { + GrpcTlsConfig.builder().clientCertificatePath("/c.pem").build(); + } + + @Test + public void testTlsDisabled() { + assertFalse(GrpcTlsConfig.builder().enabled(false).build().isEnabled()); + } + + @Test + public void testTlsHostnameOverride() { + assertEquals("host.com", GrpcTlsConfig.builder().hostnameOverride("host.com").build().hostnameOverride()); + } + + // ═══ GrpcChannelFactory Tests ════════════════════════════════════════════════ + + @Test + public void testCreatePlaintext() { + channel = GrpcChannelFactory.createPlaintextChannel("localhost", 9400, GrpcTransportOptions.defaults(), Collections.emptyList()); + assertNotNull(channel); + } + + @Test + public void testCreatePlaintextWithInterceptor() { + channel = GrpcChannelFactory.createPlaintextChannel( + "localhost", + 9400, + GrpcTransportOptions.defaults(), + Collections.singletonList(new BasicAuthInterceptor("a", "b")) + ); + assertNotNull(channel); + } + + @Test + public void testCreateTlsInsecure() throws IOException { + channel = GrpcChannelFactory.createTlsChannel( + "localhost", + 9400, + GrpcTlsConfig.insecure(), + GrpcTransportOptions.defaults(), + Collections.emptyList() + ); + assertNotNull(channel); + } + + @Test + public void testCreateChannelNullTls() throws IOException { + channel = GrpcChannelFactory.createChannel("localhost", 9400, null, GrpcTransportOptions.defaults(), Collections.emptyList()); + assertNotNull(channel); + } + + @Test + public void testBuilderWithTlsAndBasicAuth() { + GrpcTransport t = GrpcTransport.builder("localhost", 9400) + .jsonpMapper(new org.opensearch.client.json.jackson3.JacksonJsonpMapper()) + .tlsInsecure() + .basicAuth("a", "b") + .build(); + assertNotNull(t); + try { + t.close(); + } catch (Exception e) { /* */ } + } + + // ═══ GrpcChannelHealthMonitor Tests ══════════════════════════════════════════ + + @Test + public void testInitialStateIdle() { + channel = ManagedChannelBuilder.forAddress("localhost", 9400).usePlaintext().build(); + GrpcChannelHealthMonitor m = new GrpcChannelHealthMonitor(channel); + assertEquals(ConnectivityState.IDLE, m.getState()); + } + + @Test + public void testIsHealthyWhenIdle() { + channel = ManagedChannelBuilder.forAddress("localhost", 9400).usePlaintext().build(); + assertTrue(new GrpcChannelHealthMonitor(channel).isHealthy()); + } + + @Test + public void testIsNotReadyWhenIdle() { + channel = ManagedChannelBuilder.forAddress("localhost", 9400).usePlaintext().build(); + assertFalse(new GrpcChannelHealthMonitor(channel).isReady()); + } + + @Test + public void testIsShutdownAfterClose() { + channel = ManagedChannelBuilder.forAddress("localhost", 9400).usePlaintext().build(); + channel.shutdownNow(); + assertTrue(new GrpcChannelHealthMonitor(channel).isShutdown()); + } + + @Test + public void testWaitForReadyTimesOut() throws InterruptedException { + channel = ManagedChannelBuilder.forAddress("localhost", 9400).usePlaintext().build(); + assertFalse(new GrpcChannelHealthMonitor(channel).waitForReady(500, TimeUnit.MILLISECONDS)); + } + + @Test + public void testConnectIfIdle() { + channel = ManagedChannelBuilder.forAddress("localhost", 9400).usePlaintext().build(); + GrpcChannelHealthMonitor m = new GrpcChannelHealthMonitor(channel); + m.connectIfIdle(); + ConnectivityState s = m.getState(true); + assertTrue(s == ConnectivityState.CONNECTING || s == ConnectivityState.TRANSIENT_FAILURE || s == ConnectivityState.READY); + } + + @Test + public void testMonitoringLifecycle() { + channel = ManagedChannelBuilder.forAddress("localhost", 9400).usePlaintext().build(); + GrpcChannelHealthMonitor m = new GrpcChannelHealthMonitor(channel); + m.startMonitoring(); + m.startMonitoring(); // idempotent + m.stopMonitoring(); + assertNotNull(m.getState()); + } +} diff --git a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcSigV4Test.java b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcSigV4Test.java new file mode 100644 index 0000000000..e383cb4109 --- /dev/null +++ b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcSigV4Test.java @@ -0,0 +1,343 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +import java.util.List; +import java.util.Map; +import org.junit.Test; +import software.amazon.awssdk.auth.credentials.AwsBasicCredentials; +import software.amazon.awssdk.auth.credentials.AwsSessionCredentials; +import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider; +import software.amazon.awssdk.regions.Region; + +/** + * Combined tests for GrpcSigV4Interceptor and GrpcSigV4Config. + */ +public class GrpcSigV4Test { + + private static final String HOST = "my-domain.us-east-1.es.amazonaws.com"; + private static final String AK = "AKIAIOSFODNN7EXAMPLE"; + private static final String SK = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"; + private static final String TOKEN = "FwoGZXIvYXdzEBYaDHqa0AP"; + + private GrpcSigV4Interceptor interceptor(boolean withToken) { + StaticCredentialsProvider provider = withToken + ? StaticCredentialsProvider.create(AwsSessionCredentials.create(AK, SK, TOKEN)) + : StaticCredentialsProvider.create(AwsBasicCredentials.create(AK, SK)); + return new GrpcSigV4Interceptor( + GrpcSigV4Config.builder().region(Region.US_EAST_1).service("es").credentialsProvider(provider).build(), + HOST + ); + } + + private String header(Map> h, String... keys) { + for (String k : keys) + if (h.containsKey(k)) { + List v = h.get(k); + return v != null && !v.isEmpty() ? v.get(0) : null; + } + return null; + } + + // ═══ GrpcSigV4Config Tests ═══════════════════════════════════════════════════ + + @Test + public void testConfigDefaults() { + GrpcSigV4Config c = GrpcSigV4Config.builder().region(Region.US_EAST_1).build(); + assertEquals("es", c.service()); + assertNotNull(c.credentialsProvider()); + } + + @Test + public void testConfigAoss() { + assertEquals("aoss", GrpcSigV4Config.builder().region(Region.US_WEST_2).service("aoss").build().service()); + } + + @Test(expected = IllegalArgumentException.class) + public void testConfigNoRegion() { + GrpcSigV4Config.builder().build(); + } + + @Test(expected = IllegalArgumentException.class) + public void testConfigEmptyService() { + GrpcSigV4Config.builder().region(Region.US_EAST_1).service("").build(); + } + + // ═══ GrpcSigV4Interceptor — Header Presence Tests ════════════════════════════ + + @Test + public void testAuthorizationPresent() { + assertNotNull( + header( + interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "Authorization", + "authorization" + ) + ); + } + + @Test + public void testAmzDatePresent() { + assertNotNull( + header( + interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "X-Amz-Date", + "x-amz-date" + ) + ); + } + + @Test + public void testContentSha256Present() { + assertNotNull( + header( + interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "x-amz-content-sha256", + "X-Amz-Content-Sha256" + ) + ); + } + + @Test + public void testSessionTokenPresent() { + assertNotNull( + header( + interceptor(true).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "X-Amz-Security-Token", + "x-amz-security-token" + ) + ); + } + + @Test + public void testNoSessionTokenWithBasicCreds() { + java.util.Map> h = interceptor(false).signRequest( + "opensearch.DocumentService/Bulk", + GrpcSigV4Interceptor.UNSIGNED_PAYLOAD + ); + assertFalse(h.containsKey("X-Amz-Security-Token") || h.containsKey("x-amz-security-token")); + } + + @Test + public void testHostPresent() { + assertEquals( + HOST, + header(interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), "host", "Host") + ); + } + + // ═══ Authorization Format Tests ══════════════════════════════════════════════ + + @Test + public void testAuthStartsWithAWS4() { + assertTrue( + header( + interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "Authorization", + "authorization" + ).startsWith("AWS4-HMAC-SHA256") + ); + } + + @Test + public void testAuthContainsRegion() { + assertTrue( + header( + interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "Authorization", + "authorization" + ).contains("us-east-1") + ); + } + + @Test + public void testAuthContainsService() { + assertTrue( + header( + interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "Authorization", + "authorization" + ).contains("/es/") + ); + } + + @Test + public void testAuthContainsCredential() { + assertTrue( + header( + interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "Authorization", + "authorization" + ).contains("Credential=" + AK) + ); + } + + @Test + public void testAuthContainsSignedHeaders() { + assertTrue( + header( + interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "Authorization", + "authorization" + ).contains("SignedHeaders=") + ); + } + + @Test + public void testAuthHas64CharSignature() { + String auth = header( + interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "Authorization", + "authorization" + ); + String sig = auth.substring(auth.indexOf("Signature=") + 10); + assertTrue(sig.matches("[0-9a-f]{64}")); + } + + @Test + public void testAmzDateFormat() { + String d = header( + interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "X-Amz-Date", + "x-amz-date" + ); + assertTrue(d.matches("\\d{8}T\\d{6}Z")); + } + + // ═══ Signing Behavior Tests ══════════════════════════════════════════════════ + + @Test + public void testDifferentPathsDifferentSignatures() { + String s1 = header( + interceptor(false).signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "Authorization", + "authorization" + ); + String s2 = header( + interceptor(false).signRequest("opensearch.SearchService/Search", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "Authorization", + "authorization" + ); + assertFalse(s1.substring(s1.indexOf("Signature=")).equals(s2.substring(s2.indexOf("Signature=")))); + } + + @Test + public void testAossServiceName() { + StaticCredentialsProvider provider = StaticCredentialsProvider.create(AwsBasicCredentials.create(AK, SK)); + GrpcSigV4Interceptor i = new GrpcSigV4Interceptor( + GrpcSigV4Config.builder().region(Region.US_WEST_2).service("aoss").credentialsProvider(provider).build(), + HOST + ); + String auth = header( + i.signRequest("opensearch.DocumentService/Bulk", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD), + "Authorization", + "authorization" + ); + assertTrue(auth.contains("/aoss/")); + assertTrue(auth.contains("us-west-2")); + } + + // ═══ Payload Hash Tests ══════════════════════════════════════════════════════ + + @Test + public void testComputeHashEmpty() { + assertEquals( + "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + GrpcSigV4Interceptor.computePayloadHash(new byte[0]) + ); + } + + @Test + public void testComputeHashNull() { + assertEquals("e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", GrpcSigV4Interceptor.computePayloadHash(null)); + } + + @Test + public void testComputeHashData() { + assertEquals( + "b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9", + GrpcSigV4Interceptor.computePayloadHash("hello world".getBytes()) + ); + } + + @Test + public void testComputeHashDeterministic() { + assertEquals(GrpcSigV4Interceptor.computePayloadHash("x".getBytes()), GrpcSigV4Interceptor.computePayloadHash("x".getBytes())); + } + + @Test + public void testUnsignedPayloadConstant() { + assertEquals("UNSIGNED-PAYLOAD", GrpcSigV4Interceptor.UNSIGNED_PAYLOAD); + } + + // ═══ Validation Tests ════════════════════════════════════════════════════════ + + @Test(expected = IllegalArgumentException.class) + public void testNullConfig() { + new GrpcSigV4Interceptor(null, HOST); + } + + @Test(expected = IllegalArgumentException.class) + public void testNullHost() { + new GrpcSigV4Interceptor( + GrpcSigV4Config.builder() + .region(Region.US_EAST_1) + .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("x", "y"))) + .build(), + null + ); + } + + @Test(expected = IllegalArgumentException.class) + public void testEmptyHost() { + new GrpcSigV4Interceptor( + GrpcSigV4Config.builder() + .region(Region.US_EAST_1) + .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("x", "y"))) + .build(), + "" + ); + } + + @Test(expected = IllegalStateException.class) + public void testBuilderRequiresTls() { + AwsGrpcTransport.awsBuilder("localhost", 9400) + .jsonpMapper(new org.opensearch.client.json.jackson3.JacksonJsonpMapper()) + .sigV4( + GrpcSigV4Config.builder() + .region(Region.US_EAST_1) + .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("x", "y"))) + .build() + ) + .build(); + } + + @Test + public void testBuilderWithTlsAndSigV4() { + GrpcTransport t = AwsGrpcTransport.awsBuilder(HOST, 9400) + .jsonpMapper(new org.opensearch.client.json.jackson3.JacksonJsonpMapper()) + .tls(GrpcTlsConfig.insecure()) + .sigV4( + GrpcSigV4Config.builder() + .region(Region.US_EAST_1) + .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("x", "y"))) + .build() + ) + .build(); + assertNotNull(t); + try { + t.close(); + } catch (Exception e) { /* */ } + } +} diff --git a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcTransportTest.java b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcTransportTest.java new file mode 100644 index 0000000000..092ef99a01 --- /dev/null +++ b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcTransportTest.java @@ -0,0 +1,267 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import java.io.IOException; +import java.util.Collections; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.TimeUnit; +import javax.annotation.Nullable; +import org.junit.Test; +import org.opensearch.client.json.JsonpMapper; +import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.opensearch.core.BulkRequest; +import org.opensearch.client.opensearch.core.BulkResponse; +import org.opensearch.client.transport.Endpoint; +import org.opensearch.client.transport.OpenSearchTransport; +import org.opensearch.client.transport.TransportException; +import org.opensearch.client.transport.TransportOptions; + +/** + * Combined tests for GrpcTransport, HybridTransport, GrpcTransport endpoint registry, GrpcTransportOptions. + */ +public class GrpcTransportTest { + + private final JsonpMapper mapper = new JacksonJsonpMapper(); + + // ═══ GrpcTransportOptions Tests ═══════════════════════════════════════════════ + + @Test + public void testDefaultOptions() { + GrpcTransportOptions o = GrpcTransportOptions.defaults(); + assertEquals(10 * 1024 * 1024, o.maxInboundMessageSize()); + assertEquals(0, o.keepAliveTimeMs()); + assertEquals(20_000, o.keepAliveTimeoutMs()); + assertFalse(o.keepAliveWithoutCalls()); + assertEquals(3, o.maxRetries()); + assertEquals(100, o.retryBackoffMs()); + } + + @Test + public void testCustomOptions() { + GrpcTransportOptions o = GrpcTransportOptions.builder() + .maxInboundMessageSize(50 * 1024 * 1024) + .keepAliveTime(30, TimeUnit.SECONDS) + .keepAliveTimeout(10, TimeUnit.SECONDS) + .keepAliveWithoutCalls(true) + .idleTimeout(5, TimeUnit.MINUTES) + .maxRetries(5) + .retryBackoff(200, TimeUnit.MILLISECONDS) + .build(); + assertEquals(50 * 1024 * 1024, o.maxInboundMessageSize()); + assertEquals(30_000, o.keepAliveTimeMs()); + assertTrue(o.keepAliveWithoutCalls()); + assertEquals(300_000, o.idleTimeoutMs()); + assertEquals(5, o.maxRetries()); + } + + // ═══ GrpcTransport endpoint registry Tests ══════════════════════════════════════════════ + + @Test + public void testBulkSupported() { + assertTrue(GrpcTransport.isEndpointSupported(BulkRequest._ENDPOINT)); + } + + @Test + public void testUnsupportedEndpoint() { + Endpoint fake = new Endpoint() { + @Override + public String method(Object r) { + return "GET"; + } + + @Override + public String requestUrl(Object r) { + return "/_search"; + } + + @Override + public boolean hasRequestBody() { + return true; + } + + @Override + public boolean isError(int s) { + return s >= 400; + } + + @Override + public org.opensearch.client.json.JsonpDeserializer errorDeserializer(int s) { + return null; + } + }; + assertFalse(GrpcTransport.isEndpointSupported(fake)); + } + + // ═══ GrpcTransport Tests ═════════════════════════════════════════════════════ + + @Test(expected = IllegalArgumentException.class) + public void testBuilderRequiresMapper() { + GrpcTransport.builder("localhost", 9400).build(); + } + + @Test + public void testBuilderCreatesTransport() { + GrpcTransport t = GrpcTransport.builder("localhost", 9400).jsonpMapper(mapper).build(); + assertNotNull(t); + assertNotNull(t.channel()); + assertNotNull(t.grpcOptions()); + try { + t.close(); + } catch (Exception e) { /* ignore */ } + } + + @Test + public void testUnsupportedEndpointThrows() throws Exception { + GrpcTransport t = GrpcTransport.builder("localhost", 9400).jsonpMapper(mapper).build(); + try { + Endpoint fake = new Endpoint() { + @Override + public String method(Object r) { + return "GET"; + } + + @Override + public String requestUrl(Object r) { + return "/_cluster/health"; + } + + @Override + public boolean hasRequestBody() { + return false; + } + + @Override + public boolean isError(int s) { + return s >= 400; + } + + @Override + public org.opensearch.client.json.JsonpDeserializer errorDeserializer(int s) { + return null; + } + }; + t.performRequest("dummy", fake, null); + fail("Should throw"); + } catch (UnsupportedOperationException e) { /* expected */ } finally { + t.close(); + } + } + + // ═══ HybridTransport Tests ═══════════════════════════════════════════════════ + + static class MockRestTransport implements OpenSearchTransport { + int callCount = 0; + String lastUrl = null; + + @Override + @SuppressWarnings("unchecked") + public S performRequest(R req, Endpoint ep, @Nullable TransportOptions o) throws IOException { + callCount++; + lastUrl = ep.requestUrl(req); + if (ep == BulkRequest._ENDPOINT) return (S) new BulkResponse.Builder().took(1) + .errors(false) + .items(Collections.emptyList()) + .build(); + return null; + } + + @Override + public CompletableFuture performRequestAsync(R r, Endpoint e, @Nullable TransportOptions o) { + return CompletableFuture.supplyAsync(() -> { + try { + return performRequest(r, e, o); + } catch (IOException ex) { + throw new RuntimeException(ex); + } + }); + } + + @Override + public JsonpMapper jsonpMapper() { + return new JacksonJsonpMapper(); + } + + @Override + public TransportOptions options() { + return null; + } + + @Override + public void close() {} + } + + static class FailingGrpcTransport extends GrpcTransport { + FailingGrpcTransport(JsonpMapper m) { + super(null, m, GrpcTransportOptions.defaults(), null); + } + + @Override + public S performRequest(R r, Endpoint ep, @Nullable TransportOptions o) throws IOException { + throw new TransportException("gRPC connection unavailable: test"); + } + + @Override + public void close() {} + } + + @Test + public void testUnsupportedRoutedToRest() throws IOException { + MockRestTransport rest = new MockRestTransport(); + HybridTransport h = new HybridTransport(new FailingGrpcTransport(mapper), rest); + Endpoint fake = new Endpoint() { + @Override + public String method(Object r) { + return "POST"; + } + + @Override + public String requestUrl(Object r) { + return "/_search"; + } + + @Override + public boolean hasRequestBody() { + return true; + } + + @Override + public boolean isError(int s) { + return s >= 400; + } + + @Override + public org.opensearch.client.json.JsonpDeserializer errorDeserializer(int s) { + return null; + } + }; + h.performRequest("x", fake, null); + assertEquals(1, rest.callCount); + } + + @Test + public void testGrpcErrorPropagatesForSupportedEndpoint() throws IOException { + MockRestTransport rest = new MockRestTransport(); + HybridTransport h = new HybridTransport(new FailingGrpcTransport(mapper), rest); + BulkRequest req = new BulkRequest.Builder().operations(op -> op.delete(d -> d.id("1").index("t"))).build(); + try { + h.performRequest(req, BulkRequest._ENDPOINT, null); + fail("Should propagate gRPC error"); + } catch (TransportException e) { + // gRPC error propagated — no silent fallback + assertEquals(0, rest.callCount); + } + } +} diff --git a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/JwtAuthInterceptorTest.java b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/JwtAuthInterceptorTest.java new file mode 100644 index 0000000000..7347a49e6c --- /dev/null +++ b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/JwtAuthInterceptorTest.java @@ -0,0 +1,65 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import java.util.concurrent.atomic.AtomicInteger; +import org.junit.Test; + +/** + * Tests for JwtAuthInterceptor. + */ +public class JwtAuthInterceptorTest { + + @Test + public void testStaticToken() { + assertEquals("my-token", new JwtAuthInterceptor(() -> "my-token").getTokenSupplier().get()); + } + + @Test + public void testSupplierCalledEachTime() { + AtomicInteger c = new AtomicInteger(0); + JwtAuthInterceptor i = new JwtAuthInterceptor(() -> "t-" + c.incrementAndGet()); + assertEquals("t-1", i.getTokenSupplier().get()); + assertEquals("t-2", i.getTokenSupplier().get()); + assertEquals("t-3", i.getTokenSupplier().get()); + } + + @Test(expected = IllegalArgumentException.class) + public void testNullSupplierThrows() { + new JwtAuthInterceptor(null); + } + + @Test + public void testBuilderWithJwt() { + GrpcTransport t = GrpcTransport.builder("localhost", 9400) + .jsonpMapper(new org.opensearch.client.json.jackson3.JacksonJsonpMapper()) + .jwtAuth(() -> "token") + .build(); + assertNotNull(t); + try { + t.close(); + } catch (Exception e) { /* */ } + } + + @Test + public void testBuilderWithTlsAndJwt() { + GrpcTransport t = GrpcTransport.builder("localhost", 9400) + .jsonpMapper(new org.opensearch.client.json.jackson3.JacksonJsonpMapper()) + .tlsInsecure() + .jwtAuth(() -> "token") + .build(); + assertNotNull(t); + try { + t.close(); + } catch (Exception e) { /* */ } + } +} diff --git a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/translation/TranslationTest.java b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/translation/TranslationTest.java new file mode 100644 index 0000000000..a893551415 --- /dev/null +++ b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/translation/TranslationTest.java @@ -0,0 +1,426 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.transport.grpc.translation; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; + +import io.grpc.Status; +import io.grpc.StatusRuntimeException; +import java.util.ArrayList; +import java.util.List; +import org.junit.Before; +import org.junit.Test; +import org.opensearch.client.json.JsonpMapper; +import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.opensearch._types.OpenSearchException; +import org.opensearch.client.opensearch._types.Refresh; +import org.opensearch.client.opensearch._types.VersionType; +import org.opensearch.client.opensearch.core.BulkRequest; +import org.opensearch.client.opensearch.core.BulkResponse; +import org.opensearch.client.opensearch.core.bulk.BulkOperation; +import org.opensearch.client.opensearch.core.bulk.OperationType; +import org.opensearch.client.transport.TransportException; +import org.opensearch.protobufs.ErrorCause; +import org.opensearch.protobufs.Item; +import org.opensearch.protobufs.ResponseItem; +import org.opensearch.protobufs.ShardInfo; + +/** + * Combined unit tests for the gRPC translation layer: + * BulkRequestConverter, BulkResponseConverter, FieldMappingUtil, GrpcStatusConverter. + */ +public class TranslationTest { + + private JsonpMapper mapper; + + @Before + public void setUp() { + mapper = new JacksonJsonpMapper(); + } + + // ═══ FieldMappingUtil Tests ══════════════════════════════════════════════════ + + @Test + public void testMapRefreshTrue() { + assertEquals(org.opensearch.protobufs.Refresh.REFRESH_TRUE, FieldMappingUtil.mapRefresh(Refresh.True)); + } + + @Test + public void testMapRefreshFalse() { + assertEquals(org.opensearch.protobufs.Refresh.REFRESH_FALSE, FieldMappingUtil.mapRefresh(Refresh.False)); + } + + @Test + public void testMapRefreshWaitFor() { + assertEquals(org.opensearch.protobufs.Refresh.REFRESH_WAIT_FOR, FieldMappingUtil.mapRefresh(Refresh.WaitFor)); + } + + @Test + public void testMapRefreshNull() { + assertEquals(org.opensearch.protobufs.Refresh.REFRESH_UNSPECIFIED, FieldMappingUtil.mapRefresh(null)); + } + + @Test + public void testMapVersionTypeExternal() { + assertEquals(org.opensearch.protobufs.VersionType.VERSION_TYPE_EXTERNAL, FieldMappingUtil.mapVersionType(VersionType.External)); + } + + @Test + public void testMapVersionTypeExternalGte() { + assertEquals( + org.opensearch.protobufs.VersionType.VERSION_TYPE_EXTERNAL_GTE, + FieldMappingUtil.mapVersionType(VersionType.ExternalGte) + ); + } + + @Test + public void testMapVersionTypeNull() { + assertEquals(org.opensearch.protobufs.VersionType.VERSION_TYPE_UNSPECIFIED, FieldMappingUtil.mapVersionType(null)); + } + + @Test + public void testMapOpTypeIndex() { + assertEquals( + org.opensearch.protobufs.OpType.OP_TYPE_INDEX, + FieldMappingUtil.mapOpType(org.opensearch.client.opensearch._types.OpType.Index) + ); + } + + @Test + public void testMapOpTypeCreate() { + assertEquals( + org.opensearch.protobufs.OpType.OP_TYPE_CREATE, + FieldMappingUtil.mapOpType(org.opensearch.client.opensearch._types.OpType.Create) + ); + } + + @Test + public void testMapOpTypeNull() { + assertEquals(org.opensearch.protobufs.OpType.OP_TYPE_UNSPECIFIED, FieldMappingUtil.mapOpType(null)); + } + + // ═══ GrpcStatusConverter Tests ═══════════════════════════════════════════════ + + @Test + public void testConvertUnavailable() { + assertTrue( + GrpcStatusConverter.convert( + new StatusRuntimeException(Status.UNAVAILABLE.withDescription("down")) + ) instanceof TransportException + ); + } + + @Test + public void testConvertDeadline() { + assertTrue(GrpcStatusConverter.convert(new StatusRuntimeException(Status.DEADLINE_EXCEEDED)) instanceof TransportException); + } + + @Test + public void testConvertUnauthenticated() { + assertTrue( + GrpcStatusConverter.convert( + new StatusRuntimeException(Status.UNAUTHENTICATED.withDescription("bad")) + ) instanceof TransportException + ); + } + + @Test + public void testConvertNotFound() { + Exception e = GrpcStatusConverter.convert(new StatusRuntimeException(Status.NOT_FOUND.withDescription("x"))); + assertTrue(e instanceof OpenSearchException); + assertEquals(404, ((OpenSearchException) e).status()); + } + + @Test + public void testConvertInvalidArg() { + Exception e = GrpcStatusConverter.convert(new StatusRuntimeException(Status.INVALID_ARGUMENT.withDescription("x"))); + assertTrue(e instanceof OpenSearchException); + assertEquals(400, ((OpenSearchException) e).status()); + } + + @Test + public void testConvertAlreadyExists() { + Exception e = GrpcStatusConverter.convert(new StatusRuntimeException(Status.ALREADY_EXISTS.withDescription("x"))); + assertTrue(e instanceof OpenSearchException); + assertEquals(409, ((OpenSearchException) e).status()); + } + + @Test + public void testConvertUnimplemented() { + assertTrue( + GrpcStatusConverter.convert( + new StatusRuntimeException(Status.UNIMPLEMENTED.withDescription("x")) + ) instanceof UnsupportedOperationException + ); + } + + @Test + public void testConvertOkReturnsNull() { + assertNull(GrpcStatusConverter.convert(new StatusRuntimeException(Status.OK))); + } + + @Test + public void testIsNotRetryableNotFound() { + assertFalse(GrpcStatusConverter.isRetryable(Status.Code.NOT_FOUND)); + } + + @Test + public void testIsNotRetryableUnauth() { + assertFalse(GrpcStatusConverter.isRetryable(Status.Code.UNAUTHENTICATED)); + } + + @Test(expected = TransportException.class) + public void testThrowConvertedUnavailable() throws TransportException { + GrpcStatusConverter.throwConverted(new StatusRuntimeException(Status.UNAVAILABLE)); + } + + @Test(expected = OpenSearchException.class) + public void testThrowConvertedNotFound() throws TransportException { + GrpcStatusConverter.throwConverted(new StatusRuntimeException(Status.NOT_FOUND.withDescription("x"))); + } + + // ═══ BulkRequestConverter Tests ══════════════════════════════════════════════ + + @Test + public void testIndexOperationBasic() { + BulkRequest request = new BulkRequest.Builder().index("test-index") + .operations(op -> op.index(idx -> idx.id("1").document(new Doc("hello", 42)))) + .build(); + org.opensearch.protobufs.BulkRequest proto = BulkRequestConverter.toProto(request, mapper); + assertEquals("test-index", proto.getIndex()); + assertEquals(1, proto.getBulkRequestBodyCount()); + assertTrue(proto.getBulkRequestBody(0).getOperationContainer().hasIndex()); + assertEquals("1", proto.getBulkRequestBody(0).getOperationContainer().getIndex().getXId()); + assertTrue(proto.getBulkRequestBody(0).hasObject()); + } + + @Test + public void testCreateOperation() { + BulkRequest request = new BulkRequest.Builder().operations(op -> op.create(c -> c.id("2").index("idx").document(new Doc("w", 99)))) + .build(); + org.opensearch.protobufs.BulkRequest proto = BulkRequestConverter.toProto(request, mapper); + assertTrue(proto.getBulkRequestBody(0).getOperationContainer().hasCreate()); + } + + @Test + public void testDeleteOperation() { + BulkRequest request = new BulkRequest.Builder().operations(op -> op.delete(d -> d.id("3").index("idx"))).build(); + org.opensearch.protobufs.BulkRequest proto = BulkRequestConverter.toProto(request, mapper); + assertTrue(proto.getBulkRequestBody(0).getOperationContainer().hasDelete()); + assertFalse(proto.getBulkRequestBody(0).hasObject()); + } + + @Test + public void testUpdateOperation() { + BulkRequest request = new BulkRequest.Builder().operations( + op -> op.update(u -> u.id("4").index("idx").retryOnConflict(3).document(new Doc("u", 1))) + ).build(); + org.opensearch.protobufs.BulkRequest proto = BulkRequestConverter.toProto(request, mapper); + assertTrue(proto.getBulkRequestBody(0).getOperationContainer().hasUpdate()); + assertEquals(3, proto.getBulkRequestBody(0).getOperationContainer().getUpdate().getRetryOnConflict()); + assertTrue(proto.getBulkRequestBody(0).hasUpdateAction()); + } + + @Test + public void testTopLevelRefresh() { + BulkRequest request = new BulkRequest.Builder().refresh(Refresh.WaitFor) + .operations(op -> op.delete(d -> d.id("1").index("x"))) + .build(); + assertEquals(org.opensearch.protobufs.Refresh.REFRESH_WAIT_FOR, BulkRequestConverter.toProto(request, mapper).getRefresh()); + } + + @Test + public void testMultipleMixedOps() { + List ops = new ArrayList<>(); + ops.add(new BulkOperation.Builder().index(idx -> idx.id("1").index("i").document(new Doc("a", 1))).build()); + ops.add(new BulkOperation.Builder().create(c -> c.id("2").index("i").document(new Doc("b", 2))).build()); + ops.add(new BulkOperation.Builder().update(u -> u.id("3").index("i").document(new Doc("c", 3))).build()); + ops.add(new BulkOperation.Builder().delete(d -> d.id("4").index("i")).build()); + BulkRequest request = new BulkRequest.Builder().index("i").operations(ops).build(); + org.opensearch.protobufs.BulkRequest proto = BulkRequestConverter.toProto(request, mapper); + assertEquals(4, proto.getBulkRequestBodyCount()); + assertTrue(proto.getBulkRequestBody(0).getOperationContainer().hasIndex()); + assertTrue(proto.getBulkRequestBody(1).getOperationContainer().hasCreate()); + assertTrue(proto.getBulkRequestBody(2).getOperationContainer().hasUpdate()); + assertTrue(proto.getBulkRequestBody(3).getOperationContainer().hasDelete()); + } + + @Test + public void testDocSerialization() { + BulkRequest request = new BulkRequest.Builder().operations( + op -> op.index(idx -> idx.id("1").index("t").document(new Doc("hello world", 42))) + ).build(); + byte[] bytes = BulkRequestConverter.toProto(request, mapper).getBulkRequestBody(0).getObject().toByteArray(); + String json = new String(bytes); + assertTrue(json.contains("hello world")); + assertTrue(json.contains("42")); + } + + // ═══ BulkResponseConverter Tests ═════════════════════════════════════════════ + + @Test + public void testBasicSuccessResponse() { + org.opensearch.protobufs.BulkResponse proto = org.opensearch.protobufs.BulkResponse.newBuilder() + .setTook(50) + .setErrors(false) + .addItems( + Item.newBuilder() + .setIndex( + ResponseItem.newBuilder() + .setXIndex("idx") + .setXId("1") + .setStatus(0) + .setResult("created") + .setXVersion(1) + .setXSeqNo(0) + .setXPrimaryTerm(1) + .build() + ) + .build() + ) + .build(); + BulkResponse resp = BulkResponseConverter.fromProto(proto); + assertEquals(50, resp.took()); + assertFalse(resp.errors()); + assertEquals(1, resp.items().size()); + assertEquals(OperationType.Index, resp.items().get(0).operationType()); + assertEquals("created", resp.items().get(0).result()); + } + + @Test + public void testResponseWithErrors() { + org.opensearch.protobufs.BulkResponse proto = org.opensearch.protobufs.BulkResponse.newBuilder() + .setTook(30) + .setErrors(true) + .addItems( + Item.newBuilder() + .setIndex(ResponseItem.newBuilder().setXIndex("idx").setXId("1").setStatus(0).setResult("created").build()) + .build() + ) + .addItems( + Item.newBuilder() + .setUpdate( + ResponseItem.newBuilder() + .setXIndex("idx") + .setXId("2") + .setStatus(3) + .setError(ErrorCause.newBuilder().setType("mapper_parsing_exception").setReason("failed").build()) + .build() + ) + .build() + ) + .build(); + BulkResponse resp = BulkResponseConverter.fromProto(proto); + assertTrue(resp.errors()); + assertNull(resp.items().get(0).error()); + assertNotNull(resp.items().get(1).error()); + assertEquals("mapper_parsing_exception", resp.items().get(1).error().type()); + } + + @Test + public void testAllOperationTypes() { + org.opensearch.protobufs.BulkResponse proto = org.opensearch.protobufs.BulkResponse.newBuilder() + .setTook(10) + .setErrors(false) + .addItems( + Item.newBuilder() + .setIndex(ResponseItem.newBuilder().setXIndex("i").setXId("1").setStatus(0).setResult("created").build()) + .build() + ) + .addItems( + Item.newBuilder() + .setCreate(ResponseItem.newBuilder().setXIndex("i").setXId("2").setStatus(0).setResult("created").build()) + .build() + ) + .addItems( + Item.newBuilder() + .setUpdate(ResponseItem.newBuilder().setXIndex("i").setXId("3").setStatus(0).setResult("updated").build()) + .build() + ) + .addItems( + Item.newBuilder() + .setDelete(ResponseItem.newBuilder().setXIndex("i").setXId("4").setStatus(0).setResult("deleted").build()) + .build() + ) + .build(); + BulkResponse resp = BulkResponseConverter.fromProto(proto); + assertEquals(OperationType.Index, resp.items().get(0).operationType()); + assertEquals(OperationType.Create, resp.items().get(1).operationType()); + assertEquals(OperationType.Update, resp.items().get(2).operationType()); + assertEquals(OperationType.Delete, resp.items().get(3).operationType()); + } + + @Test + public void testResponseWithShards() { + org.opensearch.protobufs.BulkResponse proto = org.opensearch.protobufs.BulkResponse.newBuilder() + .setTook(1) + .setErrors(false) + .addItems( + Item.newBuilder() + .setIndex( + ResponseItem.newBuilder() + .setXIndex("i") + .setXId("1") + .setStatus(0) + .setXShards(ShardInfo.newBuilder().setTotal(2).setSuccessful(2).setFailed(0).build()) + .build() + ) + .build() + ) + .build(); + BulkResponse resp = BulkResponseConverter.fromProto(proto); + assertEquals(2, resp.items().get(0).shards().total()); + assertEquals(2, resp.items().get(0).shards().successful()); + } + + @Test + public void testIngestTook() { + org.opensearch.protobufs.BulkResponse proto = org.opensearch.protobufs.BulkResponse.newBuilder() + .setTook(100) + .setErrors(false) + .setIngestTook(25) + .addItems(Item.newBuilder().setIndex(ResponseItem.newBuilder().setXIndex("i").setXId("1").setStatus(0).build()).build()) + .build(); + assertEquals(Long.valueOf(25L), BulkResponseConverter.fromProto(proto).ingestTook()); + } + + // ═══ Helper ══════════════════════════════════════════════════════════════════ + + public static class Doc { + private String name; + private int value; + + public Doc() {} + + public Doc(String n, int v) { + name = n; + value = v; + } + + public String getName() { + return name; + } + + public void setName(String n) { + name = n; + } + + public int getValue() { + return value; + } + + public void setValue(int v) { + value = v; + } + } +} diff --git a/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/AbstractGrpcIT.java b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/AbstractGrpcIT.java new file mode 100644 index 0000000000..2bbb6096c2 --- /dev/null +++ b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/AbstractGrpcIT.java @@ -0,0 +1,149 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.opensearch.integTest.grpc; + +import java.io.IOException; +import org.apache.hc.core5.http.HttpHost; +import org.junit.Assume; +import org.junit.Before; +import org.junit.ClassRule; +import org.junit.runner.RunWith; +import org.junit.runners.JUnit4; +import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.opensearch.OpenSearchClient; +import org.opensearch.client.opensearch.core.InfoResponse; +import org.opensearch.client.transport.OpenSearchTransport; +import org.opensearch.client.transport.grpc.GrpcTransport; +import org.opensearch.client.transport.grpc.GrpcTransportOptions; +import org.opensearch.client.transport.grpc.HybridTransport; +import org.opensearch.client.transport.httpclient5.ApacheHttpClient5TransportBuilder; + +/** + * Base class for gRPC integration tests. + *

+ * Self-contained — does not depend on internal test framework classes from java-client. + * Uses {@link OpenSearchGrpcTestContainerRule} to start an OpenSearch container with gRPC enabled, + * then creates a {@link HybridTransport} that routes bulk over gRPC and everything else over REST. + *

+ * Subclasses get access to: + *

    + *
  • {@link #grpcClient()} — OpenSearchClient using HybridTransport (bulk → gRPC, rest → REST)
    • + *
  • {@link #getGrpcHost()} — gRPC hostname
    • + *
  • {@link #getGrpcPort()} — gRPC mapped port
    • + *
+ *

+ * All tests automatically skip if the server version is below 3.5.0 (gRPC not available). + */ +@RunWith(JUnit4.class) +public abstract class AbstractGrpcIT { + + private static final int GRPC_MIN_MAJOR = 3; + private static final int GRPC_MIN_MINOR = 5; + + @ClassRule + public static final OpenSearchGrpcTestContainerRule grpcContainer = new OpenSearchGrpcTestContainerRule(); + + private static OpenSearchClient grpcClient; + + @Before + public void initGrpcClient() throws IOException { + if (grpcClient == null) { + String grpcCluster = System.getProperty("tests.grpc.cluster"); + if (grpcCluster == null) { + // Use REST host with default gRPC port if not explicitly configured + String restCluster = getRestCluster(); + String host = restCluster.split(":")[0]; + grpcCluster = host + ":" + OpenSearchGrpcTestContainerRule.GRPC_PORT; + } + + String[] parts = grpcCluster.split(":"); + String grpcHost = parts[0]; + int grpcPort = Integer.parseInt(parts[1]); + + // Build REST transport + String restCluster = getRestCluster(); + String[] restParts = restCluster.split(":"); + HttpHost restHost = new HttpHost("http", restParts[0], Integer.parseInt(restParts[1])); + OpenSearchTransport restTransport = ApacheHttpClient5TransportBuilder.builder(restHost).build(); + + // Build gRPC transport + GrpcTransport grpcTransport = GrpcTransport.builder(grpcHost, grpcPort) + .jsonpMapper(new JacksonJsonpMapper()) + .grpcOptions(GrpcTransportOptions.builder().maxRetries(2).build()) + .build(); + + // Combine into HybridTransport + HybridTransport hybridTransport = new HybridTransport(grpcTransport, restTransport); + grpcClient = new OpenSearchClient(hybridTransport); + } + } + + /** + * Returns the REST cluster address from system properties. + */ + private String getRestCluster() { + String cluster = System.getProperty("tests.rest.cluster"); + if (cluster == null || cluster.isBlank()) { + return "localhost:9200"; + } + return cluster; + } + + /** + * Returns the OpenSearch client configured with HybridTransport (gRPC + REST). + */ + protected static OpenSearchClient grpcClient() { + return grpcClient; + } + + /** + * Returns the gRPC host from system properties. + */ + protected String getGrpcHost() { + String grpcCluster = System.getProperty("tests.grpc.cluster", "localhost:9400"); + return grpcCluster.split(":")[0]; + } + + /** + * Returns the gRPC port from system properties. + */ + protected int getGrpcPort() { + String grpcCluster = System.getProperty("tests.grpc.cluster", "localhost:9400"); + return Integer.parseInt(grpcCluster.split(":")[1]); + } + + /** + * Checks that the server version supports gRPC (3.5.0+). + * Call this at the top of each test method: + * 

+     * assumeGrpcSupported();
+     *
+ */ + protected void assumeGrpcSupported() throws IOException { + InfoResponse info = grpcClient().info(); + String version = info.version().number(); + String[] versionParts = version.split("\\."); + int major = Integer.parseInt(versionParts[0]); + int minor = Integer.parseInt(versionParts[1]); + + Assume.assumeTrue( + "gRPC transport requires OpenSearch 3.5.0+, but server is " + version, + major > GRPC_MIN_MAJOR || (major == GRPC_MIN_MAJOR && minor >= GRPC_MIN_MINOR) + ); + + // Also verify gRPC port is actually reachable + try { + java.net.Socket socket = new java.net.Socket(); + socket.connect(new java.net.InetSocketAddress(getGrpcHost(), getGrpcPort()), 2000); + socket.close(); + } catch (IOException e) { + Assume.assumeTrue("gRPC port not reachable at " + getGrpcHost() + ":" + getGrpcPort(), false); + } + } +} diff --git a/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcBulkIT.java b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcBulkIT.java new file mode 100644 index 0000000000..5f736b1c33 --- /dev/null +++ b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcBulkIT.java @@ -0,0 +1,259 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.opensearch.integTest.grpc; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import org.junit.Test; +import org.opensearch.client.opensearch._types.Refresh; +import org.opensearch.client.opensearch.core.BulkRequest; +import org.opensearch.client.opensearch.core.BulkResponse; +import org.opensearch.client.opensearch.core.GetResponse; +import org.opensearch.client.opensearch.core.InfoResponse; +import org.opensearch.client.opensearch.core.SearchResponse; +import org.opensearch.client.opensearch.core.bulk.BulkOperation; +import org.opensearch.client.opensearch.core.bulk.BulkResponseItem; +import org.opensearch.client.opensearch.core.bulk.IndexOperation; +import org.opensearch.client.opensearch.core.bulk.OperationType; +import org.opensearch.client.opensearch.indices.CreateIndexRequest; +import org.opensearch.client.opensearch.indices.DeleteIndexRequest; + +/** + * End-to-end integration tests for gRPC Bulk transport. + *

+ * Verifies the complete pipeline: + *

    + *
  1. Client builds BulkRequest (Java API types)
    2. + *
  2. GrpcTransport converts to protobuf
    3. + *
  3. Sent over gRPC channel to OpenSearch on port 9400
    4. + *
  4. Server processes and returns protobuf BulkResponse
    5. + *
  5. GrpcTransport converts response back to Java API types
    6. + *
  6. Client receives standard BulkResponse
    7. + *
+ *

+ * Tests are condensed to minimize cluster load while covering: + * - All 4 operation types (index, create, update, delete) + * - Response format completeness (_shards, _version, _seq_no, _primary_term) + * - Status code mapping (201, 200, 409, 404) + * - REST routing for non-bulk operations + *

+ * Skips automatically on OpenSearch versions below 3.5.0. + *

+ * Run: + * 

+ * ./gradlew integrationTest --tests "org.opensearch.client.opensearch.integTest.grpc.GrpcBulkIT" \
+ *   -Dtests.opensearch.version=3.5.0
+ *
+ */ +public class GrpcBulkIT extends AbstractGrpcIT { + + private static final String INDEX = "grpc-bulk-it"; + + // ─── Test Document ─────────────────────────────────────────────────────────── + + public static class Product { + private String name; + private double price; + private int quantity; + + public Product() {} + + public Product(String name, double price, int quantity) { + this.name = name; + this.price = price; + this.quantity = quantity; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public double getPrice() { + return price; + } + + public void setPrice(double price) { + this.price = price; + } + + public int getQuantity() { + return quantity; + } + + public void setQuantity(int quantity) { + this.quantity = quantity; + } + } + + // ─── Tests ─────────────────────────────────────────────────────────────────── + + /** + * Tests all 4 bulk operations in one call and validates the full response format. + * Covers: index, create, update, delete, response fields, status codes. + */ + @Test + public void testBulkMixedOpsAndResponseFormat() throws IOException { + assumeGrpcSupported(); + + String index = INDEX + "-mixed"; + grpcClient().indices().create(new CreateIndexRequest.Builder().index(index).build()); + + try { + // Step 1: Seed with index + create + List seedOps = new ArrayList<>(); + seedOps.add( + new BulkOperation.Builder().index( + new IndexOperation.Builder().index(index).id("1").document(new Product("Laptop", 999.99, 10)).build() + ).build() + ); + seedOps.add(new BulkOperation.Builder().create(c -> c.index(index).id("2").document(new Product("Mouse", 29.99, 200))).build()); + + BulkResponse seedResp = grpcClient().bulk( + new BulkRequest.Builder().index(index).operations(seedOps).refresh(Refresh.True).build() + ); + + // Validate response format on seed + assertFalse("Should have no errors", seedResp.errors()); + assertEquals(2, seedResp.items().size()); + assertTrue("took should be >= 0", seedResp.took() >= 0); + + // Validate index response item fields + BulkResponseItem indexItem = seedResp.items().get(0); + assertEquals(OperationType.Index, indexItem.operationType()); + assertEquals(index, indexItem.index()); + assertEquals("1", indexItem.id()); + assertEquals("created", indexItem.result()); + assertNotNull("Should have _version", indexItem.version()); + assertNotNull("Should have _seq_no", indexItem.seqNo()); + assertNotNull("Should have _primary_term", indexItem.primaryTerm()); + assertNotNull("Should have _shards", indexItem.shards()); + assertTrue("_shards.successful >= 1", indexItem.shards().successful() >= 1); + + // Validate create response item + BulkResponseItem createItem = seedResp.items().get(1); + assertEquals(OperationType.Create, createItem.operationType()); + assertEquals("created", createItem.result()); + + // Step 2: Mixed ops — update + delete in one call + List mixedOps = new ArrayList<>(); + mixedOps.add( + new BulkOperation.Builder().update(u -> u.index(index).id("1").document(new Product("Laptop Pro", 1299.99, 5))).build() + ); + mixedOps.add(new BulkOperation.Builder().delete(d -> d.index(index).id("2")).build()); + + BulkResponse mixedResp = grpcClient().bulk( + new BulkRequest.Builder().index(index).operations(mixedOps).refresh(Refresh.True).build() + ); + + assertFalse(mixedResp.errors()); + assertEquals(2, mixedResp.items().size()); + + // Update returns status 200 and result "updated" + BulkResponseItem updateItem = mixedResp.items().get(0); + assertEquals(OperationType.Update, updateItem.operationType()); + assertEquals("updated", updateItem.result()); + + // Delete returns result "deleted" + BulkResponseItem deleteItem = mixedResp.items().get(1); + assertEquals(OperationType.Delete, deleteItem.operationType()); + assertEquals("deleted", deleteItem.result()); + + // Step 3: Verify final state via REST + SearchResponse search = grpcClient().search(s -> s.index(index), Product.class); + assertEquals("Should have 1 doc remaining", 1, search.hits().total().value()); + + GetResponse get = grpcClient().get(g -> g.index(index).id("1"), Product.class); + assertTrue(get.found()); + assertEquals("Laptop Pro", get.source().getName()); + + } finally { + grpcClient().indices().delete(new DeleteIndexRequest.Builder().index(index).build()); + } + } + + /** + * Tests error status codes: 409 conflict and 404 not found. + */ + @Test + public void testBulkErrorStatusCodes() throws IOException { + assumeGrpcSupported(); + + String index = INDEX + "-errors"; + grpcClient().indices().create(new CreateIndexRequest.Builder().index(index).build()); + + try { + // Create a doc + List createOps = new ArrayList<>(); + createOps.add( + new BulkOperation.Builder().create(c -> c.index(index).id("exists").document(new Product("Existing", 1.0, 1))).build() + ); + grpcClient().bulk(new BulkRequest.Builder().index(index).operations(createOps).refresh(Refresh.True).build()); + + // Conflict (409): create same doc again + Not found (404): delete nonexistent + List errorOps = new ArrayList<>(); + errorOps.add( + new BulkOperation.Builder().create(c -> c.index(index).id("exists").document(new Product("Duplicate", 2.0, 2))).build() + ); + errorOps.add(new BulkOperation.Builder().delete(d -> d.index(index).id("nonexistent")).build()); + + BulkResponse errorResp = grpcClient().bulk( + new BulkRequest.Builder().index(index).operations(errorOps).refresh(Refresh.True).build() + ); + + assertTrue("Should have errors", errorResp.errors()); + assertEquals(2, errorResp.items().size()); + + // Create conflict → 409 + BulkResponseItem conflictItem = errorResp.items().get(0); + assertEquals(409, conflictItem.status()); + assertNotNull("Should have error detail", conflictItem.error()); + + // Delete not found → 404 + BulkResponseItem notFoundItem = errorResp.items().get(1); + assertEquals(404, notFoundItem.status()); + + } finally { + grpcClient().indices().delete(new DeleteIndexRequest.Builder().index(index).build()); + } + } + + /** + * Tests that non-bulk operations are routed to REST. + */ + @Test + public void testRestRouting() throws IOException { + assumeGrpcSupported(); + + // info() — REST + InfoResponse info = grpcClient().info(); + assertNotNull(info); + assertNotNull(info.version().number()); + + // Index create/search/delete — all REST + String index = INDEX + "-routing"; + grpcClient().indices().create(new CreateIndexRequest.Builder().index(index).build()); + try { + SearchResponse resp = grpcClient().search(s -> s.index(index), Product.class); + assertNotNull(resp); + assertEquals(0, resp.hits().total().value()); + } finally { + grpcClient().indices().delete(new DeleteIndexRequest.Builder().index(index).build()); + } + } +} diff --git a/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcTransportSupport.java b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcTransportSupport.java new file mode 100644 index 0000000000..d22d7ce14f --- /dev/null +++ b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcTransportSupport.java @@ -0,0 +1,46 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.opensearch.integTest.grpc; + +import java.io.IOException; +import org.apache.hc.core5.http.HttpHost; +import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.transport.OpenSearchTransport; +import org.opensearch.client.transport.grpc.GrpcTransport; +import org.opensearch.client.transport.grpc.GrpcTransportOptions; +import org.opensearch.client.transport.grpc.HybridTransport; +import org.opensearch.client.transport.httpclient5.ApacheHttpClient5TransportBuilder; + +/** + * Utility for building a HybridTransport (gRPC + REST) in integration tests. + *

+ * Self-contained — does not depend on internal test framework interfaces from java-client. + */ +final class GrpcTransportSupport { + + private GrpcTransportSupport() {} + + /** + * Builds a HybridTransport that routes Bulk over gRPC and everything else over REST. + * + * @param restHost the REST host (typically from testcontainers) + * @param grpcHost the gRPC hostname + * @param grpcPort the gRPC port + */ + static OpenSearchTransport buildHybridTransport(HttpHost restHost, String grpcHost, int grpcPort) throws IOException { + OpenSearchTransport restTransport = ApacheHttpClient5TransportBuilder.builder(restHost).build(); + + GrpcTransport grpcTransport = GrpcTransport.builder(grpcHost, grpcPort) + .jsonpMapper(new JacksonJsonpMapper()) + .grpcOptions(GrpcTransportOptions.builder().maxRetries(2).build()) + .build(); + + return new HybridTransport(grpcTransport, restTransport); + } +} diff --git a/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/OpenSearchGrpcTestContainerRule.java b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/OpenSearchGrpcTestContainerRule.java new file mode 100644 index 0000000000..c314a6dc5b --- /dev/null +++ b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/OpenSearchGrpcTestContainerRule.java @@ -0,0 +1,135 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.opensearch.integTest.grpc; + +import static org.junit.Assume.assumeTrue; + +import java.net.URI; +import org.junit.rules.ExternalResource; +import org.opensearch.testcontainers.OpenSearchContainer; +import org.opensearch.testcontainers.OpenSearchDockerImage; + +/** + * Starts an OpenSearch test container with gRPC transport enabled. + *

+ * Configures the container with: + *

    + *
  • gRPC transport enabled on port 9400
    • + *
  • Security disabled (plaintext gRPC for testing)
    • + *
  • Port 9400 exposed for gRPC connections
    • + *
+ *

+ * Sets system properties so tests can discover the cluster: + *

    + *
  • {@code tests.rest.cluster} — host:port for REST (9200)
    • + *
  • {@code tests.grpc.cluster} — host:port for gRPC (9400)
    • + *
  • {@code https} — false (security disabled)
    • + *
+ */ +final class OpenSearchGrpcTestContainerRule extends ExternalResource { + private static final String ENABLED_PROPERTY = "tests.opensearch.testcontainers.enabled"; + private static final String VERSION_PROPERTY = "tests.opensearch.version"; + private static final String IMAGE_PROPERTY = "tests.opensearch.image"; + private static final String CLUSTER_PROPERTY = "tests.rest.cluster"; + private static final String GRPC_CLUSTER_PROPERTY = "tests.grpc.cluster"; + private static final String HTTPS_PROPERTY = "https"; + + static final int GRPC_PORT = 9400; + + private static OpenSearchContainer container; + + @Override + protected void before() { + // Skip tests for versions that don't support gRPC + String version = System.getProperty(VERSION_PROPERTY, ""); + assumeTrue("OpenSearch should support gRPC", supportsGrpc(version)); + + try { + startIfNeeded(); + } catch (Exception e) { + System.err.println("OpenSearchGrpcTestContainerRule: container failed to start: " + e.getMessage()); + assumeTrue("OpenSearch container failed to start: " + e.getMessage(), false); + } + } + + private static void startIfNeeded() { + if (hasText(System.getProperty(CLUSTER_PROPERTY)) || !testcontainersEnabled()) { + return; + } + + if (container == null) { + OpenSearchContainer openSearch = createContainer(); + openSearch.start(); + container = openSearch; + } + + // REST endpoint + URI httpHostAddress = URI.create(container.getHttpHostAddress()); + System.setProperty(CLUSTER_PROPERTY, httpHostAddress.getHost() + ":" + httpHostAddress.getPort()); + System.setProperty(HTTPS_PROPERTY, "false"); + + // gRPC endpoint — get the mapped port for 9400 + String grpcHost = container.getHost(); + Integer grpcMappedPort = container.getMappedPort(GRPC_PORT); + System.setProperty(GRPC_CLUSTER_PROPERTY, grpcHost + ":" + grpcMappedPort); + } + + private static OpenSearchContainer createContainer() { + String image = System.getProperty(IMAGE_PROPERTY); + OpenSearchContainer openSearch = hasText(image) + ? new OpenSearchContainer<>(image) + : new OpenSearchContainer<>(OpenSearchDockerImage.ofVersion(requiredVersion())); + + // Disable security for plaintext testing + openSearch.withEnv("plugins.security.disabled", "true"); + + // Enable gRPC transport (we only reach here for 3.5.0+) + openSearch.withEnv("aux.transport.types", "transport-grpc"); + openSearch.withEnv("aux.transport.transport-grpc.port", String.valueOf(GRPC_PORT)); + openSearch.withExposedPorts(9200, GRPC_PORT); + + // Disable disk watermarks for CI + openSearch.withEnv("cluster.routing.allocation.disk.threshold_enabled", "false"); + + // Allow extra startup time for gRPC-enabled containers in CI + openSearch.withStartupTimeout(java.time.Duration.ofMinutes(5)); + + return openSearch; + } + + private static boolean supportsGrpc(String version) { + if (!hasText(version)) { + return false; + } + try { + String[] parts = version.replace("-SNAPSHOT", "").split("\\."); + int major = Integer.parseInt(parts[0]); + int minor = parts.length > 1 ? Integer.parseInt(parts[1]) : 0; + return major > 3 || (major == 3 && minor >= 5); + } catch (NumberFormatException e) { + return false; + } + } + + private static String requiredVersion() { + String version = System.getProperty(VERSION_PROPERTY); + if (!hasText(version)) { + throw new IllegalStateException("Missing " + VERSION_PROPERTY + " for OpenSearch Testcontainers image"); + } + return version; + } + + private static boolean testcontainersEnabled() { + return Boolean.parseBoolean(System.getProperty(ENABLED_PROPERTY, "true")); + } + + private static boolean hasText(String value) { + return value != null && !value.isBlank(); + } +} diff --git a/samples/build.gradle.kts b/samples/build.gradle.kts index 1ca5c814c9..bf83ed2cd3 100644 --- a/samples/build.gradle.kts +++ b/samples/build.gradle.kts @@ -20,11 +20,16 @@ java { dependencies { implementation(project(":java-client")) + implementation(project(":java-client-grpc")) implementation("org.apache.logging.log4j", "log4j-api","[2.17.1,3.0)") implementation("org.apache.logging.log4j", "log4j-core","[2.17.1,3.0)") implementation("org.apache.logging.log4j", "log4j-slf4j2-impl","[2.17.1,3.0)") implementation("commons-logging", "commons-logging", "1.2") implementation("com.fasterxml.jackson.core", "jackson-databind", "2.15.2") + implementation("software.amazon.awssdk", "sdk-core", "[2.21,3.0)") + implementation("software.amazon.awssdk", "auth", "[2.21,3.0)") + implementation("software.amazon.awssdk", "http-auth-aws", "[2.21,3.0)") + implementation("software.amazon.awssdk", "apache-client", "[2.21,3.0)") } application { diff --git a/samples/src/main/java/org/opensearch/client/samples/GrpcAwsSigV4.java b/samples/src/main/java/org/opensearch/client/samples/GrpcAwsSigV4.java new file mode 100644 index 0000000000..128ca0b773 --- /dev/null +++ b/samples/src/main/java/org/opensearch/client/samples/GrpcAwsSigV4.java @@ -0,0 +1,137 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.samples; + +import java.util.ArrayList; +import java.util.List; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.opensearch.OpenSearchClient; +import org.opensearch.client.opensearch._types.Refresh; +import org.opensearch.client.opensearch.core.BulkRequest; +import org.opensearch.client.opensearch.core.BulkResponse; +import org.opensearch.client.opensearch.core.bulk.BulkOperation; +import org.opensearch.client.opensearch.core.bulk.IndexOperation; +import org.opensearch.client.transport.OpenSearchTransport; +import org.opensearch.client.transport.aws.AwsSdk2Transport; +import org.opensearch.client.transport.aws.AwsSdk2TransportOptions; +import org.opensearch.client.transport.grpc.AwsGrpcTransport; +import org.opensearch.client.transport.grpc.GrpcSigV4Config; +import org.opensearch.client.transport.grpc.GrpcTlsConfig; +import org.opensearch.client.transport.grpc.GrpcTransport; +import org.opensearch.client.transport.grpc.HybridTransport; +import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider; +import software.amazon.awssdk.http.SdkHttpClient; +import software.amazon.awssdk.http.apache.ApacheHttpClient; +import software.amazon.awssdk.regions.Region; + +/** + * Demonstrates using gRPC with AWS SigV4 signing against Amazon OpenSearch Service. + *

+ * This shows how to configure: + * - gRPC transport with SigV4 authentication (for bulk) + * - AwsSdk2Transport for REST (unsupported ops like search) (for search, info, etc.) + * - Both using the same AWS credentials from the default provider chain + *

+ * Prerequisites: + * - An Amazon OpenSearch Service domain with gRPC enabled (OS 3.5+) + * - AWS credentials configured (env vars, ~/.aws/credentials, or instance profile) + * - IAM permissions for the OpenSearch domain + *

+ * Run with: {@code ./gradlew :samples:run -Dsamples.mainClass=GrpcAwsSigV4} + *

+ * Environment variables: + * - OPENSEARCH_DOMAIN: domain endpoint (e.g., search-my-domain.us-east-1.es.amazonaws.com) + * - AWS_REGION: AWS region (default: us-east-1) + * - GRPC_PORT: gRPC port (default: 9400) + */ +public class GrpcAwsSigV4 { + private static final Logger LOGGER = LogManager.getLogger(GrpcAwsSigV4.class); + + public static void main(String[] args) { + try { + var env = System.getenv(); + var domain = env.getOrDefault("OPENSEARCH_DOMAIN", "search-my-domain.us-east-1.es.amazonaws.com"); + var region = Region.of(env.getOrDefault("AWS_REGION", "us-east-1")); + var grpcPort = Integer.parseInt(env.getOrDefault("GRPC_PORT", "9400")); + var service = env.getOrDefault("OPENSEARCH_SERVICE", "es"); // "es" or "aoss" + + LOGGER.info("Connecting to {} in {} (service={})", domain, region, service); + + // ─── REST transport with SigV4 (for non-bulk operations) ───────────── + + SdkHttpClient httpClient = ApacheHttpClient.builder().build(); + OpenSearchTransport restTransport = new AwsSdk2Transport( + httpClient, + domain, + service, + region, + AwsSdk2TransportOptions.builder().build() + ); + + // ─── gRPC transport with SigV4 (for bulk operations) ───────────────── + + GrpcTransport grpcTransport = AwsGrpcTransport.awsBuilder(domain, grpcPort) + .jsonpMapper(new JacksonJsonpMapper()) + .tls(GrpcTlsConfig.builder().build()) // system trust store (AWS certs are in JVM cacerts) + .sigV4( + GrpcSigV4Config.builder() + .region(region) + .service(service) + .credentialsProvider(DefaultCredentialsProvider.create()) + .build() + ) + .build(); + + // ─── Combine into HybridTransport ──────────────────────────────────── + + HybridTransport transport = new HybridTransport(grpcTransport, restTransport); + OpenSearchClient client = new OpenSearchClient(transport); + + // ─── Use the client ────────────────────────────────────────────────── + + // info() → REST with SigV4 + var info = client.info(); + LOGGER.info("Connected: {} {}", info.version().distribution(), info.version().number()); + + // bulk() → gRPC with SigV4 + final var indexName = "grpc-sigv4-sample"; + List ops = new ArrayList<>(); + for (int i = 0; i < 10; i++) { + final int id = i; + ops.add( + new BulkOperation.Builder().index( + new IndexOperation.Builder().index(indexName) + .id(String.valueOf(id)) + .document(new GrpcBulk.SampleDoc("AWS Doc " + id, id * 2.0, id)) + .build() + ).build() + ); + } + + BulkResponse response = client.bulk(new BulkRequest.Builder().index(indexName).operations(ops).refresh(Refresh.True).build()); + LOGGER.info("Bulk via gRPC+SigV4: took={}ms, errors={}", response.took(), response.errors()); + + // search() → REST with SigV4 + var search = client.search(s -> s.index(indexName).size(3), GrpcBulk.SampleDoc.class); + LOGGER.info("Search via REST+SigV4: {} hits", search.hits().total().value()); + + // cleanup + client.indices().delete(r -> r.index(indexName)); + transport.close(); + httpClient.close(); + + LOGGER.info("Done!"); + + } catch (Exception e) { + LOGGER.error("Error: {}", e.getMessage(), e); + } + } +} diff --git a/samples/src/main/java/org/opensearch/client/samples/GrpcBulk.java b/samples/src/main/java/org/opensearch/client/samples/GrpcBulk.java new file mode 100644 index 0000000000..cada4f11a0 --- /dev/null +++ b/samples/src/main/java/org/opensearch/client/samples/GrpcBulk.java @@ -0,0 +1,209 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.samples; + +import java.util.ArrayList; +import java.util.List; +import org.apache.hc.core5.http.HttpHost; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.opensearch.OpenSearchClient; +import org.opensearch.client.opensearch._types.Refresh; +import org.opensearch.client.opensearch.core.BulkRequest; +import org.opensearch.client.opensearch.core.BulkResponse; +import org.opensearch.client.opensearch.core.SearchResponse; +import org.opensearch.client.opensearch.core.bulk.BulkOperation; +import org.opensearch.client.opensearch.core.bulk.BulkResponseItem; +import org.opensearch.client.opensearch.core.bulk.IndexOperation; +import org.opensearch.client.transport.OpenSearchTransport; +import org.opensearch.client.transport.grpc.GrpcTransport; +import org.opensearch.client.transport.grpc.HybridTransport; +import org.opensearch.client.transport.httpclient5.ApacheHttpClient5TransportBuilder; + +/** + * Demonstrates using gRPC transport for high-performance bulk operations. + *

+ * gRPC provides lower latency and higher throughput for bulk indexing compared to REST. + * The HybridTransport automatically routes bulk operations over gRPC and everything + * else (search, index management, etc.) over REST. + *

+ * Prerequisites: + * - OpenSearch 3.5+ with gRPC transport enabled: + * {@code aux.transport.types: [transport-grpc]} + * {@code aux.transport.transport-grpc.port: '9400'} + *

+ * Run with: {@code ./gradlew :samples:run -Dsamples.mainClass=GrpcBulk} + *

+ * Environment variables: + * - HOST: OpenSearch hostname (default: localhost) + * - PORT: REST port (default: 9200) + * - GRPC_PORT: gRPC port (default: 9400) + * - USERNAME: basic auth username (default: admin) + * - PASSWORD: basic auth password (default: admin) + * - HTTPS: use TLS (default: true) + */ +public class GrpcBulk { + private static final Logger LOGGER = LogManager.getLogger(GrpcBulk.class); + + public static void main(String[] args) { + try { + var env = System.getenv(); + var hostname = env.getOrDefault("HOST", "localhost"); + var restPort = Integer.parseInt(env.getOrDefault("PORT", "9200")); + var grpcPort = Integer.parseInt(env.getOrDefault("GRPC_PORT", "9400")); + var user = env.getOrDefault("USERNAME", "admin"); + var pass = env.getOrDefault("PASSWORD", "admin"); + var https = Boolean.parseBoolean(env.getOrDefault("HTTPS", "false")); + + // ─── Build the HybridTransport ─────────────────────────────────────── + // Bulk → gRPC (port 9400), everything else → REST (port 9200) + + OpenSearchClient client = createClient(hostname, restPort, grpcPort, user, pass, https); + + var version = client.info().version(); + LOGGER.info("Server: {}@{} (REST)", version.distribution(), version.number()); + + // ─── Create Index ──────────────────────────────────────────────────── + + final var indexName = "grpc-sample"; + + if (client.indices().exists(r -> r.index(indexName)).value()) { + client.indices().delete(r -> r.index(indexName)); + } + client.indices().create(r -> r.index(indexName)); + LOGGER.info("Created index: {}", indexName); + + // ─── Bulk Index via gRPC ───────────────────────────────────────────── + // This call is transparently routed over gRPC by HybridTransport + + List operations = new ArrayList<>(); + for (int i = 0; i < 100; i++) { + final int docId = i; + operations.add( + new BulkOperation.Builder().index( + new IndexOperation.Builder().index(indexName) + .id(String.valueOf(docId)) + .document(new SampleDoc("Document " + docId, docId * 1.5, docId % 10)) + .build() + ).build() + ); + } + + LOGGER.info("Bulk indexing 100 documents over gRPC..."); + BulkResponse bulkResponse = client.bulk( + new BulkRequest.Builder().index(indexName).operations(operations).refresh(Refresh.True).build() + ); + + LOGGER.info( + "Bulk response: took={}ms, errors={}, items={}", + bulkResponse.took(), + bulkResponse.errors(), + bulkResponse.items().size() + ); + + if (!bulkResponse.errors()) { + LOGGER.info("All 100 documents indexed successfully via gRPC!"); + } else { + for (BulkResponseItem item : bulkResponse.items()) { + if (item.error() != null) { + LOGGER.error(" Failed: id={}, error={}", item.id(), item.error().reason()); + } + } + } + + // ─── Search via REST ───────────────────────────────────────────────── + // Search is automatically routed to REST by HybridTransport + + LOGGER.info("Searching via REST..."); + SearchResponse searchResponse = client.search(s -> s.index(indexName).size(5), SampleDoc.class); + LOGGER.info("Search: found {} documents (showing first 5)", searchResponse.hits().total().value()); + for (var hit : searchResponse.hits().hits()) { + LOGGER.info(" {} (score={})", hit.source().name, hit.score()); + } + + // ─── Cleanup ───────────────────────────────────────────────────────── + + client.indices().delete(r -> r.index(indexName)); + LOGGER.info("Deleted index: {}", indexName); + + } catch (Exception e) { + LOGGER.error("Error: {}", e.getMessage(), e); + } + } + + /** + * Creates an OpenSearchClient with HybridTransport (gRPC + REST). + */ + private static OpenSearchClient createClient(String hostname, int restPort, int grpcPort, String user, String pass, boolean https) + throws Exception { + + // 1. REST transport (for search, index management, etc.) + var hosts = new HttpHost[] { new HttpHost(https ? "https" : "http", hostname, restPort) }; + OpenSearchTransport restTransport = ApacheHttpClient5TransportBuilder.builder(hosts).build(); + + // 2. gRPC transport (for bulk operations) + var grpcBuilder = GrpcTransport.builder(hostname, grpcPort).jsonpMapper(new JacksonJsonpMapper()); + + // Configure auth and TLS + if (https) { + grpcBuilder.tlsInsecure(); // trust all for local dev (use proper certs in production) + } + if (user != null && !user.isEmpty()) { + grpcBuilder.basicAuth(user, pass); + } + + GrpcTransport grpcTransport = grpcBuilder.build(); + + // 3. Combine into HybridTransport + HybridTransport hybridTransport = new HybridTransport(grpcTransport, restTransport); + + return new OpenSearchClient(hybridTransport); + } + + // ─── Sample Document ───────────────────────────────────────────────────────── + + public static class SampleDoc { + public String name; + public double score; + public int category; + + public SampleDoc() {} + + public SampleDoc(String name, double score, int category) { + this.name = name; + this.score = score; + this.category = category; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public double getScore() { + return score; + } + + public void setScore(double score) { + this.score = score; + } + + public int getCategory() { + return category; + } + + public void setCategory(int category) { + this.category = category; + } + } +} diff --git a/samples/src/main/java/org/opensearch/client/samples/GrpcDemo.java b/samples/src/main/java/org/opensearch/client/samples/GrpcDemo.java new file mode 100644 index 0000000000..ab475ff4d2 --- /dev/null +++ b/samples/src/main/java/org/opensearch/client/samples/GrpcDemo.java @@ -0,0 +1,267 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.client.samples; + +import java.util.ArrayList; +import java.util.List; +import org.apache.hc.core5.http.HttpHost; +import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.opensearch.OpenSearchClient; +import org.opensearch.client.opensearch._types.Refresh; +import org.opensearch.client.opensearch.core.BulkRequest; +import org.opensearch.client.opensearch.core.BulkResponse; +import org.opensearch.client.opensearch.core.SearchResponse; +import org.opensearch.client.opensearch.core.bulk.BulkOperation; +import org.opensearch.client.opensearch.core.bulk.IndexOperation; +import org.opensearch.client.transport.grpc.GrpcTransport; +import org.opensearch.client.transport.grpc.HybridTransport; +import org.opensearch.client.transport.httpclient5.ApacheHttpClient5TransportBuilder; + +public class GrpcDemo { + + static final String REST_HOST = "localhost"; + static final int REST_PORT = 9200; + static final int GRPC_PORT = 9400; + static final String INDEX = "grpc-demo-products"; + static final int NUM_DOCS = 500; + + public static void main(String[] args) throws Exception { + System.out.println("\n🎬 OpenSearch gRPC Demo — Java Client"); + System.out.println(" Cluster: localhost:9200 (REST) + localhost:9400 (gRPC)"); + System.out.println(" Documents: " + NUM_DOCS + "\n"); + + long restTime = demoRestBulk(); + long grpcTime = demoGrpcBulk(); + demoRouting(); + + // Summary + System.out.println("\n" + "=".repeat(60)); + System.out.println(" SUMMARY"); + System.out.println("=".repeat(60)); + System.out.printf(" REST bulk: %dms%n", restTime); + System.out.printf(" gRPC bulk: %dms%n", grpcTime); + if (grpcTime < restTime) { + double speedup = ((double) (restTime - grpcTime) / restTime) * 100; + System.out.printf(" Speedup: %.1f%% faster with gRPC%n", speedup); + } + System.out.println("\n Key takeaway: Same client API, zero code migration,"); + System.out.println(" REST routing for unsupported operations.\n"); + } + + // ─── Demo 1: REST Bulk (baseline) ──────────────────────────────────────── + + static long demoRestBulk() throws Exception { + System.out.println("\n" + "=".repeat(60)); + System.out.println(" DEMO 1: Bulk Indexing via REST"); + System.out.println("=".repeat(60)); + + var restTransport = ApacheHttpClient5TransportBuilder.builder(new HttpHost("http", REST_HOST, REST_PORT)).build(); + var client = new OpenSearchClient(restTransport); + + List ops = buildBulkOps(); + + long start = System.currentTimeMillis(); + BulkResponse response = client.bulk(new BulkRequest.Builder().index(INDEX).operations(ops).refresh(Refresh.True).build()); + long elapsed = System.currentTimeMillis() - start; + + System.out.printf(" ✅ Indexed %d documents via REST%n", NUM_DOCS); + System.out.printf(" ⏱ Time: %dms%n", elapsed); + System.out.printf(" 📊 Errors: %s%n", response.errors()); + System.out.printf(" 📊 Took (server): %dms%n", response.took()); + + client.indices().delete(d -> d.index(INDEX)); + restTransport.close(); + return elapsed; + } + + // ─── Demo 2: gRPC Bulk (transparent) ───────────────────────────────────── + + static long demoGrpcBulk() throws Exception { + System.out.println("\n" + "=".repeat(60)); + System.out.println(" DEMO 2: Bulk Indexing via gRPC (transparent)"); + System.out.println("=".repeat(60)); + + // REST transport for unsupported operations + var restTransport = ApacheHttpClient5TransportBuilder.builder(new HttpHost("http", REST_HOST, REST_PORT)).build(); + + // gRPC transport for bulk + var grpcTransport = GrpcTransport.builder(REST_HOST, GRPC_PORT).jsonpMapper(new JacksonJsonpMapper()).build(); + + // Combine — bulk → gRPC, everything else → REST + var hybridTransport = new HybridTransport(grpcTransport, restTransport); + var client = new OpenSearchClient(hybridTransport); + + List ops = buildBulkOps(); + + // Show what the client sends + System.out.println(); + System.out.printf(" 📤 CLIENT REQUEST (first 2 operations):%n"); + System.out.printf(" Action: POST /_bulk (routed to gRPC on port %d)%n", GRPC_PORT); + System.out.printf(" Operations: %d index operations%n", NUM_DOCS); + System.out.println(" Sample:"); + System.out.printf(" {\"index\": {\"_index\": \"%s\", \"_id\": \"0\"}}%n", INDEX); + System.out.println(" {\"name\": \"Product 0\", \"price\": 9.99, \"category\": \"electronics\", \"inStock\": true}"); + System.out.printf(" {\"index\": {\"_index\": \"%s\", \"_id\": \"1\"}}%n", INDEX); + System.out.println(" {\"name\": \"Product 1\", \"price\": 10.49, \"category\": \"clothing\", \"inStock\": true}"); + System.out.printf(" ... (%d more operations)%n", NUM_DOCS - 2); + System.out.println(); + System.out.println(" ⚙️ INTERNAL CONVERSION (transparent to user):"); + System.out.println(" BulkRequest (Java API) → BulkRequestConverter → protobuf BulkRequest"); + System.out.println(" → gRPC channel (HTTP/2, binary) → OpenSearch server:" + GRPC_PORT); + System.out.println(); + + long start = System.currentTimeMillis(); + BulkResponse response = client.bulk(new BulkRequest.Builder().index(INDEX).operations(ops).refresh(Refresh.True).build()); + long elapsed = System.currentTimeMillis() - start; + + // Show the response as the client receives it + System.out.println(" 📥 SERVER RESPONSE (as client sees it — same format as REST):"); + System.out.println(" {"); + System.out.printf(" \"took\": %d,%n", response.took()); + System.out.printf(" \"errors\": %s,%n", response.errors()); + System.out.println(" \"items\": ["); + // Show first 3 items + var items = response.items(); + for (int i = 0; i < Math.min(3, items.size()); i++) { + var item = items.get(i); + System.out.printf(" {\"%s\": {%n", item.operationType().jsonValue()); + System.out.printf(" \"_index\": \"%s\",%n", item.index()); + System.out.printf(" \"_id\": \"%s\",%n", item.id()); + System.out.printf(" \"result\": \"%s\",%n", item.result()); + System.out.printf(" \"status\": %d,%n", item.status()); + if (item.version() != null) System.out.printf(" \"_version\": %d,%n", item.version()); + if (item.seqNo() != null) System.out.printf(" \"_seq_no\": %d,%n", item.seqNo()); + if (item.primaryTerm() != null) System.out.printf(" \"_primary_term\": %d,%n", item.primaryTerm()); + if (item.shards() != null) System.out.printf( + " \"_shards\": {\"total\": %d, \"successful\": %d, \"failed\": %d}%n", + item.shards().total(), + item.shards().successful(), + item.shards().failed() + ); + System.out.printf(" }}%s%n", i < 2 ? "," : ""); + } + System.out.printf(" ... (%d more items)%n", NUM_DOCS - 3); + System.out.println(" ]"); + System.out.println(" }"); + System.out.println(); + System.out.printf(" ✅ Indexed %d documents via gRPC%n", NUM_DOCS); + System.out.printf(" ⏱ Time: %dms%n", elapsed); + System.out.printf(" 📊 Took (server): %dms%n", response.took()); + + // Search goes via REST automatically + SearchResponse search = client.search(s -> s.index(INDEX).size(3), Product.class); + System.out.printf(" 🔍 Search (via REST): found %d docs%n", search.hits().total().value()); + + client.indices().delete(d -> d.index(INDEX)); + hybridTransport.close(); + return elapsed; + } + + // ─── Demo 3: Fallback ──────────────────────────────────────────────────── + + static void demoRouting() throws Exception { + System.out.println("\n" + "=".repeat(60)); + System.out.println(" DEMO 3: REST routing — unsupported ops go to REST"); + System.out.println("=".repeat(60)); + + var restTransport = ApacheHttpClient5TransportBuilder.builder(new HttpHost("http", REST_HOST, REST_PORT)).build(); + + var grpcTransport = GrpcTransport.builder(REST_HOST, GRPC_PORT).jsonpMapper(new JacksonJsonpMapper()).build(); + + var hybridTransport = new HybridTransport(grpcTransport, restTransport); + var client = new OpenSearchClient(hybridTransport); + + // Search is not supported by gRPC — routed to REST automatically + System.out.println(" 📤 client.search() → not in gRPC registry → REST"); + var info = client.info(); + System.out.printf(" ✅ info() via REST: %s %s%n", info.version().distribution(), info.version().number()); + + // Index create/delete also goes to REST + System.out.println(" 📤 client.indices().create() → REST"); + client.indices().create(c -> c.index(INDEX + "-routing")); + System.out.println(" ✅ Index created via REST"); + + client.indices().delete(d -> d.index(INDEX + "-routing")); + System.out.println(" ✅ Index deleted via REST"); + System.out.println(); + System.out.println(" 💡 HybridTransport routes by endpoint:"); + System.out.println(" bulk() → gRPC (supported)"); + System.out.println(" search() → REST (not yet supported by gRPC)"); + System.out.println(" info() → REST (not yet supported by gRPC)"); + + hybridTransport.close(); + } + + // ─── Helpers ───────────────────────────────────────────────────────────── + + static List buildBulkOps() { + String[] categories = { "electronics", "clothing", "home", "sports" }; + List ops = new ArrayList<>(); + for (int i = 0; i < NUM_DOCS; i++) { + final int id = i; + ops.add( + new BulkOperation.Builder().index( + new IndexOperation.Builder().index(INDEX) + .id(String.valueOf(id)) + .document(new Product("Product " + id, 9.99 + id * 0.5, categories[id % 4], id % 3 != 0)) + .build() + ).build() + ); + } + return ops; + } + + public static class Product { + public String name; + public double price; + public String category; + public boolean inStock; + + public Product() {} + + public Product(String name, double price, String category, boolean inStock) { + this.name = name; + this.price = price; + this.category = category; + this.inStock = inStock; + } + + public String getName() { + return name; + } + + public void setName(String n) { + name = n; + } + + public double getPrice() { + return price; + } + + public void setPrice(double p) { + price = p; + } + + public String getCategory() { + return category; + } + + public void setCategory(String c) { + category = c; + } + + public boolean isInStock() { + return inStock; + } + + public void setInStock(boolean s) { + inStock = s; + } + } +} diff --git a/settings.gradle.kts b/settings.gradle.kts index 488278fdba..e2f1f16a3a 100644 --- a/settings.gradle.kts +++ b/settings.gradle.kts @@ -36,5 +36,6 @@ plugins { rootProject.name = "opensearch-java" include("java-client") +include("java-client-grpc") include("java-codegen") include("samples") diff --git a/validate-grpc-integration.sh b/validate-grpc-integration.sh new file mode 100755 index 0000000000..eef83e9ce8 --- /dev/null +++ b/validate-grpc-integration.sh @@ -0,0 +1,147 @@ +#!/bin/bash +# ───────────────────────────────────────────────────────────────────────────── +# gRPC Integration Test Validation Script +# +# Runs an OpenSearch 3.5+ container with gRPC enabled, then executes the +# integration tests against it. +# +# Usage: +# ./validate-grpc-integration.sh +# +# Requirements: +# - Docker running +# - Java 21+ (for integration tests) +# - Gradle wrapper in project root +# +# Environment variables (optional): +# OPENSEARCH_VERSION — OpenSearch version to test (default: 3.5.0) +# GRPC_PORT — gRPC port to expose (default: 9400) +# REST_PORT — REST port to expose (default: 9200) +# KEEP_CONTAINER — set to "true" to keep container running after tests +# ───────────────────────────────────────────────────────────────────────────── + +set -e + +OPENSEARCH_VERSION="${OPENSEARCH_VERSION:-3.5.0}" +GRPC_PORT="${GRPC_PORT:-9400}" +REST_PORT="${REST_PORT:-9200}" +CONTAINER_NAME="opensearch-grpc-test" +IMAGE="opensearchproject/opensearch:${OPENSEARCH_VERSION}" + +echo "═══════════════════════════════════════════════════════════════" +echo " gRPC Integration Test Validation" +echo " OpenSearch Version: ${OPENSEARCH_VERSION}" +echo " REST Port: ${REST_PORT}" +echo " gRPC Port: ${GRPC_PORT}" +echo "═══════════════════════════════════════════════════════════════" + +# ─── Cleanup any existing container ────────────────────────────────────────── +echo "" +echo "▶ Cleaning up any existing container..." +docker rm -f ${CONTAINER_NAME} 2>/dev/null || true + +# ─── Start OpenSearch with gRPC enabled ────────────────────────────────────── +echo "" +echo "▶ Starting OpenSearch ${OPENSEARCH_VERSION} with gRPC transport..." +docker run -d \ + --name ${CONTAINER_NAME} \ + -p ${REST_PORT}:9200 \ + -p ${GRPC_PORT}:9400 \ + -e "discovery.type=single-node" \ + -e "plugins.security.disabled=true" \ + -e "aux.transport.types=transport-grpc" \ + -e "aux.transport.transport-grpc.port=9400" \ + -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=admin" \ + -e "cluster.routing.allocation.disk.threshold_enabled=false" \ + ${IMAGE} + +# ─── Wait for cluster to be ready ──────────────────────────────────────────── +echo "" +echo "▶ Waiting for OpenSearch to be ready..." +MAX_RETRIES=30 +RETRY_INTERVAL=3 +for i in $(seq 1 $MAX_RETRIES); do + if curl -s "http://localhost:${REST_PORT}/_cluster/health" | grep -q '"status"'; then + echo " ✓ OpenSearch is ready!" + break + fi + if [ $i -eq $MAX_RETRIES ]; then + echo " ✗ OpenSearch failed to start after ${MAX_RETRIES} attempts" + docker logs ${CONTAINER_NAME} | tail -30 + docker rm -f ${CONTAINER_NAME} + exit 1 + fi + echo " Waiting... (attempt $i/$MAX_RETRIES)" + sleep $RETRY_INTERVAL +done + +# ─── Verify gRPC port is listening ─────────────────────────────────────────── +echo "" +echo "▶ Verifying gRPC port ${GRPC_PORT} is open..." +if nc -z localhost ${GRPC_PORT} 2>/dev/null; then + echo " ✓ gRPC port is open!" +else + echo " ⚠ gRPC port not responding (nc check failed, may still work)" +fi + +# ─── Show cluster info ──────────────────────────────────────────────────────── +echo "" +echo "▶ Cluster info:" +curl -s "http://localhost:${REST_PORT}" | python3 -m json.tool 2>/dev/null || curl -s "http://localhost:${REST_PORT}" +echo "" + +# ─── Run integration tests ─────────────────────────────────────────────────── +echo "" +echo "═══════════════════════════════════════════════════════════════" +echo " Running Integration Tests" +echo "═══════════════════════════════════════════════════════════════" +echo "" + +# Run the gRPC integration tests +./gradlew :java-client:integrationTest \ + --tests "org.opensearch.client.opensearch.integTest.grpc.*" \ + -Dtests.rest.cluster=localhost:${REST_PORT} \ + -Dtests.grpc.cluster=localhost:${GRPC_PORT} \ + -Dtests.opensearch.version=${OPENSEARCH_VERSION} \ + -Dtests.opensearch.testcontainers.enabled=false \ + -Dhttps=false \ + -Duser=admin \ + -Dpassword=admin \ + --info 2>&1 | tee /tmp/grpc-integration-test.log + +TEST_EXIT_CODE=${PIPESTATUS[0]} + +# ─── Results ────────────────────────────────────────────────────────────────── +echo "" +echo "═══════════════════════════════════════════════════════════════" +if [ $TEST_EXIT_CODE -eq 0 ]; then + echo " ✓ ALL INTEGRATION TESTS PASSED" +else + echo " ✗ INTEGRATION TESTS FAILED (exit code: ${TEST_EXIT_CODE})" + echo "" + echo " Test report: java-client/build/reports/tests/integrationTest/index.html" + echo " Full log: /tmp/grpc-integration-test.log" +fi +echo "═══════════════════════════════════════════════════════════════" + +# ─── Also run unit tests for sanity ────────────────────────────────────────── +echo "" +echo "▶ Running unit tests (sanity check)..." +./gradlew :java-client:test --tests "org.opensearch.client.transport.grpc.*" 2>&1 | tail -5 + +# ─── Cleanup ────────────────────────────────────────────────────────────────── +if [ "${KEEP_CONTAINER}" != "true" ]; then + echo "" + echo "▶ Stopping container..." + docker rm -f ${CONTAINER_NAME} + echo " ✓ Container removed" +else + echo "" + echo "▶ Container left running (KEEP_CONTAINER=true)" + echo " REST: http://localhost:${REST_PORT}" + echo " gRPC: localhost:${GRPC_PORT}" + echo " Stop with: docker rm -f ${CONTAINER_NAME}" +fi + +echo "" +exit $TEST_EXIT_CODE From 94e4ca4fe1c80fb1affbefc8d2507059a6ae2b06 Mon Sep 17 00:00:00 2001 From: Andriy Redko Date: Wed, 22 Jul 2026 18:44:28 -0400 Subject: [PATCH 2/2] Update to match 3.x release line baseline Signed-off-by: Andriy Redko --- java-client-grpc/build.gradle.kts | 2 +- .../org/opensearch/client/transport/grpc/GrpcChannelTest.java | 2 +- .../org/opensearch/client/transport/grpc/GrpcSigV4Test.java | 4 ++-- .../opensearch/client/transport/grpc/GrpcTransportTest.java | 2 +- .../client/transport/grpc/JwtAuthInterceptorTest.java | 4 ++-- .../client/transport/grpc/translation/TranslationTest.java | 2 +- .../client/opensearch/integTest/grpc/AbstractGrpcIT.java | 2 +- .../opensearch/integTest/grpc/GrpcTransportSupport.java | 2 +- .../main/java/org/opensearch/client/samples/GrpcAwsSigV4.java | 2 +- .../src/main/java/org/opensearch/client/samples/GrpcBulk.java | 2 +- .../src/main/java/org/opensearch/client/samples/GrpcDemo.java | 2 +- 11 files changed, 13 insertions(+), 13 deletions(-) diff --git a/java-client-grpc/build.gradle.kts b/java-client-grpc/build.gradle.kts index 951ae9f013..cac63af7b3 100644 --- a/java-client-grpc/build.gradle.kts +++ b/java-client-grpc/build.gradle.kts @@ -27,7 +27,7 @@ java { withSourcesJar() } -val opensearchVersion = "3.5.0-SNAPSHOT" +val opensearchVersion = "3.5.0" val grpcVersion = "1.68.0" val protobufVersion = "3.25.5" val opensearchProtobufVersion = "1.2.0" diff --git a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcChannelTest.java b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcChannelTest.java index 88131bf1ef..4c65ae4312 100644 --- a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcChannelTest.java +++ b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcChannelTest.java @@ -127,7 +127,7 @@ public void testCreateChannelNullTls() throws IOException { @Test public void testBuilderWithTlsAndBasicAuth() { GrpcTransport t = GrpcTransport.builder("localhost", 9400) - .jsonpMapper(new org.opensearch.client.json.jackson3.JacksonJsonpMapper()) + .jsonpMapper(new org.opensearch.client.json.jackson.JacksonJsonpMapper()) .tlsInsecure() .basicAuth("a", "b") .build(); diff --git a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcSigV4Test.java b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcSigV4Test.java index e383cb4109..47cdb3e4f8 100644 --- a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcSigV4Test.java +++ b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcSigV4Test.java @@ -313,7 +313,7 @@ public void testEmptyHost() { @Test(expected = IllegalStateException.class) public void testBuilderRequiresTls() { AwsGrpcTransport.awsBuilder("localhost", 9400) - .jsonpMapper(new org.opensearch.client.json.jackson3.JacksonJsonpMapper()) + .jsonpMapper(new org.opensearch.client.json.jackson.JacksonJsonpMapper()) .sigV4( GrpcSigV4Config.builder() .region(Region.US_EAST_1) @@ -326,7 +326,7 @@ public void testBuilderRequiresTls() { @Test public void testBuilderWithTlsAndSigV4() { GrpcTransport t = AwsGrpcTransport.awsBuilder(HOST, 9400) - .jsonpMapper(new org.opensearch.client.json.jackson3.JacksonJsonpMapper()) + .jsonpMapper(new org.opensearch.client.json.jackson.JacksonJsonpMapper()) .tls(GrpcTlsConfig.insecure()) .sigV4( GrpcSigV4Config.builder() diff --git a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcTransportTest.java b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcTransportTest.java index 092ef99a01..1ed6c102db 100644 --- a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcTransportTest.java +++ b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/GrpcTransportTest.java @@ -21,7 +21,7 @@ import javax.annotation.Nullable; import org.junit.Test; import org.opensearch.client.json.JsonpMapper; -import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.json.jackson.JacksonJsonpMapper; import org.opensearch.client.opensearch.core.BulkRequest; import org.opensearch.client.opensearch.core.BulkResponse; import org.opensearch.client.transport.Endpoint; diff --git a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/JwtAuthInterceptorTest.java b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/JwtAuthInterceptorTest.java index 7347a49e6c..0975f51395 100644 --- a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/JwtAuthInterceptorTest.java +++ b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/JwtAuthInterceptorTest.java @@ -41,7 +41,7 @@ public void testNullSupplierThrows() { @Test public void testBuilderWithJwt() { GrpcTransport t = GrpcTransport.builder("localhost", 9400) - .jsonpMapper(new org.opensearch.client.json.jackson3.JacksonJsonpMapper()) + .jsonpMapper(new org.opensearch.client.json.jackson.JacksonJsonpMapper()) .jwtAuth(() -> "token") .build(); assertNotNull(t); @@ -53,7 +53,7 @@ public void testBuilderWithJwt() { @Test public void testBuilderWithTlsAndJwt() { GrpcTransport t = GrpcTransport.builder("localhost", 9400) - .jsonpMapper(new org.opensearch.client.json.jackson3.JacksonJsonpMapper()) + .jsonpMapper(new org.opensearch.client.json.jackson.JacksonJsonpMapper()) .tlsInsecure() .jwtAuth(() -> "token") .build(); diff --git a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/translation/TranslationTest.java b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/translation/TranslationTest.java index a893551415..c61ad95cfe 100644 --- a/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/translation/TranslationTest.java +++ b/java-client-grpc/src/test/java/org/opensearch/client/transport/grpc/translation/TranslationTest.java @@ -21,7 +21,7 @@ import org.junit.Before; import org.junit.Test; import org.opensearch.client.json.JsonpMapper; -import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.json.jackson.JacksonJsonpMapper; import org.opensearch.client.opensearch._types.OpenSearchException; import org.opensearch.client.opensearch._types.Refresh; import org.opensearch.client.opensearch._types.VersionType; diff --git a/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/AbstractGrpcIT.java b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/AbstractGrpcIT.java index 2bbb6096c2..5021d0efda 100644 --- a/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/AbstractGrpcIT.java +++ b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/AbstractGrpcIT.java @@ -15,7 +15,7 @@ import org.junit.ClassRule; import org.junit.runner.RunWith; import org.junit.runners.JUnit4; -import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.json.jackson.JacksonJsonpMapper; import org.opensearch.client.opensearch.OpenSearchClient; import org.opensearch.client.opensearch.core.InfoResponse; import org.opensearch.client.transport.OpenSearchTransport; diff --git a/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcTransportSupport.java b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcTransportSupport.java index d22d7ce14f..d899b0d66f 100644 --- a/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcTransportSupport.java +++ b/java-client-grpc/src/test/java11/org/opensearch/client/opensearch/integTest/grpc/GrpcTransportSupport.java @@ -10,7 +10,7 @@ import java.io.IOException; import org.apache.hc.core5.http.HttpHost; -import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.json.jackson.JacksonJsonpMapper; import org.opensearch.client.transport.OpenSearchTransport; import org.opensearch.client.transport.grpc.GrpcTransport; import org.opensearch.client.transport.grpc.GrpcTransportOptions; diff --git a/samples/src/main/java/org/opensearch/client/samples/GrpcAwsSigV4.java b/samples/src/main/java/org/opensearch/client/samples/GrpcAwsSigV4.java index 128ca0b773..cbda7d8511 100644 --- a/samples/src/main/java/org/opensearch/client/samples/GrpcAwsSigV4.java +++ b/samples/src/main/java/org/opensearch/client/samples/GrpcAwsSigV4.java @@ -12,7 +12,7 @@ import java.util.List; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; -import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.json.jackson.JacksonJsonpMapper; import org.opensearch.client.opensearch.OpenSearchClient; import org.opensearch.client.opensearch._types.Refresh; import org.opensearch.client.opensearch.core.BulkRequest; diff --git a/samples/src/main/java/org/opensearch/client/samples/GrpcBulk.java b/samples/src/main/java/org/opensearch/client/samples/GrpcBulk.java index cada4f11a0..9119cba652 100644 --- a/samples/src/main/java/org/opensearch/client/samples/GrpcBulk.java +++ b/samples/src/main/java/org/opensearch/client/samples/GrpcBulk.java @@ -13,7 +13,7 @@ import org.apache.hc.core5.http.HttpHost; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; -import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.json.jackson.JacksonJsonpMapper; import org.opensearch.client.opensearch.OpenSearchClient; import org.opensearch.client.opensearch._types.Refresh; import org.opensearch.client.opensearch.core.BulkRequest; diff --git a/samples/src/main/java/org/opensearch/client/samples/GrpcDemo.java b/samples/src/main/java/org/opensearch/client/samples/GrpcDemo.java index ab475ff4d2..09839c56c5 100644 --- a/samples/src/main/java/org/opensearch/client/samples/GrpcDemo.java +++ b/samples/src/main/java/org/opensearch/client/samples/GrpcDemo.java @@ -11,7 +11,7 @@ import java.util.ArrayList; import java.util.List; import org.apache.hc.core5.http.HttpHost; -import org.opensearch.client.json.jackson3.JacksonJsonpMapper; +import org.opensearch.client.json.jackson.JacksonJsonpMapper; import org.opensearch.client.opensearch.OpenSearchClient; import org.opensearch.client.opensearch._types.Refresh; import org.opensearch.client.opensearch.core.BulkRequest;