Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Bump `org.apache.httpcomponents.client5:httpclient5` from 5.6 to 5.6.1 ([#1967](https://github.com/opensearch-project/opensearch-java/pull/1967))

### Added
- Add `_Custom` variant kind to all discriminated tagged unions (e.g. `Aggregate`, `Suggest`, `Query`, `Property`, `Processor`) to gracefully handle plugin-provided types without deserialization failures
- Run Java client integration tests with a Testcontainers-managed OpenSearch instance by default ([#2033](https://github.com/opensearch-project/opensearch-java/pull/2033))
- Detect AWS SDK `Apache5HttpClient` in `AwsSdk2Transport` body-method guardrail ([#1903](https://github.com/opensearch-project/opensearch-java/pull/1970))
- Support Jackson 3.x release line ([#1810](https://github.com/opensearch-project/opensearch-java/pull/1810))
Expand All @@ -25,6 +26,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Add transparent gRPC transport with HybridTransport (bulk over gRPC, REST fallback), translation layer, TLS, basic auth, AWS SigV4, and JWT support ([#2062](https://github.com/opensearch-project/opensearch-java/pull/2062))

### Fixed
- Fix `JsonData._DESERIALIZER` to correctly handle a pre-consumed parser event in the three-argument `deserialize` overload

## [Unreleased 3.x]
### Added
Expand Down
39 changes: 38 additions & 1 deletion guides/json.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
- [Deserialization](#deserialization)
- [Using withJson](#using-withjson)
- [Using static _DESERIALIZER](#using-static-_deserializer)
- [Custom (Plugin-Provided) Variant Types](#custom-plugin-provided-variant-types)
- [Reading a custom variant](#reading-a-custom-variant)
- [Building a custom variant](#building-a-custom-variant)


# Working With JSON
Expand Down Expand Up @@ -109,4 +112,38 @@ private IndexTemplateMapping getInstance(String templateJsonString) {
return indexTemplateMapping;
}
}
```
```

## Custom (Plugin-Provided) Variant Types

All discriminated tagged unions in this client — both externally-tagged (e.g. `Aggregate`, `Suggest`) and internally-tagged (e.g. `Query`, `Property`, `Processor`, `Analyzer`, `TokenFilterDefinition`) — support a `_Custom` variant kind.

When the server returns a discriminator value that isn't natively modeled (for example, a type introduced by a plugin), the client captures it as `Kind._Custom` with the raw JSON preserved as `JsonData`, rather than throwing a deserialization error.

### Reading a custom variant

```java
SearchResponse<IndexData> response = client.search(searchRequest, IndexData.class);

for (Map.Entry<String, Aggregate> entry : response.aggregations().entrySet()) {
Aggregate agg = entry.getValue();
if (agg._isCustom()) {
// _customKind() returns the type name from the server (e.g. "my_plugin_agg")
String typeName = agg._customKind();
// _custom() returns the raw JSON body as JsonData
JsonData rawData = agg._custom();
System.out.printf("Custom aggregation '%s' of type '%s': %s%n",
entry.getKey(), typeName, rawData.toJson());
}
}
```

### Building a custom variant

```java
Aggregate custom = new Aggregate.Builder()
._custom("my_plugin_agg", JsonData.of(Map.of("score", 42)))
.build();
```

The same `_isCustom()`, `_customKind()`, `_custom()`, and `Builder._custom(type, data)` methods are available on every discriminated tagged union type.
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
import javax.annotation.Generated;
import javax.annotation.Nonnull;
import org.opensearch.client.json.ExternallyTaggedUnion;
import org.opensearch.client.json.JsonData;
import org.opensearch.client.json.JsonEnum;
import org.opensearch.client.json.JsonpDeserializer;
import org.opensearch.client.json.JsonpMapper;
Expand Down Expand Up @@ -122,7 +123,9 @@ public enum Kind implements JsonEnum {
Umterms("umterms"),
ValueCount("value_count"),
VariableWidthHistogram("variable_width_histogram"),
WeightedAvg("weighted_avg");
WeightedAvg("weighted_avg"),
/** A custom variant type not natively supported by this client. */
_Custom(null);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the pull request @acote-coveo , mostly all code for the Java client is being generated (this is why every source file has @Generated("org.opensearch.client.codegen.CodeGenerator") annotation and the comment on top).

Please submit the change to the OpenSearch OpenAPI specification so it could be picked up automatically: https://github.com/opensearch-project/opensearch-api-specification/blob/main/spec/schemas/_common.aggregations.yaml#L9

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reta Thanks for the quick feedback. I just wanted to mention that I didn't explicitly modify the generated, I instead customized the generator template because I assumed that this kind of customization should not be part of the API specification. To me, the API specification should describe what OpenSearch supports. The custom kind that I added is a client-side concern to allow customization over the typed DSL.

For example: https://github.com/opensearch-project/opensearch-java/pull/2072/changes#diff-93745660477f5c12415912b6c105074b530c83c722b2771390de7f4e9b458474R6-R8

Let me know with my answer if you still think this should be document at the API level.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I were to add this to the API Specification, I would probably add a new custom annotation to mark the variants that are non-exhaustive and from the generator, I would look for this tag to know if I should generate the _custom variant code I just added. The main benefits would be that:

  1. We could control which variants are open or not.
  2. Other generators would have the same signals to generate exhaustive vs non-exhaustive variants.

@reta reta Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reta Thanks for the quick feedback. I just wanted to mention that I didn't explicitly modify the generated, I instead customized the generator template because I assumed that this kind of customization should not be part of the API specification.

My apologies @acote-coveo , jumped into quick conclusion. The support of custom types (queries, aggregations, etc) is a long standing one indeed. However, not every component does support such extensibility, I think we could combine specification approach (fe add X-Extensible: true/false attribute to the schema) and the generator changes that you are are suggesting,. It will at least bring clarify what could be customized, hope it make sense,.

Thank you.


private final String jsonValue;

Expand All @@ -138,6 +141,7 @@ public String jsonValue() {

private final Kind _kind;
private final AggregateVariant _value;
private final String _customKind;

@Override
public final Kind _kind() {
Expand All @@ -149,14 +153,23 @@ public final AggregateVariant _get() {
return _value;
}

/**
* Returns the actual type name when {@code _kind() == Kind._Custom}, otherwise {@code null}.
*/
public final String _customKind() {
return _customKind;
}

public Aggregate(AggregateVariant value) {
this._kind = ApiTypeHelper.requireNonNull(value._aggregateKind(), this, "<variant kind>");
this._value = ApiTypeHelper.requireNonNull(value, this, "<variant value>");
this._customKind = null;
}

private Aggregate(Builder builder) {
this._kind = ApiTypeHelper.requireNonNull(builder._kind, builder, "<variant kind>");
this._value = ApiTypeHelper.requireNonNull(builder._value, builder, "<variant value>");
this._customKind = builder._customKind;
}

public static Aggregate of(Function<Aggregate.Builder, ObjectBuilder<Aggregate>> fn) {
Expand Down Expand Up @@ -1139,6 +1152,25 @@ public WeightedAvgAggregate weightedAvg() {
return TaggedUnionUtils.get(this, Kind.WeightedAvg);
}

/**
* Is this variant instance of kind {@code _custom}?
*/
public boolean _isCustom() {
return _kind == Kind._Custom;
}

/**
* Get the raw JSON data for a custom (plugin-provided) variant type.
*
* @throws IllegalStateException if the current variant is not the {@code _custom} kind.
*/
public JsonData _custom() {
if (_kind != Kind._Custom) {
throw new IllegalStateException("Expected variant kind '_custom' but got '" + _kind + "'");
}
return ((CustomVariant) _value).data();
}

@Override
public void serialize(JsonGenerator generator, JsonpMapper mapper) {
mapper.serialize(_value, generator);
Expand All @@ -1157,12 +1189,14 @@ public static Builder builder() {
public static class Builder extends ObjectBuilderBase implements ObjectBuilder<Aggregate> {
private Kind _kind;
private AggregateVariant _value;
private String _customKind;

public Builder() {}

private Builder(Aggregate o) {
this._kind = o._kind;
this._value = o._value;
this._customKind = o._customKind;
}

public ObjectBuilder<Aggregate> adjacencyMatrix(AdjacencyMatrixAggregate v) {
Expand Down Expand Up @@ -1809,6 +1843,19 @@ public ObjectBuilder<Aggregate> weightedAvg(Function<WeightedAvgAggregate.Builde
return this.weightedAvg(fn.apply(new WeightedAvgAggregate.Builder()).build());
}

/**
* Set a custom (plugin-provided) variant.
*
* @param type the variant type name as returned by the server
* @param data the raw JSON body of the variant result
*/
public ObjectBuilder<Aggregate> _custom(String type, JsonData data) {
this._kind = Kind._Custom;
this._customKind = ApiTypeHelper.requireNonNull(type, this, "<custom variant type>");
this._value = new CustomVariant(ApiTypeHelper.requireNonNull(data, this, "<custom variant data>"));
return this;
}

@Override
public Aggregate build() {
_checkSingleUse();
Expand All @@ -1818,6 +1865,10 @@ public Aggregate build() {

public static final ExternallyTaggedUnion.TypedKeysDeserializer<Aggregate> _TYPED_KEYS_DESERIALIZER;

private static Aggregate _buildCustom(String type, JsonData data) {
return new Builder()._custom(type, data).build();
}

static {
Map<String, JsonpDeserializer<? extends AggregateVariant>> deserializers = new HashMap<>();
deserializers.put("adjacency_matrix", AdjacencyMatrixAggregate._DESERIALIZER);
Expand Down Expand Up @@ -1882,14 +1933,16 @@ public Aggregate build() {
deserializers.put("variable_width_histogram", VariableWidthHistogramAggregate._DESERIALIZER);
deserializers.put("weighted_avg", WeightedAvgAggregate._DESERIALIZER);

_TYPED_KEYS_DESERIALIZER = new ExternallyTaggedUnion.Deserializer<>(deserializers, Aggregate::new).typedKeys();
_TYPED_KEYS_DESERIALIZER = new ExternallyTaggedUnion.Deserializer<>(deserializers, Aggregate::new, Aggregate::_buildCustom)
.typedKeys();
}

@Override
public int hashCode() {
int result = 17;
result = 31 * result + Objects.hashCode(this._kind);
result = 31 * result + Objects.hashCode(this._value);
result = 31 * result + Objects.hashCode(this._customKind);
return result;
}

Expand All @@ -1898,6 +1951,43 @@ public boolean equals(Object o) {
if (this == o) return true;
if (o == null || this.getClass() != o.getClass()) return false;
Aggregate other = (Aggregate) o;
return Objects.equals(this._kind, other._kind) && Objects.equals(this._value, other._value);
return Objects.equals(this._kind, other._kind)
&& Objects.equals(this._value, other._value)
&& Objects.equals(this._customKind, other._customKind);
}

// Wrapper so JsonData fits the variant interface slot for custom/plugin types
private static final class CustomVariant implements AggregateVariant, PlainJsonSerializable {
private final JsonData data;

CustomVariant(JsonData data) {
this.data = data;
}

public JsonData data() {
return data;
}

@Override
public Kind _aggregateKind() {
return Kind._Custom;
}

@Override
public void serialize(JsonGenerator generator, JsonpMapper mapper) {
data.serialize(generator, mapper);
}

@Override
public int hashCode() {
return data.hashCode();
}

@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
return data.equals(((CustomVariant) o).data);
}
}
}
Loading
Loading