Skip to content

Commit 0e46059

Browse files
authored
MAL: add extension function SPI, proper numeric types in codegen (#13761)
**1. Extension Function SPI (`namespace::method()` syntax)** Adds a pluggable extension mechanism for MAL using `::` separator syntax, enabling external modules to contribute MAL functions that compile into direct static method calls. - `MalFunctionExtension` SPI interface with `name()` returning namespace - `@MALContextFunction` annotation on **static** methods — auto-detected via reflection at startup - First parameter must be `SampleFamily` (auto-bound), return type must be `SampleFamily` - Non-static annotated methods throw `IllegalArgumentException` at startup - Duplicate namespace or method names throw `IllegalArgumentException` at startup - `m.getDeclaringClass()` used for accurate FQCN in codegen (not `ext.getClass()`) - `List` parameters validated as `List<String>` via generic type inspection - Supports `String`, `double`, `float`, `long`, `int`, `List<String>` parameter types - Direct static method call codegen — no reflection or registry dispatch at runtime Example MAL script: ``` metric.sum(['svc']).test::scale(3.0) ``` Generated code: ```java sf = TestMalExtension.scale(sf, 3.0); ``` **2. Proper numeric types in MAL codegen** Integer-valued literals now emit `Long.valueOf(NL)` instead of `Double.valueOf(N.0)` in: - Binary arithmetic: `metric * 100` → `sf.multiply(Long.valueOf(100L))` - Built-in method args: `.multiply(100)` → `Long.valueOf(100L)` - Standalone number expressions: `SampleFamily.EMPTY.plus(Long.valueOf(100L))` Fractional values still use `Double.valueOf()`. All 1268 v1-v2 comparison tests pass. **3. GenAI model matcher singleton** `GenAIModelMatcher.getInstance()` — lazy singleton initialized from `gen-ai-config.yml`. `GenAIProviderPrefixMatcher.build()` simplified to delegate to singleton (removed unused config param).
1 parent f7f2e97 commit 0e46059

18 files changed

Lines changed: 790 additions & 73 deletions

File tree

docs/en/concepts-and-designs/mal.md

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,45 @@ buckets, will multiply the bucket value by 1000.)
236236
1. `element`: element in the array.
237237
2. `tags`: tags in each sample.
238238

239+
## Extension Functions
240+
241+
MAL supports extension functions via the `namespace::method()` syntax. Extensions are pluggable modules
242+
that add custom capabilities to MAL without modifying the core language. They are discovered at startup
243+
via Java `ServiceLoader`.
244+
245+
### Syntax
246+
247+
Extension functions are called with a `::` separator between the namespace and method name:
248+
249+
```
250+
metric.sum(['svc']).myext::transform(2.0)
251+
metric.genai::estimateCost()
252+
```
253+
254+
The `::` separator distinguishes extension calls from built-in `SampleFamily` methods (like `.sum()`,
255+
`.tag()`, `.filter()`). Method names only need to be unique within their namespace.
256+
257+
### Supported Parameter Types
258+
259+
Extension methods accept the following argument types from MAL expressions:
260+
261+
| Java Type | MAL Argument Example |
262+
|-----------|---------------------|
263+
| `String` | `"value"` |
264+
| `double` | `2.0` |
265+
| `int` | `100` |
266+
| `List<String>` | `["tag1", "tag2"]` |
267+
268+
The first parameter is always `SampleFamily`, which is automatically bound to the current value in the
269+
method chain. Only the additional parameters need to be specified in the MAL expression.
270+
271+
### Error Handling
272+
273+
The MAL compiler validates extension calls at compile time:
274+
- Unknown namespace or method name results in a compilation error.
275+
- Mismatched argument count results in a compilation error.
276+
- Type mismatches between MAL arguments and Java parameter types result in a compilation error.
277+
239278
## Down Sampling Operation
240279
MAL should instruct meter-system on how to downsample for metrics. It doesn't only refer to aggregate raw samples to
241280
`minute` level, but also expresses data from `minute` in higher levels, such as `hour` and `day`.

oap-server/analyzer/gen-ai-analyzer/src/main/java/org/apache/skywalking/oap/analyzer/genai/GenAIAnalyzerModuleProvider.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ public void onInitialized(final GenAIConfig initialized) {
6666
public void prepare() throws ServiceNotProvidedException, ModuleStartException {
6767
GenAIConfigLoader loader = new GenAIConfigLoader(config);
6868
config = loader.loadConfig();
69-
GenAIProviderPrefixMatcher matcher = GenAIProviderPrefixMatcher.build(config);
69+
GenAIProviderPrefixMatcher matcher = GenAIProviderPrefixMatcher.build();
7070
this.registerServiceImplementation(
7171
IGenAIMeterAnalyzerService.class,
7272
new GenAIMeterAnalyzer(matcher)

oap-server/analyzer/gen-ai-analyzer/src/main/java/org/apache/skywalking/oap/analyzer/genai/matcher/GenAIProviderPrefixMatcher.java

Lines changed: 8 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -22,54 +22,27 @@
2222
import org.apache.skywalking.oap.server.library.util.genai.GenAIModelMatcher;
2323
import org.apache.skywalking.oap.server.library.util.genai.GenAIPricingConfig;
2424

25-
import java.util.stream.Collectors;
26-
2725
/**
28-
* Delegates to {@link GenAIModelMatcher} in library-util.
29-
* Converts module-specific {@link GenAIConfig} to the shared {@link GenAIPricingConfig}.
26+
* Thin wrapper over the singleton {@link GenAIModelMatcher} that converts
27+
* results to module-specific {@link GenAIConfig.Model} types.
3028
*/
3129
public class GenAIProviderPrefixMatcher {
3230

33-
private final GenAIModelMatcher delegate;
34-
35-
private GenAIProviderPrefixMatcher(GenAIModelMatcher delegate) {
36-
this.delegate = delegate;
31+
private GenAIProviderPrefixMatcher() {
3732
}
3833

39-
public static GenAIProviderPrefixMatcher build(GenAIConfig config) {
40-
GenAIPricingConfig pricingConfig = toPricingConfig(config);
41-
return new GenAIProviderPrefixMatcher(GenAIModelMatcher.build(pricingConfig));
34+
public static GenAIProviderPrefixMatcher build() {
35+
// Ensure singleton is initialized (lazy init from gen-ai-config.yml)
36+
GenAIModelMatcher.getInstance();
37+
return new GenAIProviderPrefixMatcher();
4238
}
4339

4440
public MatchResult match(String modelName) {
45-
GenAIModelMatcher.MatchResult result = delegate.match(modelName);
41+
GenAIModelMatcher.MatchResult result = GenAIModelMatcher.getInstance().match(modelName);
4642
GenAIConfig.Model modelConfig = toModuleModel(result.getModelConfig());
4743
return new MatchResult(result.getProvider(), modelConfig);
4844
}
4945

50-
private static GenAIPricingConfig toPricingConfig(GenAIConfig config) {
51-
GenAIPricingConfig pricingConfig = new GenAIPricingConfig();
52-
pricingConfig.setProviders(
53-
config.getProviders().stream().map(p -> {
54-
GenAIPricingConfig.Provider pp = new GenAIPricingConfig.Provider();
55-
pp.setProvider(p.getProvider());
56-
pp.setPrefixMatch(p.getPrefixMatch());
57-
pp.setModels(
58-
p.getModels().stream().map(m -> {
59-
GenAIPricingConfig.Model pm = new GenAIPricingConfig.Model();
60-
pm.setName(m.getName());
61-
pm.setAliases(m.getAliases());
62-
pm.setInputEstimatedCostPerM(m.getInputEstimatedCostPerM());
63-
pm.setOutputEstimatedCostPerM(m.getOutputEstimatedCostPerM());
64-
return pm;
65-
}).collect(Collectors.toList())
66-
);
67-
return pp;
68-
}).collect(Collectors.toList())
69-
);
70-
return pricingConfig;
71-
}
72-
7346
private static GenAIConfig.Model toModuleModel(GenAIPricingConfig.Model pm) {
7447
if (pm == null) {
7548
return null;

oap-server/analyzer/gen-ai-analyzer/src/test/java/org/apache/skywalking/oap/genai/analyzer/GenAIMeterAnalyzerTest.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ void setUp() throws ModuleStartException {
4848
GenAIConfigLoader loader = new GenAIConfigLoader(config);
4949
loadedConfig = loader.loadConfig();
5050

51-
matcher = GenAIProviderPrefixMatcher.build(loadedConfig);
51+
matcher = GenAIProviderPrefixMatcher.build();
5252
analyzer = new GenAIMeterAnalyzer(matcher);
5353
}
5454

@@ -233,7 +233,7 @@ void testEstimatedCost() throws ModuleStartException {
233233
GenAIConfigLoader loader = new GenAIConfigLoader(config);
234234
GenAIConfig loadedConfig = loader.loadConfig();
235235

236-
GenAIProviderPrefixMatcher matcher = GenAIProviderPrefixMatcher.build(loadedConfig);
236+
GenAIProviderPrefixMatcher matcher = GenAIProviderPrefixMatcher.build();
237237

238238
GenAIProviderPrefixMatcher.MatchResult result = matcher.match("gpt-5.4-pro");
239239
assertNotNull(result.getModelConfig());

oap-server/analyzer/meter-analyzer/CLAUDE.md

Lines changed: 89 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,10 +41,16 @@ oap-server/analyzer/meter-analyzer/
4141
rt/
4242
MalExpressionPackageHolder.java — Class loading anchor (empty marker)
4343
MalRuntimeHelper.java — Static helpers called by generated code (divReverse, regexMatch, isTruthy)
44+
MalExtensionRegistry.java — SPI registry for extension functions (namespace::method)
45+
46+
src/main/java/.../spi/
47+
MalFunctionExtension.java — SPI interface for extension namespaces
48+
MALContextFunction.java — Annotation marking callable methods
4449
4550
src/test/java/.../compiler/
46-
MALScriptParserTest.java — 20 parser tests
47-
MALClassGeneratorTest.java — 32 generator tests
51+
MALScriptParserTest.java — 22 parser tests
52+
MALClassGeneratorTest.java — 20 generator tests
53+
MALExtensionFunctionTest.java — 9 extension SPI tests
4854
```
4955

5056
## Package & Class Naming
@@ -59,12 +65,93 @@ All v2 classes live under `org.apache.skywalking.oap.meter.analyzer.v2.*` to avo
5965
| Filter classes | `org.apache.skywalking.oap.meter.analyzer.v2.compiler.rt.{yamlName}_L{lineNo}_filter` |
6066
| Package holder | `org.apache.skywalking.oap.meter.analyzer.v2.compiler.rt.MalExpressionPackageHolder` |
6167
| Runtime helper | `org.apache.skywalking.oap.meter.analyzer.v2.compiler.rt.MalRuntimeHelper` |
68+
| Extension registry | `org.apache.skywalking.oap.meter.analyzer.v2.compiler.rt.MalExtensionRegistry` |
69+
| Extension SPI | `org.apache.skywalking.oap.meter.analyzer.v2.spi.MalFunctionExtension` |
70+
| Extension annotation | `org.apache.skywalking.oap.meter.analyzer.v2.spi.MALContextFunction` |
6271
| Functional interface | `org.apache.skywalking.oap.meter.analyzer.v2.dsl.MalExpression` |
6372

6473
Class names are built from `yamlSource` (file name + line number) and `classNameHint` (rule name or `filter`).
6574
Example: `vm_L25_cpu_total_percentage` (expression), `gateway_service_L33_filter` (filter).
6675
Falls back to `MalExpr_<N>` (global counter) when no hint is set.
6776

77+
## Extension Function SPI (`namespace::method()`)
78+
79+
MAL supports custom extension functions via the `namespace::method()` syntax. Extensions are discovered
80+
at startup via `java.util.ServiceLoader`.
81+
82+
### Syntax
83+
84+
```
85+
metric.sum(['svc']).genai::estimateCost()
86+
metric.test::scale(2.0)
87+
```
88+
89+
The `::` separator distinguishes extension calls from built-in `SampleFamily` methods. The namespace
90+
avoids global method name conflicts — method names only need to be unique within their namespace.
91+
92+
### Implementing an Extension
93+
94+
1. Create a class implementing `MalFunctionExtension` with **static** `@MALContextFunction` methods:
95+
96+
```java
97+
public class MyExtension implements MalFunctionExtension {
98+
@Override
99+
public String name() { return "myext"; }
100+
101+
@MALContextFunction
102+
public static SampleFamily transform(SampleFamily sf, double factor) {
103+
return sf.multiply(Double.valueOf(factor));
104+
}
105+
106+
@MALContextFunction
107+
public static SampleFamily filterTag(SampleFamily sf, String key, String value) {
108+
return sf.tagEqual(key, value);
109+
}
110+
}
111+
```
112+
113+
2. Register via SPI file `META-INF/services/org.apache.skywalking.oap.meter.analyzer.v2.spi.MalFunctionExtension`:
114+
```
115+
com.example.MyExtension
116+
```
117+
118+
3. Use in MAL scripts: `.myext::transform(2.0)`, `.myext::filterTag("env", "prod")`
119+
120+
### Method Requirements
121+
122+
- Methods **must** be `static` (non-static methods throw `IllegalArgumentException` at startup)
123+
- First parameter **must** be `SampleFamily` (auto-bound to the current chain value)
124+
- Return type **must** be `SampleFamily`
125+
- Additional parameters are matched from MAL arguments by type:
126+
127+
| Java Type | MAL Argument |
128+
|-----------|-------------|
129+
| `String` | String literal: `"value"` |
130+
| `double` / `int` | Number literal: `2.0`, `100` |
131+
| `List<String>` | String list: `["tag1", "tag2"]` |
132+
133+
### Compile-Time Validation
134+
135+
The compiler validates at expression compilation time:
136+
- Namespace exists in registry
137+
- Method exists in that namespace
138+
- Argument count matches (excluding the implicit `SampleFamily` first param)
139+
- Argument types are compatible with the method signature
140+
141+
### Generated Code
142+
143+
The compiler generates direct static method calls — no reflection or registry dispatch at runtime.
144+
145+
For `.myext::transform(2.0)`, the compiler generates:
146+
```java
147+
sf = com.example.MyExtension.transform(sf, 2.0);
148+
```
149+
150+
For zero-arg extensions like `.myext::noop()`:
151+
```java
152+
sf = com.example.MyExtension.noop(sf);
153+
```
154+
68155
## Javassist Constraints
69156

70157
- **No anonymous inner classes**: Javassist cannot compile `new Consumer() { ... }` or `new Function() { ... }` in method bodies.

oap-server/analyzer/meter-analyzer/src/main/antlr4/org/apache/skywalking/mal/rt/grammar/MALLexer.g4

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ L_BRACE: '{';
4444
R_BRACE: '}';
4545
SEMI: ';';
4646
COLON: ':';
47+
DOUBLE_COLON: '::';
4748
QUESTION: '?';
4849
ARROW: '->';
4950
ASSIGN: '=';

oap-server/analyzer/meter-analyzer/src/main/antlr4/org/apache/skywalking/mal/rt/grammar/MALParser.g4

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,8 @@ functionCall
7676
;
7777

7878
methodCall
79-
: IDENTIFIER L_PAREN argumentList? R_PAREN
79+
: IDENTIFIER DOUBLE_COLON IDENTIFIER L_PAREN argumentList? R_PAREN // extension: .ns::method()
80+
| IDENTIFIER L_PAREN argumentList? R_PAREN // built-in: .method()
8081
;
8182

8283
// ==================== Arguments ====================

0 commit comments

Comments
 (0)