[python] add typeddict models-mode for Python HTTP client emitter#10439
[python] add typeddict models-mode for Python HTTP client emitter#10439iscai-msft wants to merge 54 commits into
models-mode for Python HTTP client emitter#10439Conversation
Add a new 'typeddict' value for the models-mode option that generates Python TypedDict classes instead of DPG model classes. Key features: - TypedDict classes with Required[T]/NotRequired[T] annotations - TypedDict inheritance for non-discriminated models - Discriminated models: Union of leaf TypedDicts, no abstract base class - Input-only: operations accept TypedDict input, return dict output - Wire names used as TypedDict keys - _model_base.py still generated for serialization utilities Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
commit: |
|
All changed packages have been documented.
Show changes
|
|
You can try these changes here
|
…ypespec into python/addTypedDict
…hon/addTypedDict
- TypedDictModelType returns 'JSON' for response type annotations - Response.type_annotation/docstring passes is_response=True - Typeddict deserialization uses response.json() directly - Removed NotRequired from TypedDictModelSerializer (total=False handles it) - Updated mock API tests to verify JSON dict responses Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add client/naming typeddict variant to regenerate-common.ts - Create test_client_naming_typeddict.py with 11 tests verifying TypedDict uses wire names (defaultName, wireName) not client names - Tests cover: ClientNameModel, LanguageClientNameModel, ClientNameAndJsonEncodedNameModel, ClientModel, PythonModel Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TypedDict is already JSON, so the MutableMapping[str, Any] overload is unnecessary. Only keep TypedDict model + IO[bytes] overloads. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Typeddict mode uses response.json() directly, so _deserialize is never called. Skip importing it to avoid W0611 unused-import lint warning. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove unused MutableMapping/Any imports from TypedDictModelType.imports() - Skip _deserialize import in paging_operation.py for typeddict mode Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ypespec into python/addTypedDict
TypedDictModelType returns 'JSON' for response type annotations but never defined the JSON = MutableMapping[str, Any] type alias, causing NameError at runtime. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
34e0cda to
95db199
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TypedDicts don't differentiate input/output. The parent classes don't use is_response either, so the pop overrides were just noise. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The typeddict test should pass plain dicts with wire names, not model class constructors which expect client names. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ict-only models - Skip importing types module from itself when model is in same namespace - Skip _deserialize import when response type is typed-dict-only (uses response.json() instead) - Fix paging operation _deserialize import for typed-dict-only item types Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Internal models in types.py should use bare names like all other models, since they're imported by name under TYPE_CHECKING. Also fix import path for internal models to use .models._models instead of .models. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…anges into regenerate.ts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds alpha support for generating/using Python TypedDict-style model shapes in the HTTP client Python emitter, alongside restructuring named-union output into a dedicated _unions.py module and updating tests/regeneration config to cover the new behaviors.
Changes:
- Introduces
models-mode=typeddict,types.pyTypedDict generation, andtyped-dict-only-modelsbehavior hooks in codegen. - Moves named union aliases out of the old types file flow into a new
_unions.pyfile with a dedicated serializer/template. - Adds unit + mock API tests and updates regeneration script/config to produce typeddict variants.
Reviewed changes
Copilot reviewed 28 out of 28 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/http-client-python/tests/unit/test_typeddict.py | Adds unit coverage for types.py TypedDict output and typed-dict-only behaviors. |
| packages/http-client-python/tests/mock_api/shared/test_typetest_model_usage_typeddictonly.py | Adds mock API validation for typed-dict-only input/output behavior. |
| packages/http-client-python/tests/mock_api/shared/test_typetest_model_inheritance_single_discriminator_typeddict.py | Adds mock API coverage for discriminated models in typeddict scenario. |
| packages/http-client-python/tests/mock_api/shared/test_typetest_model_inheritance_not_discriminated_typeddict.py | Adds mock API coverage for non-discriminated model inheritance in typeddict scenario. |
| packages/http-client-python/tests/mock_api/azure/test_client_naming_typeddict.py | Adds mock API coverage to ensure wire-name keys are used for TypedDict interactions. |
| packages/http-client-python/generator/pygen/codegen/templates/unions.py.jinja2 | New template for _unions.py named union aliases. |
| packages/http-client-python/generator/pygen/codegen/templates/types.py.jinja2 | Reworked types.py template to emit TypedDicts and discriminated unions. |
| packages/http-client-python/generator/pygen/codegen/templates/model_typeddict.py.jinja2 | Adds a (currently unused) TypedDict model template. |
| packages/http-client-python/generator/pygen/codegen/templates/model_container.py.jinja2 | Routes typeddict base models through the shared model template include. |
| packages/http-client-python/generator/pygen/codegen/serializers/unions_serializer.py | New serializer to render _unions.py with correct imports. |
| packages/http-client-python/generator/pygen/codegen/serializers/types_serializer.py | Implements TypedDict rendering/import logic for types.py including discriminated-base unions. |
| packages/http-client-python/generator/pygen/codegen/serializers/model_serializer.py | Adds TypedDictModelSerializer for generating TypedDict-shaped “models” output. |
| packages/http-client-python/generator/pygen/codegen/serializers/builder_serializer.py | Skips deserialization and returns raw JSON for typed-dict-only responses/items. |
| packages/http-client-python/generator/pygen/codegen/serializers/init.py | Writes types.py per namespace; writes _unions.py at top-level; filters typed-dict-only models out of model classes. |
| packages/http-client-python/generator/pygen/codegen/models/utils.py | Adds NamespaceType.UNIONS_FILE to support _unions.py import behavior. |
| packages/http-client-python/generator/pygen/codegen/models/response.py | Switches named-union imports to _unions and threads is_response flag through annotation/docstring paths. |
| packages/http-client-python/generator/pygen/codegen/models/property.py | Avoids importing rest_field/rest_discriminator when rendering properties for types.py. |
| packages/http-client-python/generator/pygen/codegen/models/parameter.py | Switches named-union imports to _unions. |
| packages/http-client-python/generator/pygen/codegen/models/paging_operation.py | Avoids importing _deserialize when paging item type is typed-dict-only. |
| packages/http-client-python/generator/pygen/codegen/models/operation.py | Avoids importing _deserialize when a response type is typed-dict-only. |
| packages/http-client-python/generator/pygen/codegen/models/model_type.py | Adds is_typed_dict_only and adjusts typing/import logic for types.py and typed-dict-only references. |
| packages/http-client-python/generator/pygen/codegen/models/enum_type.py | Adjusts enum type annotations/imports to behave better within types.py. |
| packages/http-client-python/generator/pygen/codegen/models/combined_type.py | Redirects named-union references/imports to _unions. |
| packages/http-client-python/generator/pygen/codegen/models/code_model.py | Excludes typed-dict-only models from public_model_types. |
| packages/http-client-python/generator/pygen/init.py | Extends option validation to allow models-mode=typeddict and normalizes typed-dict-only-models. |
| packages/http-client-python/eng/scripts/ci/regenerate.ts | Inlines regeneration-common logic and adds/updates emitter options for typeddict-related generated packages. |
| packages/http-client-python/eng/scripts/ci/regenerate-common.ts | Removes shared regeneration helper module (logic inlined into regenerate.ts). |
| .chronus/changes/python-addTypedDict-2026-3-21-17-47-3.md | Adds Chronus entry describing the feature. |
…hon/addTypedDict
…of literals instead of enums when td only
- Fix cross-namespace parent imports in types.py to import from types module - Add isidentifier() check for wire names (not just iskeyword) for functional TypedDict form - Use __doc__ assignment for functional-form TypedDict docstrings - Add missing op_tools import in model_typeddict.py.jinja2 - Fix test docstring to match actual behavior - Update changelog entry with correct file names (types.py, _unions.py) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix cross-namespace parent imports in types_serializer.py and model_type.py to use get_relative_import_path with module_name parameter instead of naive string concatenation - Remove unused model_typeddict.py.jinja2 template (dead code) - Fix misleading test name: rename test_models_mode_dpg_no_typeddict_models to test_models_mode_dpg_typeddict_models_included Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| : property.serializationOptions?.json?.name) ?? property.name, | ||
| type: getType(context, sourceType), | ||
| optional: property.optional, | ||
| nullable: isNullable, |
There was a problem hiding this comment.
we are conflating nullable and optional. this is one step to help with the conflation, since it's extra important in typeddicts to not type things as optional when we mean nullable
Revert to the original no-sdk-clients check without tying the exemption to models-mode=typeddict. Models-only generation should not be gated on a specific models-mode. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| def type_annotation(self, **kwargs: Any) -> str: | ||
| if self.name: | ||
| return f'"_types.{self.name}"' | ||
| return f'"_unions.{self.name}"' |
There was a problem hiding this comment.
moved the existing union types to a unions.py file, keeping types.py purely for typeddict generation. So now, really instead of using JSON, we will be able to use the typeddict representations instead for those overloads. If we want to as well, we can not generate models and only generate typeddicts as well
| """ | ||
| if self.code_model.options["models-mode"]: | ||
| if self.is_typeddict_mode: | ||
| # In typeddict mode, enums are Literal aliases defined in types.py |
| self.cross_language_definition_id: Optional[str] = self.yaml_data.get("crossLanguageDefinitionId") | ||
| self.usage: int = self.yaml_data.get("usage", UsageFlags.Input.value | UsageFlags.Output.value) | ||
| self.client_namespace: str = self.yaml_data.get("clientNamespace", code_model.namespace) | ||
| self.is_typed_dict_only: bool = ( |
There was a problem hiding this comment.
we want to be able to flag individual models as typeddict only as well. we will need to add a decorator in tcgc in order to support this
| /* eslint-disable no-console */ | ||
| /** | ||
| * Shared helpers, types, constants, and data tables used by `regenerate.ts`. | ||
| * | ||
| * This file is meant to be **byte-identical** between this package and the | ||
| * upstream `@typespec/http-client-python`. typespec-python syncs it from | ||
| * <repo-root>/core/packages/http-client-python/eng/scripts/ci/regenerate-common.ts | ||
| * via `pnpm sync`. | ||
| * | ||
| * Per-repo divergence (paths, emitter name, single-phase vs two-phase | ||
| * orchestration, argv/help text) lives in each repo's own `regenerate.ts`, | ||
| * which builds a `RegenerateContext` and feeds it into the helpers exported | ||
| * from this module. | ||
| */ |
There was a problem hiding this comment.
This is the 2nd time that the regenerate-common.ts is to be deleted...
As the annotation declares, the intension of this file is to sync similar change with easy script to typespec-python so that we don't need to manually maintain test for 2 repos. Here is my PR Azure/typespec-azure#4399 so pls keep this file. @iscai-msft
There was a problem hiding this comment.
oh ok, my bad, I didn't realize that, will add it back
| // Python convert all the type of file part to FileType so clear these models' usage so that they won't be generated | ||
| addDisableGenerationMap(context, property.type); | ||
| } | ||
| const isNullable = !isMultipartFileInput && sourceType.kind === "nullable"; |
There was a problem hiding this comment.
you're right, I had forgotten about that. We have added discriminator to the things we are allowed to generate, but for the other ones, now that we've decided to always generate TD, @johanste should we just not generate TD types for these cases and rely on JSON in the overload?
Replace the JSON (MutableMapping) overload with a TypedDict reference from types.py for model input parameters. This gives users proper type-checking and IDE completion when passing dict literals. Changes: - preprocess: insert TypedDict model reference instead of any-object for both model and list/dict body params with DPG model elements - TypedDictModelType: add type_annotation, imports, docstring_type, and instance_check_template overrides - build_type: handle base='typeddict' to create TypedDictModelType - code_model: exclude typeddict refs from model_types (annotation-only) Generated operations now show: @overload def op(body: _models.X, ...) -> ... @overload def op(body: types.X, ...) -> ... @overload def op(body: IO[bytes], ...) -> ... JSON overloads are preserved for spread bodies (base=json) where no model class exists. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file is shared with typespec-python via pnpm sync and should not be deleted. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

fixes #8800
Add a new 'typeddict' value for the
models-modeoption that generates PythonTypedDictclasses instead of DPG model classes. Key features:_model_base.pystill generated for serialization utilities