Skip to content

[python] add typeddict models-mode for Python HTTP client emitter#10439

Open
iscai-msft wants to merge 54 commits into
microsoft:mainfrom
iscai-msft:python/addTypedDict
Open

[python] add typeddict models-mode for Python HTTP client emitter#10439
iscai-msft wants to merge 54 commits into
microsoft:mainfrom
iscai-msft:python/addTypedDict

Conversation

@iscai-msft
Copy link
Copy Markdown
Member

@iscai-msft iscai-msft commented Apr 21, 2026

fixes #8800

Add a new 'typeddict' value for the models-mode option that generates Python TypedDict classes instead of DPG model classes. Key features:

  • TypedDict classes with Required[T]/NotRequired[T] annotations
  • TypedDict inheritance for non-discriminated models
  • Discriminated models: Union of leaf TypedDicts, no abstract base class
  • Input-only: operations accept TypedDict input, return dict output
  • Wire names used as TypedDict keys
  • _model_base.py still generated for serialization utilities

Add a new 'typeddict' value for the models-mode option that generates
Python TypedDict classes instead of DPG model classes. Key features:

- TypedDict classes with Required[T]/NotRequired[T] annotations
- TypedDict inheritance for non-discriminated models
- Discriminated models: Union of leaf TypedDicts, no abstract base class
- Input-only: operations accept TypedDict input, return dict output
- Wire names used as TypedDict keys
- _model_base.py still generated for serialization utilities

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@microsoft-github-policy-service microsoft-github-policy-service Bot added the emitter:client:python Issue for the Python client emitter: @typespec/http-client-python label Apr 21, 2026
@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new Bot commented Apr 21, 2026

Open in StackBlitz

npm i https://pkg.pr.new/@typespec/http-client-python@10439

commit: 3cea12c

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 21, 2026

All changed packages have been documented.

  • @typespec/http-client-python
Show changes

@typespec/http-client-python - feature ✏️

[python] Always generate TypedDict typing hints for input models in the types.py file, and named union aliases in the _unions.py file

@azure-sdk
Copy link
Copy Markdown
Collaborator

azure-sdk commented Apr 21, 2026

You can try these changes here

🛝 Playground 🌐 Website 🛝 VSCode Extension

iscai-msft and others added 12 commits April 21, 2026 14:37
- TypedDictModelType returns 'JSON' for response type annotations
- Response.type_annotation/docstring passes is_response=True
- Typeddict deserialization uses response.json() directly
- Removed NotRequired from TypedDictModelSerializer (total=False handles it)
- Updated mock API tests to verify JSON dict responses

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add client/naming typeddict variant to regenerate-common.ts
- Create test_client_naming_typeddict.py with 11 tests verifying TypedDict
  uses wire names (defaultName, wireName) not client names
- Tests cover: ClientNameModel, LanguageClientNameModel,
  ClientNameAndJsonEncodedNameModel, ClientModel, PythonModel

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TypedDict is already JSON, so the MutableMapping[str, Any] overload
is unnecessary. Only keep TypedDict model + IO[bytes] overloads.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Typeddict mode uses response.json() directly, so _deserialize is never
called. Skip importing it to avoid W0611 unused-import lint warning.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove unused MutableMapping/Any imports from TypedDictModelType.imports()
- Skip _deserialize import in paging_operation.py for typeddict mode

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TypedDictModelType returns 'JSON' for response type annotations but
never defined the JSON = MutableMapping[str, Any] type alias, causing
NameError at runtime.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@iscai-msft iscai-msft force-pushed the python/addTypedDict branch from 34e0cda to 95db199 Compare April 29, 2026 15:39
iscai-msft and others added 6 commits May 12, 2026 12:55
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TypedDicts don't differentiate input/output. The parent classes don't
use is_response either, so the pop overrides were just noise.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The typeddict test should pass plain dicts with wire names, not
model class constructors which expect client names.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ict-only models

- Skip importing types module from itself when model is in same namespace
- Skip _deserialize import when response type is typed-dict-only (uses response.json() instead)
- Fix paging operation _deserialize import for typed-dict-only item types

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Internal models in types.py should use bare names like all other models,
since they're imported by name under TYPE_CHECKING. Also fix import path
for internal models to use .models._models instead of .models.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…anges into regenerate.ts

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@msyyc msyyc requested review from Copilot and removed request for swathipil May 25, 2026 04:54
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds alpha support for generating/using Python TypedDict-style model shapes in the HTTP client Python emitter, alongside restructuring named-union output into a dedicated _unions.py module and updating tests/regeneration config to cover the new behaviors.

Changes:

  • Introduces models-mode=typeddict, types.py TypedDict generation, and typed-dict-only-models behavior hooks in codegen.
  • Moves named union aliases out of the old types file flow into a new _unions.py file with a dedicated serializer/template.
  • Adds unit + mock API tests and updates regeneration script/config to produce typeddict variants.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
packages/http-client-python/tests/unit/test_typeddict.py Adds unit coverage for types.py TypedDict output and typed-dict-only behaviors.
packages/http-client-python/tests/mock_api/shared/test_typetest_model_usage_typeddictonly.py Adds mock API validation for typed-dict-only input/output behavior.
packages/http-client-python/tests/mock_api/shared/test_typetest_model_inheritance_single_discriminator_typeddict.py Adds mock API coverage for discriminated models in typeddict scenario.
packages/http-client-python/tests/mock_api/shared/test_typetest_model_inheritance_not_discriminated_typeddict.py Adds mock API coverage for non-discriminated model inheritance in typeddict scenario.
packages/http-client-python/tests/mock_api/azure/test_client_naming_typeddict.py Adds mock API coverage to ensure wire-name keys are used for TypedDict interactions.
packages/http-client-python/generator/pygen/codegen/templates/unions.py.jinja2 New template for _unions.py named union aliases.
packages/http-client-python/generator/pygen/codegen/templates/types.py.jinja2 Reworked types.py template to emit TypedDicts and discriminated unions.
packages/http-client-python/generator/pygen/codegen/templates/model_typeddict.py.jinja2 Adds a (currently unused) TypedDict model template.
packages/http-client-python/generator/pygen/codegen/templates/model_container.py.jinja2 Routes typeddict base models through the shared model template include.
packages/http-client-python/generator/pygen/codegen/serializers/unions_serializer.py New serializer to render _unions.py with correct imports.
packages/http-client-python/generator/pygen/codegen/serializers/types_serializer.py Implements TypedDict rendering/import logic for types.py including discriminated-base unions.
packages/http-client-python/generator/pygen/codegen/serializers/model_serializer.py Adds TypedDictModelSerializer for generating TypedDict-shaped “models” output.
packages/http-client-python/generator/pygen/codegen/serializers/builder_serializer.py Skips deserialization and returns raw JSON for typed-dict-only responses/items.
packages/http-client-python/generator/pygen/codegen/serializers/init.py Writes types.py per namespace; writes _unions.py at top-level; filters typed-dict-only models out of model classes.
packages/http-client-python/generator/pygen/codegen/models/utils.py Adds NamespaceType.UNIONS_FILE to support _unions.py import behavior.
packages/http-client-python/generator/pygen/codegen/models/response.py Switches named-union imports to _unions and threads is_response flag through annotation/docstring paths.
packages/http-client-python/generator/pygen/codegen/models/property.py Avoids importing rest_field/rest_discriminator when rendering properties for types.py.
packages/http-client-python/generator/pygen/codegen/models/parameter.py Switches named-union imports to _unions.
packages/http-client-python/generator/pygen/codegen/models/paging_operation.py Avoids importing _deserialize when paging item type is typed-dict-only.
packages/http-client-python/generator/pygen/codegen/models/operation.py Avoids importing _deserialize when a response type is typed-dict-only.
packages/http-client-python/generator/pygen/codegen/models/model_type.py Adds is_typed_dict_only and adjusts typing/import logic for types.py and typed-dict-only references.
packages/http-client-python/generator/pygen/codegen/models/enum_type.py Adjusts enum type annotations/imports to behave better within types.py.
packages/http-client-python/generator/pygen/codegen/models/combined_type.py Redirects named-union references/imports to _unions.
packages/http-client-python/generator/pygen/codegen/models/code_model.py Excludes typed-dict-only models from public_model_types.
packages/http-client-python/generator/pygen/init.py Extends option validation to allow models-mode=typeddict and normalizes typed-dict-only-models.
packages/http-client-python/eng/scripts/ci/regenerate.ts Inlines regeneration-common logic and adds/updates emitter options for typeddict-related generated packages.
packages/http-client-python/eng/scripts/ci/regenerate-common.ts Removes shared regeneration helper module (logic inlined into regenerate.ts).
.chronus/changes/python-addTypedDict-2026-3-21-17-47-3.md Adds Chronus entry describing the feature.

Comment thread packages/http-client-python/generator/pygen/codegen/models/model_type.py Outdated
Comment thread packages/http-client-python/tests/unit/test_typeddict.py Outdated
Comment thread .chronus/changes/python-addTypedDict-2026-3-21-17-47-3.md Outdated
iscai-msft and others added 7 commits May 26, 2026 13:51
- Fix cross-namespace parent imports in types.py to import from types module
- Add isidentifier() check for wire names (not just iskeyword) for functional TypedDict form
- Use __doc__ assignment for functional-form TypedDict docstrings
- Add missing op_tools import in model_typeddict.py.jinja2
- Fix test docstring to match actual behavior
- Update changelog entry with correct file names (types.py, _unions.py)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@iscai-msft iscai-msft requested a review from l0lawrence as a code owner June 1, 2026 16:26
- Fix cross-namespace parent imports in types_serializer.py and
  model_type.py to use get_relative_import_path with module_name
  parameter instead of naive string concatenation
- Remove unused model_typeddict.py.jinja2 template (dead code)
- Fix misleading test name: rename test_models_mode_dpg_no_typeddict_models
  to test_models_mode_dpg_typeddict_models_included

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
: property.serializationOptions?.json?.name) ?? property.name,
type: getType(context, sourceType),
optional: property.optional,
nullable: isNullable,
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are conflating nullable and optional. this is one step to help with the conflation, since it's extra important in typeddicts to not type things as optional when we mean nullable

Revert to the original no-sdk-clients check without tying the
exemption to models-mode=typeddict. Models-only generation should
not be gated on a specific models-mode.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
def type_annotation(self, **kwargs: Any) -> str:
if self.name:
return f'"_types.{self.name}"'
return f'"_unions.{self.name}"'
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved the existing union types to a unions.py file, keeping types.py purely for typeddict generation. So now, really instead of using JSON, we will be able to use the typeddict representations instead for those overloads. If we want to as well, we can not generate models and only generate typeddicts as well

"""
if self.code_model.options["models-mode"]:
if self.is_typeddict_mode:
# In typeddict mode, enums are Literal aliases defined in types.py
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chatted with @johanste and in TD only mode, we don't want to generate enums as enum types, but instead as unions of literals. @johanste is this something we want to do everywhere?

self.cross_language_definition_id: Optional[str] = self.yaml_data.get("crossLanguageDefinitionId")
self.usage: int = self.yaml_data.get("usage", UsageFlags.Input.value | UsageFlags.Output.value)
self.client_namespace: str = self.yaml_data.get("clientNamespace", code_model.namespace)
self.is_typed_dict_only: bool = (
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we want to be able to flag individual models as typeddict only as well. we will need to add a decorator in tcgc in order to support this

Comment on lines -1 to -14
/* eslint-disable no-console */
/**
* Shared helpers, types, constants, and data tables used by `regenerate.ts`.
*
* This file is meant to be **byte-identical** between this package and the
* upstream `@typespec/http-client-python`. typespec-python syncs it from
* <repo-root>/core/packages/http-client-python/eng/scripts/ci/regenerate-common.ts
* via `pnpm sync`.
*
* Per-repo divergence (paths, emitter name, single-phase vs two-phase
* orchestration, argv/help text) lives in each repo's own `regenerate.ts`,
* which builds a `RegenerateContext` and feeds it into the helpers exported
* from this module.
*/
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the 2nd time that the regenerate-common.ts is to be deleted...
As the annotation declares, the intension of this file is to sync similar change with easy script to typespec-python so that we don't need to manually maintain test for 2 repos. Here is my PR Azure/typespec-azure#4399 so pls keep this file. @iscai-msft

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh ok, my bad, I didn't realize that, will add it back

// Python convert all the type of file part to FileType so clear these models' usage so that they won't be generated
addDisableGenerationMap(context, property.type);
}
const isNullable = !isMultipartFileInput && sourceType.kind === "nullable";
Copy link
Copy Markdown
Contributor

@msyyc msyyc Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to issue description, we should refuse generation for unsupported type. Then I understand we should add check in emitter to scan all types and report error diagnostics if there is unsupported type:

Image

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're right, I had forgotten about that. We have added discriminator to the things we are allowed to generate, but for the other ones, now that we've decided to always generate TD, @johanste should we just not generate TD types for these cases and rely on JSON in the overload?

iscai-msft and others added 3 commits June 2, 2026 13:48
Replace the JSON (MutableMapping) overload with a TypedDict reference
from types.py for model input parameters. This gives users proper
type-checking and IDE completion when passing dict literals.

Changes:
- preprocess: insert TypedDict model reference instead of any-object
  for both model and list/dict body params with DPG model elements
- TypedDictModelType: add type_annotation, imports, docstring_type,
  and instance_check_template overrides
- build_type: handle base='typeddict' to create TypedDictModelType
- code_model: exclude typeddict refs from model_types (annotation-only)

Generated operations now show:
  @overload def op(body: _models.X, ...) -> ...
  @overload def op(body: types.X, ...) -> ...
  @overload def op(body: IO[bytes], ...) -> ...

JSON overloads are preserved for spread bodies (base=json) where
no model class exists.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file is shared with typespec-python via pnpm sync and
should not be deleted.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

emitter:client:python Issue for the Python client emitter: @typespec/http-client-python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Python] Add alpha TypedDict support

4 participants