Repository navigation
improve DX for type inference and format detection - #7531
Merged
Merged
Conversation
nwt
reviewed
Oct 26, 2026
| // FileType returns the fused type of the input at path or nil if the input | ||
| // is dynamically typed. See anyio.Source.Type. | ||
| func (e *Environment) FileType(ctx context.Context, sctx *super.Context, path, format string) (super.Type, error) { | ||
| if e.Dynamic || e.sources == nil { |
Collaborator
There was a problem hiding this comment.
This should check for e.Stdin!=nil.
Collaborator
Author
There was a problem hiding this comment.
As we discussed...
nwt
approved these changes
Oct 26, 2026
This commit further improves upon the recent changes to auto-detection and type inference. Two problems were apparent with the previous approach: "from http://... | ..." failed with unknown format and "super -static -i auto -c '...' -" failed with cannot get type of non-seekable. This commit solves these problems by reading the inputs into memory (and reusing the memory buffering across type inference and query scan). The buffering occurs in a new entity anyio.Source and a sources table maps URIs of inputs to a Source. -static now forces the input to be read and buffered when the input doesn't have native type support. In the future, we will have better HTTP support for range requests and we can devise a better approach for type and format inference for HTTP. For now, reading the whole body has nice DX at the cost of bad perf for large and otherwise seekable HTTP endpoints.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit further improves upon the recent changes to auto-detection and type inference. Two problems were apparent with the previous approach: "from http://... | ..." failed with unknown format and "super -static -i auto -c '...' -" failed with cannot get type of non-seekable.
This commit solves these problems by reading the inputs into memory (and reusing the memory buffering across type inference and query scan). The buffering occurs in a new entity anyio.Source and a sources table maps URIs of inputs to a Source.
-static now forces the input to be read and buffered when the input doesn't have native type support.
In the future, we will have better HTTP support for range requests and we can devise a better approach for type and format inference for HTTP. For now, reading the whole body has nice DX at the cost of bad perf for large and otherwise seekable HTTP endpoints.