Skip to content

gh-146256: Add --jsonl collector to the profiling.sampling#146257

Draft
maurycy wants to merge 35 commits intopython:mainfrom
maurycy:tachyon-ndjson-kolektor
Draft

gh-146256: Add --jsonl collector to the profiling.sampling#146257
maurycy wants to merge 35 commits intopython:mainfrom
maurycy:tachyon-ndjson-kolektor

Conversation

@maurycy
Copy link
Copy Markdown
Contributor

@maurycy maurycy commented Mar 21, 2026

This PR adds --jsonl discussed in #146256.

It's a draft - it does not contain tests yet. The class is below 2**8 lines of code and does not touch existing profiling.sampling code, so I took a leap. I'm already very uncomfortable with creating a PR for an issue that wasn't discussed. I don't want to clutter it with tests, until the format is solidified.

For example:

sudo -E \
  uv run \
    --python /Users/maurycy/src/github.com/maurycy/cpython/python.exe \
      python \
        -m profiling.sampling \
           run \
             --jsonl \
             -o /tmp/profile.jsonl /tmp/hello_world.py

Where /tmp/hello_world.py could be:

import time
print("Hello, World!")
time.sleep(0.1)

Given the similarity, the class just inherits from StackTraceCollector. (I think this is a good hint that the collectors should be separate from renderers.)

Visual Studio Code Extension

For the purpose of demonstrating the --jsonl usefulness, I have vibe-coded (with Claude Code) a simple VSCode Extension (only that) that leverages this play profile and displays it in the editor:

image

I think that, once we have --stream it will be much more exciting.

Apart from headless profilers: updating the real-time hot spots from the production in VSCode, or, well, making agents' life easier.

You can fetch the vibe-coded VSCode Extension here (no guarantees):

Or:

mkdir -p ~/.vscode/extensions/profiling-heatmap                                                                                                                                                                                   
curl -sL 'https://gist.githubusercontent.com/maurycy/50a80586636a90216efc86065bbfd6de/raw/719e46f4f086c819edbbee7b20c45c41eae15e8b/extension.js' -o ~/.vscode/extensions/profiling-heatmap/extension.js                           
curl -sL 'https://gist.githubusercontent.com/maurycy/fa4c3acd9e8e681a609bddc9ad04c4ae/raw/1edd3c2716a4a9e55aba8ed94850fd70f61fa606/package.json' -o ~/.vscode/extensions/profiling-heatmap/package.json                           
echo "Restart VSCode, then: Cmd+Shift+P -> 'Profiling: Load JSONL Profile'"   

Please do not forget about removing ~/.vscode/extensions/profiling-heatmap/ after tests.

@maurycy maurycy requested a review from pablogsal as a code owner March 21, 2026 15:59
@maurycy maurycy marked this pull request as draft March 21, 2026 16:00
@maurycy maurycy changed the title gh-146256: Add --ndjson flag to the profiling.sampling gh-146256: Add --jsonl flag to the profiling.sampling Mar 21, 2026
@maurycy maurycy changed the title gh-146256: Add --jsonl flag to the profiling.sampling gh-146256: Add --jsonl collector to the profiling.sampling Mar 23, 2026
maurycy added 8 commits March 30, 2026 19:22
…ktor

* upstream/main: (119 commits)
  pythongh-144270: Make SubElement parent and tag positional-only (pythonGH-144845)
  pythongh-146558: JIT optimize dict access for objects with known hash (python#146559)
  pythongh-139922: always run MSVC 64-bit tail-calling CI (pythonGH-146570)
  pythongh-126835: Fix _PY_IS_SMALL_INT() macro (python#146631)
  pythongh-146587: fix type slot assignment incase of multiple slots for same name (python#146593)
  pythongh-138122: Add differential flame graph (python#145785)
  pythongh-146416: Emscripten: Improve standard stream handling in node_entry.mjs (python#146417)
  pythongh-146444: Don't package as part of iOS 'build hosts' target (python#146628)
  pythongh-138850: Add --disable-epoll to configure (pythonGH-145768)
  pythongh-146444: Make Platforms/Apple/ compatible with Python 3.9 (python#146624)
  pythongh-138577: Fix keyboard shortcuts in getpass with echo_char (python#141597)
  pythongh-146556: Fix infinite loop in annotationlib.get_annotations() on circular __wrapped__ (python#146557)
  pythongh-146579: _zstd: Fix decompression options dict error message (python#146577)
  pythongh-146083: Upgrade bundled Expat to 2.7.5 (python#146085)
  pythongh-146080: fix a crash in SNI callbacks when the SSL object is gone (python#146573)
  pythongh-146090: fix memory management of internal `sqlite3` callback contexts (python#146569)
  pythongh-145876: Do not mask KeyErrors raised during dictionary unpacking in call (pythonGH-146472)
  pythongh-146004: fix test_args_from_interpreter_flags on windows (python#146580)
  pythongh-139003: Use frozenset for module level attributes in _pyrepl.utils (python#139004)
  pythonGH-146527: Add more data to GC statistics and add it to PyDebugOffsets (python#146532)
  ...
@pablogsal
Copy link
Copy Markdown
Member

@ivonastojanovic can you take a look?

@maurycy
Copy link
Copy Markdown
Contributor Author

maurycy commented Mar 31, 2026

@ivonastojanovic @pablogsal Thank you.

Please note that I've started adding test coverage, so it might be worth waiting a day with a proper review (it's already interesting: confused myself with skip_idle), if you find the direction promising.

I will mark it as Ready for review immediately.

Perhaps #146256 and #145464 are the best places to discuss the format and the ideas.

maurycy added 9 commits March 31, 2026 12:25
* main:
  pythongh-145458: use `self.skip_idle` consistently in the tachyon profiler (python#145459)
  pythongh-146615: Fix format specifiers in Objects/ directory (pythonGH-146620)
  pythongh-146615: Fix format specifiers in Python/ directory (pythonGH-146619)
  pythongh-146615: Fix format specifiers in test cextensions (pythonGH-146618)
  pythongh-146615: Fix format specifiers in extension modules (pythonGH-146617)
  pythongh-146615: Fix crash in __get__() for METH_METHOD descriptors with invalid type argument (pythonGH-146634)
  pythongh-146376: Reduce timeout in Emscripten GHA workflow (python#146378)
  pythongh-146442: Fix various bugs in compiler pipeline (python#146443)
  pythongh-146238: Support half-floats in the array module (python#146242)
  pythongh-145056: Add support for merging collections.UserDict and frozendict (pythonGH-146465)
  pythongh-145056: Fix merging of collections.OrderedDict and frozendict (pythonGH-146466)
  pythongh-139633: Run netrc file permission check only once per parse (pythonGH-139634)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants