a Twitter thread from @pyblogsal

1.

Python 3.12 will add support for the Linux perf profiler! 🔥🔥 Perf is one of the most powerful and performant profilers for Linux that allows getting a ridiculous amount of information such as CPU counters, cache misses, context switching and much more. 🧵👇

2.

This means that since Python 3.12, perf will be able to "see" python calls and show them in the stack. This allows you to generate very easily these kinds of flame graphs that have both C/C++/Rust code and Python code.

3.

This is super powerful because there are **a lot** of Python libraries that call native code 👾(C/C++/rust...) underneath and this will allow you to see how much time is spent on actual Python and how much in native code.🐍

4.

This will help you solve questions like "should I improve my Python code or should I change the way I am using numpy or pandas?"🤔 or "How much time is being spent in TensorFlow and how much time is being spent on code preparing the model and marshalling the results back?"

5.

Not only that but you can also correlate that with all the things perf exposes to answer things like "how cache efficient is this numpy code?", "how much is the GIL hurting my code here?", "how many context switches are happening in this region over here" ...

6.

Of course, perf allows other outputs and many libraries know how to display results that come from perf. For instance, this is a tree-like output that perf can generate:

7.

And if you are not interested in native code, you can just filter it and use it as a pure Python profiler. For instance, here is a pure-python flame graph:

8.

Perf not only is a ridiculously fast profiler (especially when it unwinds from kernel space) but also allows attaching to processes, profiling ALL processes in a machine, gathering hardware counters, seeing into kernel space and much more. Is the ultimate profiler for Linux! 🤯

9.

You can read more here:

https://docs.python.org/pl/dev/howto/perf_profiling.html

Read this thread on Twitter