Python GC at Instagram

I've found a series of articles by Instagram developers about the methods they use to support the web version written in Python and Django.

In the first part (https://engineering.instagram.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172) there was a story about disabling the garbage collector in Python via gc.set_threshold(0). gc.disable() practically did not work, because third-party libraries like to turn on the garbage collector (which, by the way, is almost unnecessary, because there is still reference counting, and the collector is only needed for cycles after overcoming the alocations minus dealotations threshold). The garbage collector led to Copy-on-Write for memory pages shared with the main process, which increased memory consumption.

In the second article (https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf), about a year after, the story continued. The application stopped coping with the load and memory consumption increased again. As a result, it was decided to move some of the objects to the old generation via gc.freeze() and turn on GC. The result exceeded expectations.

Telegram channel: https://t.me/sea_plus_plus