Memory use in long running processes is one of those problems that is a non-issue... until you've completely run out of memory. Now that you have hit the proverbial brick wall, how do you recover? In this talk, you'll learn about: memory profiling tools, utilizing Elasticsearch to track memory use on a live system, memory management of tiny objects, and other insights along the way.
At the core of your company's flagship app is a long running process that eventually becomes hungry for memory. You have to watch it like a hawk and run rolling restarts randomly throughout the day. It sure would be nice to know exactly what's happening to memory! In this talk, you'll learn about available Python memory profiling tools, using Elasticsearch to track per request memory use on a live system, memory pools and arenas for tiny object memory management, system memory versus process memory visible to a garbage collector, use of interprocess communication to manage your instrumentation, querying garbage collector for object inventory, memory allocation during package loading, and plain human perseverance. Also, we'll discuss the fact that you absolutely must get off Python 2.7.x for any long running processes.
Presently a Tech Lead in a SaaS company (OnShift, Inc.), Ilya has a wide span of software development experience. He has worked on everything from PC-based numerical controls for profile cutting and medical devices to medical web portals and labor management software. He has also had a great opportunity to teach software bootcamp, as well as mentor many interns and junior developers. He lives with his wonderful family and works in Cleveland, OH.