"Compacting the Uncompactable" by Bobby Powers

Channel:Strange Loop Conference · Length:40:18 · Published:2019-09-14 · Views:29271

Watch on YouTube

Summary

Bobby Powers presents Mesh, a drop-in malloc replacement that eliminates memory fragmentation in unmodified C/C++ programs by compacting memory. The talk covers how it works despite C's fixed object addresses, and its benefits for programs like Redis.

Why HN recommends it

Recommended as a great talk on solving fragmentation for C/C++ and runtimes like Ruby and Python without code changes.

There was a great talk at Strange Loop about a drop-in malloc replacement which compacts. — HN, 2022

C, C++, Rust, Go & systems languages · Software performance & optimization

Subtitles (excerpt)

so I'm Bobby and I'm gonna talk to you today about compacting the uncompact able than providing memory compaction for C and C++ programs and I want to start out today by by starting with a story and this story is the impossible dreams there's an older version of this story that stars Don Quixote the Man of La Mancha and in this this older version of the story he wants either reconquer or reintroduced chivalry to all of Spain now today though the version we're going to talk about features the Malek of La Mancha and rather than reality or reconquer all of Spain we want to allocate all of Spain so in this story here we have Spain and when our hero gets Malek requests he looks around on a map and they find you know three chunks of Spain and and hands them back to whoever requested it and you know at the beginning of the story this this this process is pretty easy we can just sort of march on down the map here allocating the next free chunk we find however over time some of these old areas that we had allocated are no longer needed and our freed and so so so eventually we end up in a situation where we have little chunks of free space spread throughout the map here and we get a new request we get a new large

On Hacker News

There’s a great paper from the group of Emery Berger, where they merge memory pages that become sparse. A kind of garbage collection for C and C++ languages. There’s also a good presentation about it on youtube. [LINK] The presentation: [VIDEO] — superjan, 2025-01
There was a great talk at Strange Loop about a drop-in malloc replacement which compacts. Apparently it actually led to memory usage improvements in industrial projects like Redis: [VIDEO] — rtfeldman, 2022-07
There's an interesting talk [0] from Bobby Powers on how to make a malloc() and free() that perform compaction, to be able to release memory back to the operating system more frequently (and improve cache hit rates, etc.). But this isn't standard at all yet. [1]: [VIDEO] — remexre, 2020-12
> but malloc/free based systems also have a relatively high memory overhead due to fragmentation In case you haven't seen it, there's been some interesting progress on that front: [VIDEO] — ssokolow, 2022-06

Related videos