TechTown - Most Malleable Memory Management Method

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 1/110 Most
Malleable Memory Management Method Björn Fahller

Malleable Memory Management Method

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 6/110 The
heap Allows you to... – Allocate objects of any size and alignment – Allocate and deallocate from any thread • Even allocate in one thread and deallocate in another – Let objects lifetimes vary wildly

heap Allows you to... – Allocate objects of any size and alignment – Allocate and deallocate from any thread • Even allocate in one thread and deallocate in another – Let objects lifetimes vary wildly Complex structure with lookup time and space overhead

heap Allows you to... – Allocate objects of any size and alignment – Allocate and deallocate from any thread • Even allocate in one thread and deallocate in another – Let objects lifetimes vary wildly Poor locality of reference

heap Allows you to... – Allocate objects of any size and alignment – Allocate and deallocate from any thread • Even allocate in one thread and deallocate in another – Let objects lifetimes vary wildly Fragmentation may make everything worse over time

heap Allows you to... – Allocate objects of any size and alignment – Allocate and deallocate from any thread • Even allocate in one thread and deallocate in another – Let objects lifetimes vary wildly Synchronisation overhead

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 11/110 So
you turn to allocators...

you turn to allocators... https://www.edvardmunch.org/the-scream.jsp

you turn to allocators... https://www.edvardmunch.org/the-scream.jsp But PMR allocators suck less

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 14/110 Polymorphic
Memory Resource “container” PMR allocator <<memory resource>>

Memory Resource “container” PMR allocator <<memory resource>> A “container” has a PMR allocator

Memory Resource “container” PMR allocator <<memory resource>> A “container” has a PMR allocator A PMR allocator references a memory resource

Memory Resource “container” PMR allocator <<memory resource>> A “container” has a PMR allocator A PMR allocator references a memory resource Standard containers are conveniently available under namespace std::pmr, e.g. std::pmr::vector<int>

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 18/110 memory_resource
class std::pmr::memory_resource { public: void* allocate(size_t bytes, size_t alignment); void deallocate(void* addr, size_t bytes, size_t alignment); bool is_equal(const memory_resource&) const noexcept; private: virtual void* do_allocate(size_t bytes, size_t align) = 0; virtual void do_deallocate(void* addr, size_t bytes, size_t align) = 0; virtual bool do_is_equal(const memory_resource&) const noexcept = 0; }; <memory_resource>

class std::pmr::memory_resource { public: void* allocate(size_t bytes, size_t alignment); void deallocate(void* addr, size_t bytes, size_t alignment); bool is_equal(const memory_resource&) const noexcept; private: virtual void* do_allocate(size_t bytes, size_t align) = 0; virtual void do_deallocate(void* addr, size_t bytes, size_t align) = 0; virtual bool do_is_equal(const memory_resource&) const noexcept = 0; }; <memory_resource> Inherit and implement these

class std::pmr::memory_resource { public: void* allocate(size_t bytes, size_t alignment); void deallocate(void* addr, size_t bytes, size_t alignment); bool is_equal(const memory_resource&) const noexcept; private: virtual void* do_allocate(size_t bytes, size_t align) = 0; virtual void do_deallocate(void* addr, size_t bytes, size_t align) = 0; virtual bool do_is_equal(const memory_resource&) const noexcept = 0; }; <memory_resource> Mildly annoying that alignment uses size_t and not align_val_t Inherit and implement these

class tracing_resource final : public std::pmr::memory_resource { public: tracing_resource(std::ostream& os) : os_(os) {} private: void* do_allocate(size_t bytes, size_t align) override { auto addr = operator new (bytes, std::align_val_t(align)); os_ << "allocate(" << bytes << ", " << align << ") -> " << addr << '\n'; return addr; } void do_deallocate(void* addr, size_t bytes, size_t align) override { os_ << "deallocate(" << addr << ", " << bytes << ", " << align << ")\n"; operator delete(addr, bytes, std::align_val_t(align)); } bool do_is_equal(const std::pmr::memory_resource& other) const noexcept override { return &other == this; } std::ostream& os_; };

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 26/110 https://godbolt.org/z/P7jojKdTf

Malleable Memory Management Method It is very important that your memory resource instance outlives all containers that use it.

Malleable Memory Management Method It is very important that your memory resource instance outlives all containers that use it. It’s very easy to miss using a PMR type, and not so easy to detect the mistake

Malleable Memory Management Method It is very important that your memory resource instance outlives all containers that use it. It’s very easy to miss using a PMR type, and not so easy to detect the mistake There are tools, though

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 30/110 string
frequency example class histogram { public: void add(const std::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::unordered_map<std::string, size_t> words_; };

frequency example class histogram { public: void add(const std::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::unordered_map<std::string, size_t> words_; }; int main() { std::string word; histogram hist; while (std::cin >> word) { hist.add(word); } hist.print_top(5, std::cout); }

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 39/110 Live
Demo

Malleable Memory Management Method Nothing surprising here, but good to have a baseline and a tool Let’s PMR it!

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 41/110 PMR
histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; };

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; PMR-ify the container and strings

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; Get an allocator to use

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; In C++17 it was necessary to express a type here, but in C++20 onwards, just use the diamond

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; int main() { std::pmr::string word; histogram hist; while (std::cin >> word) { hist.add(word); } hist.print_top(5, std::cout); }

Demo

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; int main() { std::pmr::string word; histogram hist; while (std::cin >> word) { hist.add(word); } hist.print_top(5, std::cout); }

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; int main() { std::pmr::string word; histogram hist; while (std::cin >> word) { hist.add(word); } hist.print_top(5, std::cout); } Not much new, because a default constructed polymorphic_allocator uses the default memory_resource, which is new_delete_resource.

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; int main() { std::pmr::string word; histogram hist; while (std::cin >> word) { hist.add(word); } hist.print_top(5, std::cout); } But we did see it being used!

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 50/110 Choosing
a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource You’ve seen this one. Plain operator new and operator delete.

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource A resource that always fails to allocate. Can be useful when testing.

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource A thread safe pool of memory blocks, in which allocations are simple indexing operations into an available slot in a block.

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource Same as above, but without the overhead of thread synchronization. Very fast when you know allocation and deallocation will happen in the same thread.

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource Very fast resource, where each allocation is at a monotonically increasing space in the available buffer. Deallocation only happens when the resource is destroyed. Useful in very local scopes.

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource You can cascade memory resources. Construct with a pointer to an upstream resource

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource void do_work(std::pmr::polymorphic_allocator<> alloc); auto heap = std::pmr::new_delete_resource(); std::pmr::monotonic_buffer_resource monotonic_buffers(heap); std::pmr::unsynchronized_pool_resource pool(&monotonic_buffers); do_work(&pool);

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource void do_work(std::pmr::polymorphic_allocator<> alloc); auto heap = std::pmr::new_delete_resource(); std::pmr::monotonic_buffer_resource monotonic_buffers(heap); std::pmr::unsynchronized_pool_resource pool(&monotonic_buffers); do_work(&pool); Construct a polymorphic allocator using pool.

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource void do_work(std::pmr::polymorphic_allocator<> alloc); auto heap = std::pmr::new_delete_resource(); std::pmr::monotonic_buffer_resource monotonic_buffers(heap); std::pmr::unsynchronized_pool_resource pool(&monotonic_buffers); do_work(&pool); Constrct a pool resource using monotonic_buffers.

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource void do_work(std::pmr::polymorphic_allocator<> alloc); auto heap = std::pmr::new_delete_resource(); std::pmr::monotonic_buffer_resource monotonic_buffers(heap); std::pmr::unsynchronized_pool_resource pool(&monotonic_buffers); do_work(&pool); Construct a monotonic buffer resource using the heap.

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource void do_work(std::pmr::polymorphic_allocator<> alloc); auto heap = std::pmr::new_delete_resource(); std::pmr::monotonic_buffer_resource monotonic_buffers(heap); std::pmr::unsynchronized_pool_resource pool(&monotonic_buffers); do_work(&pool); Get the new_delete_allocator.

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource void do_work(std::pmr::polymorphic_allocator<> alloc); auto heap = std::pmr::new_delete_resource(); std::pmr::monotonic_buffer_resource monotonic_buffers(heap); std::pmr::unsynchronized_pool_resource pool(&monotonic_buffers); do_work(&pool);

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource void do_work(std::pmr::polymorphic_allocator<> alloc); auto heap = std::pmr::new_delete_resource(); std::pmr::monotonic_buffer_resource monotonic_buffers(heap); std::pmr::unsynchronized_pool_resource pool(&monotonic_buffers); do_work(&pool); The allocator will use the pool to allocate memory.

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource void do_work(std::pmr::polymorphic_allocator<> alloc); auto heap = std::pmr::new_delete_resource(); std::pmr::monotonic_buffer_resource monotonic_buffers(heap); std::pmr::unsynchronized_pool_resource pool(&monotonic_buffers); do_work(&pool); If the pool does not have any free memory, it allocates more via the monotonic buffer.

a memory resource The header <memory_resource> offers: std::pmr::new_delete_resource std::pmr::null_resource std::pmr::synchronized_pool_resource std::pmr::unsynchronized_pool_resource std::pmr::monotonic_buffer_resource void do_work(std::pmr::polymorphic_allocator<> alloc); auto heap = std::pmr::new_delete_resource(); std::pmr::monotonic_buffer_resource monotonic_buffers(heap); std::pmr::unsynchronized_pool_resource pool(&monotonic_buffers); do_work(&pool); If the monotonic buffer doesn’t have any memory, it gets more from the heap

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 66/110 A
slight detour class tracing_resource final : public std::pmr::memory_resource { public: tracing_resource(std::ostream& os) : os_(os) {} private: void* do_allocate(size_t bytes, size_t align) override { auto addr = operator new (bytes, std::align_val_t(align)); os_ << "allocate(" << bytes << ", " << align << ") -> " << addr << '\n'; return addr; } void do_deallocate(void* addr, size_t bytes, size_t align) override { os_ << "deallocate(" << addr << ", " << bytes << ", " << align << ")\n"; operator delete(addr, bytes, std::align_val_t(align)); } bool do_is_equal(const std::pmr::memory_resource& other) const noexcept override { return &other == this; } std::ostream& os_; };

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 67/110 A
slight detour class tracing_resource final : public std::pmr::memory_resource { public: tracing_resource(std::ostream& os) : os_(os) {} private: void* do_allocate(size_t bytes, size_t align) override { auto addr = operator new (bytes, std::align_val_t(align)); os_ << "allocate(" << bytes << ", " << align << ") -> " << addr << '\n'; return addr; } void do_deallocate(void* addr, size_t bytes, size_t align) override { os_ << "deallocate(" << addr << ", " << bytes << ", " << align << ")\n"; operator delete(addr, bytes, std::align_val_t(align)); } bool do_is_equal(const std::pmr::memory_resource& other) const noexcept override { return &other == this; } std::ostream& os_; }; Maybe going directly to the heap isn’t such a great idea?

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 68/110 Improving
the tracing resource class tracing_resource final : public std::pmr::memory_resource { public: tracing_resource(std::ostream& os, std::pmr::memory_resource* next = std::pmr::get_default_resource()) : os_(os), next_(next) {} private: void* do_allocate(size_t bytes, size_t align) override { auto* addr = next_->allocate(bytes, align); os_ << "allocate(" << bytes << ", " << align << ") -> " << addr << '\n'; return addr; } void do_deallocate(void* addr, size_t bytes, size_t align) override { os_ << "deallocate(" << addr << ", " << bytes << ", " << align << ")\n"; next_->deallocate(addr, bytes, align); } bool do_is_equal(const std::pmr::memory_resource& other) const noexcept override { auto p = dynamic_cast<const tracing_resource*>(&other); return p && next_ == p->next_; } std::ostream& os_; std::pmr::memory_resource* next_; };

the tracing resource class tracing_resource final : public std::pmr::memory_resource { public: tracing_resource(std::ostream& os, std::pmr::memory_resource* next = std::pmr::get_default_resource()) : os_(os), next_(next) {} private: void* do_allocate(size_t bytes, size_t align) override { auto* addr = next_->allocate(bytes, align); os_ << "allocate(" << bytes << ", " << align << ") -> " << addr << '\n'; return addr; } void do_deallocate(void* addr, size_t bytes, size_t align) override { os_ << "deallocate(" << addr << ", " << bytes << ", " << align << ")\n"; next_->deallocate(addr, bytes, align); } bool do_is_equal(const std::pmr::memory_resource& other) const noexcept override { auto p = dynamic_cast<const tracing_resource*>(&other); return p && next_ == p->next_; } std::ostream& os_; std::pmr::memory_resource* next_; };

the tracing resource class tracing_resource final : public std::pmr::memory_resource { public: tracing_resource(std::ostream& os, std::pmr::memory_resource* next = std::pmr::get_default_resource()) : os_(os), next_(next) {} private: void* do_allocate(size_t bytes, size_t align) override { auto* addr = next_->allocate(bytes, align); os_ << "allocate(" << bytes << ", " << align << ") -> " << addr << '\n'; return addr; } void do_deallocate(void* addr, size_t bytes, size_t align) override { os_ << "deallocate(" << addr << ", " << bytes << ", " << align << ")\n"; next_->deallocate(addr, bytes, align); } bool do_is_equal(const std::pmr::memory_resource& other) const noexcept override { auto p = dynamic_cast<const tracing_resource*>(&other); return p && next_ == p->next_; } std::ostream& os_; std::pmr::memory_resource* next_; }; Get an underlying memory resource

the tracing resource class tracing_resource final : public std::pmr::memory_resource { public: tracing_resource(std::ostream& os, std::pmr::memory_resource* next = std::pmr::get_default_resource()) : os_(os), next_(next) {} private: void* do_allocate(size_t bytes, size_t align) override { auto* addr = next_->allocate(bytes, align); os_ << "allocate(" << bytes << ", " << align << ") -> " << addr << '\n'; return addr; } void do_deallocate(void* addr, size_t bytes, size_t align) override { os_ << "deallocate(" << addr << ", " << bytes << ", " << align << ")\n"; next_->deallocate(addr, bytes, align); } bool do_is_equal(const std::pmr::memory_resource& other) const noexcept override { auto p = dynamic_cast<const tracing_resource*>(&other); return p && next_ == p->next_; } std::ostream& os_; std::pmr::memory_resource* next_; }; Use the default memory resource if none is given

the tracing resource class tracing_resource final : public std::pmr::memory_resource { public: tracing_resource(std::ostream& os, std::pmr::memory_resource* next = std::pmr::get_default_resource()) : os_(os), next_(next) {} private: void* do_allocate(size_t bytes, size_t align) override { auto* addr = next_->allocate(bytes, align); os_ << "allocate(" << bytes << ", " << align << ") -> " << addr << '\n'; return addr; } void do_deallocate(void* addr, size_t bytes, size_t align) override { os_ << "deallocate(" << addr << ", " << bytes << ", " << align << ")\n"; next_->deallocate(addr, bytes, align); } bool do_is_equal(const std::pmr::memory_resource& other) const noexcept override { auto p = dynamic_cast<const tracing_resource*>(&other); return p && next_ == p->next_; } std::ostream& os_; std::pmr::memory_resource* next_; }; Use it to allocate and deallocate memory

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; };

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; Back to chosing an allocator for the word frequency histogram

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; int main() { std::pmr::string word; std::pmr::unsynchronized_pool_resource r; histogram hist(&r); while (std::cin >> word) { hist.add(word); } hist.print_top(5, std::cout); }

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::vector<count> popular(words_.begin(), words_.end()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; int main() { std::pmr::string word; std::pmr::unsynchronized_pool_resource r; histogram hist(&r); while (std::cin >> word) { hist.add(word); } hist.print_top(5, std::cout); } I choose an unsynchronized pool resource.

Demo

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::pmr::vector<count> popular(words_.begin(), words_.end(), words_.get_allocator()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; };

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::pmr::vector<count> popular(words_.begin(), words_.end(), words_.get_allocator()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; 84 allocations down from 133000

histogram class histogram { public: histogram(const std::pmr::polymorphic_allocator<>& alloc = {}): words_(alloc){} void add(const std::pmr::string& word) { ++words_[word]; } void print_top(size_t n, std::ostream& os) const { using count = std::pair<std::pmr::string, size_t>; std::pmr::vector<count> popular(words_.begin(), words_.end(), words_.get_allocator()); std::ranges::partial_sort(popular, popular.begin() + n, std::greater{}, &count::second); for (const auto& stat : popular | std::ranges::views::take(n)) { os << stat.first << '\t' << stat.second << '\n'; } } private: std::pmr::unordered_map<std::pmr::string, size_t> words_; }; But you really need tools like heaptrack to find the places you’ve missed.

for your types class counted_string { public: counted_string(size_t count, std::pmr::string str) : str_(std::move(str)) , count_(count) {} std::string_view str() const { return str_; } size_t count() const { return count_; } friend bool operator<(const counted_string& lh, const counted_string& rh) { return lh.count_ < rh.count_; } protected: std::pmr::string str_; size_t count_; };

for your types A PMR enabled type needs:

for your types A PMR enabled type needs: – A public allocator_type type

for your types A PMR enabled type needs: – A public allocator_type type – Constructors with the allocator type as the last argument

for your types A PMR enabled type needs: – A public allocator_type type – Constructors with the allocator type as the last argument – Variadic constructors with std::allocator_arg_t and allocator_type as the first parameters

for your types A PMR enabled type needs: – A public allocator_type type – Constructors with the allocator type as the last argument – Variadic constructors with std::allocator_arg_t and allocator_type as the first parameters – And a copy-like constructor with allocator

for your types A PMR enabled type needs: – A public allocator_type type – Constructors with the allocator type as the last argument – Variadic constructors with std::allocator_arg_t and allocator_type as the first parameters – And a copy-like constructor with allocator – And move-like constructor with allocator

for your types class counted_string { public: using allocator_type = std::pmr::polymorphic_allocator<>; counted_string(size_t count,std::pmr::string str, const allocator_type& alloc = {}); template <typename ... Ts> counted_string(std::allocator_arg_t, const allocator_type& alloc, size_t count, Ts&& ... ts); counted_string(const counted_string& orig); counted_string(const counted_string& orig, const allocator_type& alloc); counted_string(counted_string&& orig) noexcept; counted_string(counted_string&& orig, const allocator_type& alloc) noexcept; std::string_view str() const { return str_; } size_t count() const { return count_; } friend bool operator<(const counted_string& lh, const counted_string& rh); protected: std::pmr::string str_; size_t count_; };

for your types class counted_string { public: using allocator_type = std::pmr::polymorphic_allocator<>; counted_string(size_t count,std::pmr::string str, const allocator_type& alloc = {}); template <typename ... Ts> counted_string(std::allocator_arg_t, const allocator_type& alloc, size_t count, Ts&& ... ts); counted_string(const counted_string& orig); counted_string(const counted_string& orig, const allocator_type& alloc); counted_string(counted_string&& orig) noexcept; counted_string(counted_string&& orig, const allocator_type& alloc) noexcept; std::string_view str() const { return str_; } size_t count() const { return count_; } friend bool operator<(const counted_string& lh, const counted_string& rh); protected: std::pmr::string str_; size_t count_; }; Signal that this type uses a PMR allocator.

for your types class counted_string { public: using allocator_type = std::pmr::polymorphic_allocator<>; counted_string(size_t count,std::pmr::string str, const allocator_type& alloc = {}); template <typename ... Ts> counted_string(std::allocator_arg_t, const allocator_type& alloc, size_t count, Ts&& ... ts); counted_string(const counted_string& orig); counted_string(const counted_string& orig, const allocator_type& alloc); counted_string(counted_string&& orig) noexcept; counted_string(counted_string&& orig, const allocator_type& alloc) noexcept; std::string_view str() const { return str_; } size_t count() const { return count_; } friend bool operator<(const counted_string& lh, const counted_string& rh); protected: std::pmr::string str_; size_t count_; }; Constructor with allocator last

for your types class counted_string { public: using allocator_type = std::pmr::polymorphic_allocator<>; counted_string(size_t count,std::pmr::string str, const allocator_type& alloc = {}); template <typename ... Ts> counted_string(std::allocator_arg_t, const allocator_type& alloc, size_t count, Ts&& ... ts); counted_string(const counted_string& orig); counted_string(const counted_string& orig, const allocator_type& alloc); counted_string(counted_string&& orig) noexcept; counted_string(counted_string&& orig, const allocator_type& alloc) noexcept; std::string_view str() const { return str_; } size_t count() const { return count_; } friend bool operator<(const counted_string& lh, const counted_string& rh); protected: std::pmr::string str_; size_t count_; }; Variadic constructor with non-allocator args at end

for your types class counted_string { public: using allocator_type = std::pmr::polymorphic_allocator<>; counted_string(size_t count,std::pmr::string str, const allocator_type& alloc = {}); template <typename ... Ts> counted_string(std::allocator_arg_t, const allocator_type& alloc, size_t count, Ts&& ... ts); counted_string(const counted_string& orig); counted_string(const counted_string& orig, const allocator_type& alloc); counted_string(counted_string&& orig) noexcept; counted_string(counted_string&& orig, const allocator_type& alloc) noexcept; std::string_view str() const { return str_; } size_t count() const { return count_; } friend bool operator<(const counted_string& lh, const counted_string& rh); protected: std::pmr::string str_; size_t count_; }; Copy-like constructor with allocator, used by containers

for your types class counted_string { public: using allocator_type = std::pmr::polymorphic_allocator<>; counted_string(size_t count,std::pmr::string str, const allocator_type& alloc = {}); template <typename ... Ts> counted_string(std::allocator_arg_t, const allocator_type& alloc, size_t count, Ts&& ... ts); counted_string(const counted_string& orig); counted_string(const counted_string& orig, const allocator_type& alloc); counted_string(counted_string&& orig) noexcept; counted_string(counted_string&& orig, const allocator_type& alloc) noexcept; std::string_view str() const { return str_; } size_t count() const { return count_; } friend bool operator<(const counted_string& lh, const counted_string& rh); protected: std::pmr::string str_; size_t count_; }; Move-like constructor with allocator, used by containers

conclusion • Making your own types use PMR allocators is not hard, but it’s easy to make mistakes

conclusion • Making your own types use PMR allocators is not hard, but it’s easy to make mistakes • Using PMR types with standard containers is easy

conclusion • Making your own types use PMR allocators is not hard, but it’s easy to make mistakes • Using PMR types with standard containers is easy • PMR is very flexible because the memory resources and the types that uses them are decoupled

conclusion • Making your own types use PMR allocators is not hard, but it’s easy to make mistakes • Using PMR types with standard containers is easy • PMR is very flexible because the memory resources and the types that uses them are decoupled • Memory usage is quite opaque, so it’s not easy to see where you’ve made mistakes

conclusion • Making your own types use PMR allocators is not hard, but it’s easy to make mistakes • Using PMR types with standard containers is easy • PMR is very flexible because the memory resources and the types that uses them are decoupled • Memory usage is quite opaque, so it’s not easy to see where you’ve made mistakes • PMR is still not a panacea for your memory management needs, but it’s often sufficient

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 102/110 Just
one more thing... http://mojtv.hr/images/fb2de057-6db9-45f8-ba8a-6891149a4a87.jpg

one more thing... http://mojtv.hr/images/fb2de057-6db9-45f8-ba8a-6891149a4a87.jpg A pmr container never changes its memory resource.

one more thing... http://mojtv.hr/images/fb2de057-6db9-45f8-ba8a-6891149a4a87.jpg A pmr container never changes its memory resource. Not even when move-assigning.

one more thing... http://mojtv.hr/images/fb2de057-6db9-45f8-ba8a-6891149a4a87.jpg A pmr container never changes its memory resource. Not even when move-assigning. • If resource operator== says the memory resources are the same, then the internal pointers are taken over, otherwise it’s element-wise move assignment to newly allocated memory.

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 109/110 Learn
more https://tinyurl.com/cppweekly-on-pmr YouTube playlist with 6 episodes of Jason Turner’s C++Weekly covering PMR https://youtu.be/RLezJuqNcEQ CppConn 2019 – Pablo Halpern and Alisdair Meredith, “Congratulations! You survived C++11 allocators!”

Mmmmm – NDC{TechTown} 2023 © Björn Fahller @bjorn_fahller@mastodon.social 110/110 bjorn@fahller.se
@bjorn_fahller@mastodon.social @rollbear Björn Fahller Most Malleable Memory Management Method

TechTown - Most Malleable Memory Management Method

TechTown - Most Malleable Memory Management Method

More Decks by Björn Fahller

Featured

Transcript