Package io.netty.buffer

Class AdaptivePoolingAllocator

  • All Implemented Interfaces:
    AdaptiveByteBufAllocator.AdaptiveAllocatorApi
    @UnstableApi
final class AdaptivePoolingAllocator
extends java.lang.Object
implements AdaptiveByteBufAllocator.AdaptiveAllocatorApi
    An auto-tuning pooling allocator, that follows an anti-generational hypothesis.

    The allocator is organized into a list of Magazines, and each magazine has a chunk-buffer that they allocate buffers from.

    The magazines hold the mutexes that ensure the thread-safety of the allocator, and each thread picks a magazine based on the id of the thread. This spreads the contention of multi-threaded access across the magazines. If contention is detected above a certain threshold, the number of magazines are increased in response to the contention.

    The magazines maintain histograms of the sizes of the allocations they do. The histograms are used to compute the preferred chunk size. The preferred chunk size is one that is big enough to service 10 allocations of the 99-percentile size. This way, the chunk size is adapted to the allocation patterns.

    Computing the preferred chunk size is a somewhat expensive operation. Therefore, the frequency with which this is done, is also adapted to the allocation pattern. If a newly computed preferred chunk is the same as the previous preferred chunk size, then the frequency is reduced. Otherwise, the frequency is increased.

    This allows the allocator to quickly respond to changes in the application workload, without suffering undue overhead from maintaining its statistics.

    Since magazines are "relatively thread-local", the allocator has a chunk cache that allows excess chunks from any magazine to be shared with other magazines.

    • Field Detail

      • IS_LOW_MEM

        private static final boolean IS_LOW_MEM

      • DISABLE_THREAD_LOCAL_MAGAZINES_ON_LOW_MEM

        private static final boolean DISABLE_THREAD_LOCAL_MAGAZINES_ON_LOW_MEM
        Whether the IS_LOW_MEM setting should disable thread-local magazines. This can have fairly high performance overhead.

      • MIN_CHUNK_SIZE

        static final int MIN_CHUNK_SIZE
        The 128 KiB minimum chunk size is chosen to encourage the system allocator to delegate to mmap for chunk allocations. For instance, glibc will do this. This pushes any fragmentation from chunk size deviations off physical memory, onto virtual memory, which is a much, much larger space. Chunks are also allocated in whole multiples of the minimum chunk size, which itself is a whole multiple of popular page sizes like 4 KiB, 16 KiB, and 64 KiB.
        See Also:
        Constant Field Values

      • MAX_STRIPES

        private static final int MAX_STRIPES

      • MAX_CHUNK_SIZE

        private static final int MAX_CHUNK_SIZE
        The maximum size of a pooled chunk, in bytes. Allocations bigger than this will never be pooled.

        This number is 8 MiB, and is derived from the limitations of internal histograms.

      • MAX_POOLED_BUF_SIZE

        private static final int MAX_POOLED_BUF_SIZE

      • CHUNK_REUSE_QUEUE

        static final int CHUNK_REUSE_QUEUE
        The capacity if the chunk reuse queues, that allow chunks to be shared across magazines in a group. The default size is twice NettyRuntime.availableProcessors(), same as the maximum number of magazines per magazine group.

      • CHUNK_PURGE_POLLS_THREAD_LOCAL

        static final long CHUNK_PURGE_POLLS_THREAD_LOCAL

      • CHUNK_PURGE_POLLS_SHARED

        static final long CHUNK_PURGE_POLLS_SHARED

      • CHUNK_PURGE_THRESHOLD

        static final int CHUNK_PURGE_THRESHOLD

      • THREAD_LOCAL_CACHE_MAX_BYTES

        static final int THREAD_LOCAL_CACHE_MAX_BYTES
        Per-size-class upper bound (in bytes) on the thread-local chunk cache. When a size class cache holds this many bytes worth of chunks, further offers are rejected and the chunk is marked for immediate deallocation. Chunks already in the cache are only evicted by the purge mechanism (they must be full and idle for CHUNK_PURGE_THRESHOLD consecutive purge cycles).

      • THREAD_LOCAL_CACHE_MIN_BYTES

        static final int THREAD_LOCAL_CACHE_MIN_BYTES
        Per-size-class lower bound (in bytes) on the thread-local chunk cache. The purge mechanism will not evict chunks below this retention floor, even if they are full and idle. Clamped to THREAD_LOCAL_CACHE_MAX_BYTES if the configured value exceeds it. When equal to THREAD_LOCAL_CACHE_MAX_BYTES, purge eviction is effectively disabled.

      • MAGAZINE_BUFFER_QUEUE_CAPACITY

        private static final int MAGAZINE_BUFFER_QUEUE_CAPACITY
        The capacity if the magazine local buffer queue. This queue just pools the outer ByteBuf instance and not the actual memory and so helps to reduce GC pressure.

      • SIZE_CLASSES

        private static final int[] SIZE_CLASSES
        The size classes are chosen based on the following observation:

        Most allocations, particularly ones above 256 bytes, aim to be a power-of-2. However, many use cases, such as framing protocols, are themselves operating or moving power-of-2 sized payloads, to which they add a small amount of overhead, such as headers or checksums. This means we seem to get a lot of mileage out of having both power-of-2 sizes, and power-of-2-plus-a-bit.

        On the conflicting requirements of both having as few chunks as possible, and having as little wasted memory within each chunk as possible, this seems to strike a surprisingly good balance for the use cases tested so far.

      • SIZE_CLASSES_COUNT

        private static final int SIZE_CLASSES_COUNT

      • SIZE_INDEXES

        private static final byte[] SIZE_INDEXES