Allocations, Memory, and Data Locality

Allocation creates storage for a value or object. Retention keeps that storage alive because something still refers to it. Data locality concerns how closely the values an operation needs are arranged in memory. Reducing allocation, reducing retained memory, and improving locality are related but different optimizations. 1

A program can allocate rapidly without leaking. It can also allocate very little while a cache retains far more data than necessary.

Read a packet payload without copying it first

A byte packet contains a payload between a header and a trailer. An operation only needs to sum those payload bytes. Creating a separate array for every read duplicates data the operation could inspect in place.

Compare a copied range with a view of the existing range. This example sums bytes to keep ownership visible; it is not a checksum algorithm for security or corruption detection.

public static class Payload
{
    public static byte[] Copy(byte[] packet, int offset, int count)
    {
        return packet.AsSpan(offset, count).ToArray();
    }

    public static int SumCopy(byte[] packet, int offset, int count)
    {
        byte[] payload = Copy(packet, offset, count);
        int total = 0;
        foreach (byte value in payload) total += value;
        return total;
    }

    public static int SumView(byte[] packet, int offset, int count)
    {
        ReadOnlySpan<byte> payload = packet.AsSpan(offset, count);
        int total = 0;
        foreach (byte value in payload) total += value;
        return total;
    }
}

byte[] packet = { 9, 1, 2, 3, 9 };
byte[] saved = Payload.Copy(packet, 1, 3);
Console.WriteLine(Payload.SumView(packet, 1, 3)); // 6
packet[2] = 20;
Console.WriteLine(Payload.SumView(packet, 1, 3)); // 24
Console.WriteLine(Payload.SumView(saved, 0, 3));  // 6

def copy_payload(packet, offset, count):
    if offset < 0 or count < 0 or offset + count > len(packet):
        raise ValueError("Invalid payload range.")
    return bytes(packet[offset:offset + count])


def sum_copy(packet, offset, count):
    return sum(copy_payload(packet, offset, count))


def sum_view(packet, offset, count):
    if offset < 0 or count < 0 or offset + count > len(packet):
        raise ValueError("Invalid payload range.")
    with memoryview(packet) as whole:
        with whole[offset:offset + count] as payload:
            return sum(payload)

packet = bytearray([9, 1, 2, 3, 9])
saved = copy_payload(packet, 1, 3)
print(sum_view(packet, 1, 3))  # 6
packet[2] = 20
print(sum_view(packet, 1, 3))  # 24
print(sum_view(saved, 0, 3))   # 6

Copies sometimes buy the independence you need

A view is not a snapshot. If another thread or a buffer pool changes the packet during use, avoiding the copy may change correctness. Never return a pooled buffer while someone still uses a view into it. A small retained view can also keep a much larger backing allocation alive.

Choose ownership first. Use a copy when the consumer needs independent lifetime or stable historical contents. Use a view when the producer’s buffer remains valid and the operation can safely share it.

Locality is about the data you actually touch

CPUs fetch memory in chunks. Processing nearby values can make better use of cached data than following references to scattered objects. An array of references does not necessarily place the referenced objects’ fields next to each other. For a hot loop, keeping frequently used fields compactly arranged may reduce memory traffic. 1

The payload example traverses contiguous bytes in both versions; it demonstrates avoided copying, not a measured cache-locality speedup. A locality claim needs a representative workload and suitable profiling evidence. Concurrent writes to nearby data can introduce their own cache-coordination costs.

Measure allocation rate, retained memory, and the operation’s actual latency separately. Do not replace a clear data model with pooled mutable buffers or densely packed structures unless the measured benefit justifies the new lifetime and maintenance obligations.

Sources

1. Robert Nystrom — Data Locality.
2. Microsoft — Memory and Span.
3. Python — Memory views.