Developer knowledge network · moderated exchange

UnreliableCode қауымдастығы

Әзірлеушілерді зерттеу, кері инженерия және кодтау қауымдастығы

Knowledge indexТірі
4Categories
919Threads
2.8KЖазбалар
Tutorial

Processing High-Throughput Network Streams with System.IO.Pipelines (PipeReader / PipeWriter) [StackOverflow Architecture Guide]

channel_concurrency
Reactive & Pipeline
MEMBER
Өкіл: 156
Қосылу күні: Jan 2022
Хабарламалар: 10
Рахмет: 16
2 апта бұрын · Aug 8, 2026 1:10 AM
#1

Why System.IO.Pipelines (the engine behind ASP.NET Core Kestrel) beats traditional Stream and byte[] buffers:

  • Memory Management: Automatically manages buffer pooling and zero-copy slicing.
  • Backpressure: PipeWriter pauses upstream producers when downstream consumers are busy.
  • Line/Packet Parsing:
CSHARP
while (true)
{
    ReadResult result = await reader.ReadAsync();
    ReadOnlySequence<byte> buffer = result.Buffer;
    while (TryReadLine(ref buffer, out ReadOnlySequence<byte> line))
    {
        ProcessLine(line);
    }
    reader.AdvanceTo(buffer.Start, buffer.End); // Tells pipeline which bytes were consumed!
    if (result.IsCompleted) break;
}
csharp_async_master
Async & Task Expert
MEMBER
Өкіл: 57
Қосылу күні: Mar 2019
Хабарламалар: 18
Рахмет: 40
2 апта бұрын · Aug 8, 2026 5:15 AM
#2

System.IO.Pipelines is how Kestrel handles millions of HTTP requests per second with flat memory usage.

dotnet_runtime_architect
.NET Core Specialist
MEMBER
Өкіл: 103
Қосылу күні: Apr 2018
Хабарламалар: 40
Рахмет: 24
2 апта бұрын · Aug 8, 2026 3:28 PM
#3

The AdvanceTo(consumed, examined) design solves the classic 'incomplete packet' problem cleanly.