Schedule - PostgreSQL Development Conference 2024

Streaming I/O and vectored I/O

Date: 2024-05-29
Time: 09:00–09:50
Room: Fletcher(1900)
Level: Advanced

Streaming I/O and vectored I/O

Traditionally, PostgreSQL did all relation I/O as sequential 8KB pread() or pwrite() system calls. This talk is about a new "streaming" programming model that allows multiple blocks to be read or written at the same time, and eventually in the background ahead of time.

I will talk about several things that cover recent and ongoing commits:

  • overview of system file I/O interfaces
  • storage manager changes for multi-buffer operations
  • buffer manager changes for multi-buffer operations
  • the stream abstraction
  • "synchronous" streaming I/O
  • partial steps to asynchronous I/O
  • use cases (heap/index scans, vacuum, recovery, ...)

This corresponds to commitfest entry https://commitfest.postgresql.org/46/4532/ (and earlier related work).

I will also talk more briefly about some aspects of the next stages of the larger AIO project:

  • overview of kernel buffering, pros, cons and motivations for escaping from it
  • why asynchronous I/O and direct I/O go together
  • a peek at WIP asynchronous I/O system
  • asynchronous streaming I/O

Talk recorded at https://youtu.be/7HHcD2shS4o.

Speaker

Thomas Munro