Ask HN: Books on designing disk-optimized data structures?
6 by memset | 4 comments on Hacker News.
Are there canonical books, resources, or readings for how to design data structures that will be primarily read and written to a disk rather than memory? Most of what I learned in school about big-O assumes that, for example, random access is O(1). However, random disk reads are really slow due to spacial locality. People who write databases obviously have solutions to this problem - for example, DuckDB is based on a number of papers that have come out over the years on this topic. If I wanted to design, ie, a tree structure which was intended to be read/written from a disk, are there general principles or patterns the have been developed to take advantage of locality of reference, minimize random reads, or decrease the overhead of writes, that I could familiarize myself with? What is the CLRS for disk?
6 by memset | 4 comments on Hacker News.
Are there canonical books, resources, or readings for how to design data structures that will be primarily read and written to a disk rather than memory? Most of what I learned in school about big-O assumes that, for example, random access is O(1). However, random disk reads are really slow due to spacial locality. People who write databases obviously have solutions to this problem - for example, DuckDB is based on a number of papers that have come out over the years on this topic. If I wanted to design, ie, a tree structure which was intended to be read/written from a disk, are there general principles or patterns the have been developed to take advantage of locality of reference, minimize random reads, or decrease the overhead of writes, that I could familiarize myself with? What is the CLRS for disk?
Comments
Post a Comment