Daily Shaarli

All links of one day in a single page.

November 30, 2022

Big Data Storage - Bert Hubert's writings
thumbnail

This is a page about some of the mechanics of ‘big data’, specifically how to store, transfer and process perhaps 100s of millions or billions of rows/events.
Although some people think big data starts at 65k rows, at which point Excel used to lose it, I’m talking about gigabytes of data here.
Infrastructure for this kind of thing abounds. Data can however come from lots of sources, and go to lots of different places.