Today I’m in San Francisco at Small Data SF 2025. I went to the conference last year and thought it was a great event. Watching people talk about data and how we might look at managing smaller systems, at dealing with the challenges of exploding volumes by querying, storing, and handling less data was fascinating. The event had me me really thinking about ways in which we can build better performing (and cheaper) systems.
To be clear, small data isn’t very little data. Often this is still 100s of GB, perhaps low TBs, but it’s getting away from the idea of thinking we’ll be working on PB-sized big data systems, or that we even need to.
Last year there were lots of talks on data analysis, querying, and even AI, but using smaller sets of data in practical ways that provide value to organizations and individuals by judiciously choosing data sets. Either recent or representative data.
This helped me think of new ways for subsetting, which is something I’ve been pushing at Redgate for our TDM product.
I’m looking forward to the talks. This is a quick trip. I skipped the workshops yesterday since they weren’t that great last year (too many product/company pitches from Silicon Valley) and flew out last night after coaching kids in their first practice. At the conference today listening to talks and doing networking get together after before flying back early tomorrow.
A quick trip, but I’m sure I’ll lots to write about (and think about) in the future.