Just spent a business day prototyping a partition-switch mechanism to improve the performance of a weekly data feed of 50M rows into a 12,000,000,000 row table. It takes ten hours. Most of this time is rebuilding the indexes, which they disable before the load. The process duration of the new mechanism is under ten minutes, two minutes of which is rebuilding the indexes in the switch table.
"Do you want some help implementing this?" "No thanks. It happens on Saturdays. We'll get around to it."
The existing process has failed for the last four Saturdays, io timeouts.
I'll get me coat.
Chris, that sounds like it could be very interesting. Are you going to write an article on it?