Write big table to csv

• Jeff Moden

  SSC Guru

  Points: 1004664

  More actions

  December 14, 2013 at 2:28 pm

  #1673480

  pven (12/14/2013)

  ---

  ...what is a good method to give users freedom to make their own queries...

  To be absolutely honest, that statement is a bit of an oxymoron because most users are ill equiped to write queries. They don't know about proper indexing, SARGability, Divide'n'Conquer, etc, etc, ad infinitum.

  That, notwithstanding, the key here is that the method you're using is probably just fine for allowing users to "make their own queries". You need to go see what those queries actually are and find out why they're taking so long.

  You also talk about them making CSVs from large tables. Hopefully, the CSVs aren't that large but you stated they're writing 6GB in about 90 minutes. The question now becomes, what are they doing with all of that data and why aren't they writing queries to summarize the data? It could be they're doing things with PowerPivot and the like to aggregate data and it would probably suit them better if you did review their queries, what their goals actually are, and provide better queries that take parameters instead of letting them write their own.

  --Jeff Moden

  ---

  RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
  First step towards the paradigm shift of writing Set Based code:
  ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

  Change is inevitable... Change for the better is not.
  

  ---

  Helpful Links:
  How to post code problems
  How to Post Performance Problems
  Create a Tally Function (fnTally)