December 10, 2019 at 12:16 pm
I have questions of exporting SQL server data, the details as below, thanks!
December 10, 2019 at 2:38 pm
If you're really looking to automate and control this, SSIS is the magic place for data movement within SQL Server.
However, if you want to stick to a command line driven mechanism, I'd look to two choices.
First, SQLCMD.EXE. This can absolutely control export to a CSV file. BCP is ancient technology, only there for backwards compatibility. SQLCMD is a better choice all round. It's easier to script and control, supports parameters and more.
Second, PowerShell is your buddy. You can easily export to a CSV file using a PowerShell command. Even more control than SQLCMD and you can combine your PowerShell scripts with the DBATools module to get really crazy with your automation.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
December 10, 2019 at 3:04 pm
Heh... no, Grant... you and I are ancient technology. 😀 BCP is still a nasty fast work horse. While it wasn't designed to handle true CSV exports nor even a straight comma separated version, it still works fine and fast. While I understand the lure of SSIS to do such a thing, I'm close enough to say that I'll give it up and retire first before I put an SSIS package in play. 😀
--Jeff Moden
Change is inevitable... Change for the better is not.
December 10, 2019 at 3:17 pm
Yeah, I am ancient. I'd still pick SQLCMD or PowerShell over bcp though.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
December 10, 2019 at 3:24 pm
Are you exporting the data to load into a different SQL Server instance? If so, then you should "native" format on the output. That will save a lot of time on the subsequent load.
SQL DBA,SQL Server MVP(07, 08, 09) A socialist is someone who will give you the shirt off *someone else's* back.
December 10, 2019 at 3:38 pm
What is the context for what you're trying to do, how are you running the export and how big is the data set? There are a number of options, bcp, SSIS, sqlcmd, powershell that all have their own advantages and disadvantages in terms of setting up and running.
December 10, 2019 at 5:55 pm
Yeah, I am ancient. I'd still pick SQLCMD or PowerShell over bcp though.
Heh... guess I'm more ancient. I've not found much that will actually beat BCP for performance for the types of things I've had to do. To each their own, though.
--Jeff Moden
Change is inevitable... Change for the better is not.
December 10, 2019 at 9:38 pm
Grant Fritchey wrote:Yeah, I am ancient. I'd still pick SQLCMD or PowerShell over bcp though.
Heh... guess I'm more ancient. I've not found much that will actually beat BCP for performance for the types of things I've had to do. To each their own, though.
I agree that BCP is normally very performant. Sometimes you need a format file, which is often a real pain to get working correctly, but once you do BCP works great.
The best part of SSIS is the very easy async exec available. I would use SSIS if I needed to run multiple data moves at the same time.
SQL DBA,SQL Server MVP(07, 08, 09) A socialist is someone who will give you the shirt off *someone else's* back.
December 11, 2019 at 1:15 am
Are you exporting the data to load into a different SQL Server instance? If so, then you should "native" format on the output. That will save a lot of time on the subsequent load.
No, I only export data from SQL Server into a file (csv or other format file )
December 11, 2019 at 4:21 am
Jeff Moden wrote:Grant Fritchey wrote:Yeah, I am ancient. I'd still pick SQLCMD or PowerShell over bcp though.
Heh... guess I'm more ancient. I've not found much that will actually beat BCP for performance for the types of things I've had to do. To each their own, though.
I agree that BCP is normally very performant. Sometimes you need a format file, which is often a real pain to get working correctly, but once you do BCP works great.
The best part of SSIS is the very easy async exec available. I would use SSIS if I needed to run multiple data moves at the same time.
Totally agreed on parallel loads whether asynchronous or synchronous. Makes it easy. Flow control is one of the things that I think SSIS is good at.
Like you probably have, I've also done such things without SSIS.
--Jeff Moden
Change is inevitable... Change for the better is not.
December 11, 2019 at 4:29 am
Jeff Moden wrote:Grant Fritchey wrote:Yeah, I am ancient. I'd still pick SQLCMD or PowerShell over bcp though.
Heh... guess I'm more ancient. I've not found much that will actually beat BCP for performance for the types of things I've had to do. To each their own, though.
I agree that BCP is normally very performant. Sometimes you need a format file, which is often a real pain to get working correctly, but once you do BCP works great.
The best part of SSIS is the very easy async exec available. I would use SSIS if I needed to run multiple data moves at the same time.
As I want to make a report to let user export data from SQL Server database, if we use SSIS to export data, and if IT guys set the SSIS package in the backend, how can we let user use it in the frontend ? thanks!
December 11, 2019 at 5:21 am
ScottPletcher wrote:Jeff Moden wrote:Grant Fritchey wrote:Yeah, I am ancient. I'd still pick SQLCMD or PowerShell over bcp though.
Heh... guess I'm more ancient. I've not found much that will actually beat BCP for performance for the types of things I've had to do. To each their own, though.
I agree that BCP is normally very performant. Sometimes you need a format file, which is often a real pain to get working correctly, but once you do BCP works great.
The best part of SSIS is the very easy async exec available. I would use SSIS if I needed to run multiple data moves at the same time.
As I want to make a report to let user export data from SQL Server database, if we use SSIS to export data, and if IT guys set the SSIS package in the backend, how can we let user use it in the frontend ? thanks!
That's a part of the reason I don't do such things and, instead, will write stored procedures that the user can execute without having privs to anything else in the system.
--Jeff Moden
Change is inevitable... Change for the better is not.
December 11, 2019 at 12:36 pm
ScottPletcher wrote:Jeff Moden wrote:Grant Fritchey wrote:Yeah, I am ancient. I'd still pick SQLCMD or PowerShell over bcp though.
Heh... guess I'm more ancient. I've not found much that will actually beat BCP for performance for the types of things I've had to do. To each their own, though.
I agree that BCP is normally very performant. Sometimes you need a format file, which is often a real pain to get working correctly, but once you do BCP works great.
The best part of SSIS is the very easy async exec available. I would use SSIS if I needed to run multiple data moves at the same time.
As I want to make a report to let user export data from SQL Server database, if we use SSIS to export data, and if IT guys set the SSIS package in the backend, how can we let user use it in the frontend ? thanks!
I'm going to ditto Jeff, but say that in this case, I'd go with PowerShell as the magic behind the report.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
December 11, 2019 at 3:18 pm
ScottPletcher wrote:Jeff Moden wrote:Grant Fritchey wrote:Yeah, I am ancient. I'd still pick SQLCMD or PowerShell over bcp though.
Heh... guess I'm more ancient. I've not found much that will actually beat BCP for performance for the types of things I've had to do. To each their own, though.
I agree that BCP is normally very performant. Sometimes you need a format file, which is often a real pain to get working correctly, but once you do BCP works great.
The best part of SSIS is the very easy async exec available. I would use SSIS if I needed to run multiple data moves at the same time.
As I want to make a report to let user export data from SQL Server database, if we use SSIS to export data, and if IT guys set the SSIS package in the backend, how can we let user use it in the frontend ? thanks!
Well what is the front end?
December 11, 2019 at 4:38 pm
892717952 wrote:ScottPletcher wrote:Jeff Moden wrote:Grant Fritchey wrote:Yeah, I am ancient. I'd still pick SQLCMD or PowerShell over bcp though.
Heh... guess I'm more ancient. I've not found much that will actually beat BCP for performance for the types of things I've had to do. To each their own, though.
I agree that BCP is normally very performant. Sometimes you need a format file, which is often a real pain to get working correctly, but once you do BCP works great.
The best part of SSIS is the very easy async exec available. I would use SSIS if I needed to run multiple data moves at the same time.
As I want to make a report to let user export data from SQL Server database, if we use SSIS to export data, and if IT guys set the SSIS package in the backend, how can we let user use it in the frontend ? thanks!
I'm going to ditto Jeff, but say that in this case, I'd go with PowerShell as the magic behind the report.
Or a simple batch file that makes the call with a "trusted" connection based on whomever the user is.
Of course, I have to ditto ZZartin's observation about not knowing what the "front end" actually is. We need to know that so we can determine what kind of call to make. It might simply be a call to a properly setup stored procedure.
--Jeff Moden
Change is inevitable... Change for the better is not.
Viewing 15 posts - 1 through 15 (of 18 total)
You must be logged in to reply to this topic. Login to reply