January 23, 2019 at 9:26 pm
Comments posted to this topic are about the item Fast Enough
January 24, 2019 at 1:34 am
I think that when people see SQL Server they see a very visible and huge cost. I'm finding that even those who you have to squint a bit when they class themselves as millenials are reluctant to adopt the disciplines needed to put together a decent DB design.
These people are quick to jump to the NOSQL options and a palette of technologies to "solve" their problems but many of those problems are of their own making.
In my experience a lot of the costs in alternative technologies are hidden. Few think of the integration costs or that technology 'x' does 90% of what is required so now you need technology 'y' as well. Each component introduces a new thing to learn, a new thing that has to be supported in some way and a new thing to integrate into the whole. It isn't a trivial exercise to get all the components up, running, stable, playing nicely with each other. And then a component needs to be upgraded and that can be painful even with DevOps approaches.
There are some awesome pieces of software out there but there is a part of me that wonders how many of them exist because a problem has been created? How many solutions are actually coping strategies for choices that looked after the interests of a few in isolation rather than the bigger picture.
There is a lot that I wouldn't use SQL Server for but it's like rich mans shoes. A pair last a lifetime whereas a poor man spends far more because his don't last.
January 24, 2019 at 6:56 am
The database engine chosen for an application needs to match the expected data access patterns. RDMS are excellent for scenarios where you need to query, insert, and update manageable subsets of data while maintaining a high degree of referential integrity and transactional consistency. This covers most line of business applications. However, RDMS are poor when it comes to bulk loading, aggregate reporting, and BLOB storage. For non-normalized data like XML and JSON, it's only mediocre. Sure RDMS have been extended over time to support this functionality, but if you are starting a project from scratch and the primary data type is not normalized, then there are better options like CosmosDB or MongoDB specialized for that purpose.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
January 24, 2019 at 9:06 am
There are a lot of valuable open source stacks to solve problems. They don't solve the human issues.
412-977-3526 call/text
January 24, 2019 at 9:34 am
I had an old boss that got angry with me even though we were great friends especially professionally. He had a Developer write code to download and import the data for 1.2 million customers from a 3rd party data provider so that we could track daily progress of a customer migration we were doing. The Developer did a good job but it took more than 15 minute for him to just import the data into SQL Server never mind validating any of the data.
I suggested that BULK INSERT would be much faster and that I could set up some validation that would work during the import itself and would also sequester bad rows in a file for rework... and got yelled at with the normal "just because you can do something in SQL Server doesn't mean you should" and that I was a "one pony show" and that "to a hammer, everything is a nail", and all sorts of the same junk many of us hear from other people. I didn't yell back nor fault him for his unkind comments because he just doesn't know what he doesn't know.
I got with the Developer (who I got along with very well) and he gave me all the details I needed to do a rewrite using T-SQL and then implemented the code. My boss, the guy that yelled at me, was the one that ran the code every morning. After the change I implemented, it ran the import, validation, and transfer to the final table in 55 seconds. My boss was totally unaware but saw how short it ran and went running to the Developer saying that it ran too fast and that something must be wrong and it needed to be fixed NOW! The Developer said there's nothing wrong with the code (he had already produced the reports) and said to talk to me. After checking the reports to make sure they were accurate, my boss called me into the office and said "Ok, what did you do". I told him "I used a hammer while I was riding my pony". He smiled and said... "Heh... and now you're going to call me an idiot". My response was, "No... you already know that". Like I said... we were good friends and have remained so. To this day, he'll occasional call me and tell me he needs to "Hammer something out" and asks if I can help. The answer is always yes.
The point I'm trying to make here is that people already have the tools sitting right in front of them to pull off miracles. They simply don't take the time to learn them because they're not "hip" enough because they're not "all the rage" of the latest shiny new object . They fail to realize that (and I actually had a friend make me a hand-drawn piece of artwork with this) "Before you can think outside the box, you must FIRST realize... you're in a box".
--Jeff Moden
Change is inevitable... Change for the better is not.
January 24, 2019 at 11:37 am
Jeff Moden - Thursday, January 24, 2019 9:34 AMI had an old boss that got angry with me even though we were great friends especially professionally. He had a Developer write code to download and import the data for 1.2 million customers from a 3rd party data provider so that we could track daily progress of a customer migration we were doing. The Developer did a good job but it took more than 15 minute for him to just import the data into SQL Server never mind validating any of the data.I suggested that BULK INSERT would be much faster and that I could set up some validation that would work during the import itself and would also sequester bad rows in a file for rework... and got yelled at with the normal "just because you can do something in SQL Server doesn't mean you should" and that I was a "one pony show" and that "to a hammer, everything is a nail", and all sorts of the same junk many of us hear from other people. I didn't yell back nor fault him for his unkind comments because he just doesn't know what he doesn't know.
I got with the Developer (who I got along with very well) and he gave me all the details I needed to do a rewrite using T-SQL and then implemented the code. My boss, the guy that yelled at me, was the one that ran the code every morning. After the change I implemented, it ran the import, validation, and transfer to the final table in 55 seconds. My boss was totally unaware but saw how short it ran and went running to the Developer saying that it ran too fast and that something must be wrong and it needed to be fixed NOW! The Developer said there's nothing wrong with the code (he had already produced the reports) and said to talk to me. After checking the reports to make sure they were accurate, my boss called me into the office and said "Ok, what did you do". I told him "I used a hammer while I was riding my pony". He smiled and said... "Heh... and now you're going to call me an idiot". My response was, "No... you already know that". Like I said... we were good friends and have remained so. To this day, he'll occasional call me and tell me he needs to "Hammer something out" and asks if I can help. The answer is always yes.
The point I'm trying to make here is that people already have the tools sitting right in front of them to pull off miracles. They simply don't take the time to learn them because they're not "hip" enough because they're not "all the rage" of the latest shiny new object . They fail to realize that (and I actually had a friend make me a hand-drawn piece of artwork with this) "Before you can think outside the box, you must FIRST realize... you're in a box".
I had a similar thing happen. Web accessed database, took forever for customers to upload their data. I found out what the query was, created an index and voila! The boss said the same thing to me "What did you do?". Of course, I was the first one laid off because I wasn't as up on C# as the others......
January 24, 2019 at 12:25 pm
Jeff Moden - Thursday, January 24, 2019 9:34 AM...
The point I'm trying to make here is that people already have the tools sitting right in front of them to pull off miracles. They simply don't take the time to learn them because they're not "hip" enough because they're not "all the rage" of the latest shiny new object . They fail to realize that (and I actually had a friend make me a hand-drawn piece of artwork with this) "Before you can think outside the box, you must FIRST realize... you're in a box".
Hey Jeff,
I recall back in the 90s one of my first IT gigs was working alongside a gray bearded guy who proclaimed that FoxPro could do anything that shiny new Visual Basic and SQL Server could do, only faster and with fewer lines of code. Do you remember the FoxPro guys?
🙂
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
January 24, 2019 at 4:26 pm
miapjp - Thursday, January 24, 2019 11:37 AMI had a similar thing happen. Web accessed database, took forever for customers to upload their data. I found out what the query was, created an index and voila! The boss said the same thing to me "What did you do?". Of course, I was the first one laid off because I wasn't as up on C# as the others......
I keep it simple. If they ask me to write C#, I remind them that I was looking for a job when I found this one. 😉
--Jeff Moden
Change is inevitable... Change for the better is not.
January 24, 2019 at 4:39 pm
Eric M Russell - Thursday, January 24, 2019 12:25 PMJeff Moden - Thursday, January 24, 2019 9:34 AM...
The point I'm trying to make here is that people already have the tools sitting right in front of them to pull off miracles. They simply don't take the time to learn them because they're not "hip" enough because they're not "all the rage" of the latest shiny new object . They fail to realize that (and I actually had a friend make me a hand-drawn piece of artwork with this) "Before you can think outside the box, you must FIRST realize... you're in a box".Hey Jeff,
I recall back in the 90s one of my first IT gigs was working alongside a gray bearded guy who proclaimed that FoxPro could do anything that shiny new Visual Basic and SQL Server could do, only faster and with fewer lines of code. Do you remember the FoxPro guys?
🙂
I do indeed except the guys I worked with were using it as late as 2007.
The parallel here is that if they decide to do the same with SQL Server, I'm old enough to retire and whore myself out for legacy support at an incredible price. 😀
--Jeff Moden
Change is inevitable... Change for the better is not.
January 24, 2019 at 6:35 pm
Jeff Moden - Thursday, January 24, 2019 4:39 PMEric M Russell - Thursday, January 24, 2019 12:25 PMJeff Moden - Thursday, January 24, 2019 9:34 AM...
The point I'm trying to make here is that people already have the tools sitting right in front of them to pull off miracles. They simply don't take the time to learn them because they're not "hip" enough because they're not "all the rage" of the latest shiny new object . They fail to realize that (and I actually had a friend make me a hand-drawn piece of artwork with this) "Before you can think outside the box, you must FIRST realize... you're in a box".Hey Jeff,
I recall back in the 90s one of my first IT gigs was working alongside a gray bearded guy who proclaimed that FoxPro could do anything that shiny new Visual Basic and SQL Server could do, only faster and with fewer lines of code. Do you remember the FoxPro guys?
🙂I do indeed except the guys I worked with were using it as late as 2007.
The parallel here is that if they decide to do the same with SQL Server, I'm old enough to retire and whore myself out for legacy support at an incredible price. 😀
I just came across some active documentation at work that referenced VFP. I am afraid. (Yes, they were referencing Visual Fox Pro)
412-977-3526 call/text
January 25, 2019 at 6:52 am
robert.sterbal 56890 - Thursday, January 24, 2019 6:35 PMI just came across some active documentation at work that referenced VFP. I am afraid. (Yes, they were referencing Visual Fox Pro)
I work for a national retailer, and up until a few years ago, the POS was at least partially running on FoxPro, and data was transferred between the stores and home office by copying .dbf files across the WAN. The end user applications have since moved to .NET, but even today there are still some legacy back office processes creating .dbf files. This little database engine with it's self contained runtime will still be hiding under rocks for another generation.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
January 25, 2019 at 8:35 am
I built a few FoxPro and VFP apps. They were much better than VB at the time, and worked well. Of course, you'd want to use SQL Server as a backend, not the dbase defaults.
January 25, 2019 at 9:25 am
Jeff Moden - Thursday, January 24, 2019 9:34 AMI had an old boss that got angry with me even though we were great friends especially professionally. He had a Developer write code to download and import the data for 1.2 million customers from a 3rd party data provider so that we could track daily progress of a customer migration we were doing. The Developer did a good job but it took more than 15 minute for him to just import the data into SQL Server never mind validating any of the data.I suggested that BULK INSERT would be much faster and that I could set up some validation that would work during the import itself and would also sequester bad rows in a file for rework... and got yelled at with the normal "just because you can do something in SQL Server doesn't mean you should" and that I was a "one pony show" and that "to a hammer, everything is a nail", and all sorts of the same junk many of us hear from other people. I didn't yell back nor fault him for his unkind comments because he just doesn't know what he doesn't know.
I got with the Developer (who I got along with very well) and he gave me all the details I needed to do a rewrite using T-SQL and then implemented the code. My boss, the guy that yelled at me, was the one that ran the code every morning. After the change I implemented, it ran the import, validation, and transfer to the final table in 55 seconds. My boss was totally unaware but saw how short it ran and went running to the Developer saying that it ran too fast and that something must be wrong and it needed to be fixed NOW! The Developer said there's nothing wrong with the code (he had already produced the reports) and said to talk to me. After checking the reports to make sure they were accurate, my boss called me into the office and said "Ok, what did you do". I told him "I used a hammer while I was riding my pony". He smiled and said... "Heh... and now you're going to call me an idiot". My response was, "No... you already know that". Like I said... we were good friends and have remained so. To this day, he'll occasional call me and tell me he needs to "Hammer something out" and asks if I can help. The answer is always yes.
The point I'm trying to make here is that people already have the tools sitting right in front of them to pull off miracles. They simply don't take the time to learn them because they're not "hip" enough because they're not "all the rage" of the latest shiny new object . They fail to realize that (and I actually had a friend make me a hand-drawn piece of artwork with this) "Before you can think outside the box, you must FIRST realize... you're in a box".
I wonder why the developer there didn't use dot net's sqlbulkcopy? Its essentially the same functionality as bulk insert and has been around since 2.0 apparently.
January 25, 2019 at 9:53 am
Steve Jones - SSC Editor - Friday, January 25, 2019 8:35 AMI built a few FoxPro and VFP apps. They were much better than VB at the time, and worked well. Of course, you'd want to use SQL Server as a backend, not the dbase defaults.
When I saw the initial release of VisualStudio.NET back in 1999-ish, with it's integrated GUI and support for object oriented VB development, it reminded me of Visual FoxPro. I mean VisualStudio.NET in 2000 looked and felt like Visual FoxPro from back in 1995, as if VFP were secretly a prototype of what was to come.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
January 25, 2019 at 10:07 am
patrickmcginnis59 10839 - Friday, January 25, 2019 9:25 AMJeff Moden - Thursday, January 24, 2019 9:34 AMI had an old boss that got angry with me even though we were great friends especially professionally. He had a Developer write code to download and import the data for 1.2 million customers from a 3rd party data provider so that we could track daily progress of a customer migration we were doing. The Developer did a good job but it took more than 15 minute for him to just import the data into SQL Server never mind validating any of the data.I suggested that BULK INSERT would be much faster and that I could set up some validation that would work during the import itself and would also sequester bad rows in a file for rework... and got yelled at with the normal "just because you can do something in SQL Server doesn't mean you should" and that I was a "one pony show" and that "to a hammer, everything is a nail", and all sorts of the same junk many of us hear from other people. I didn't yell back nor fault him for his unkind comments because he just doesn't know what he doesn't know.
I got with the Developer (who I got along with very well) and he gave me all the details I needed to do a rewrite using T-SQL and then implemented the code. My boss, the guy that yelled at me, was the one that ran the code every morning. After the change I implemented, it ran the import, validation, and transfer to the final table in 55 seconds. My boss was totally unaware but saw how short it ran and went running to the Developer saying that it ran too fast and that something must be wrong and it needed to be fixed NOW! The Developer said there's nothing wrong with the code (he had already produced the reports) and said to talk to me. After checking the reports to make sure they were accurate, my boss called me into the office and said "Ok, what did you do". I told him "I used a hammer while I was riding my pony". He smiled and said... "Heh... and now you're going to call me an idiot". My response was, "No... you already know that". Like I said... we were good friends and have remained so. To this day, he'll occasional call me and tell me he needs to "Hammer something out" and asks if I can help. The answer is always yes.
The point I'm trying to make here is that people already have the tools sitting right in front of them to pull off miracles. They simply don't take the time to learn them because they're not "hip" enough because they're not "all the rage" of the latest shiny new object . They fail to realize that (and I actually had a friend make me a hand-drawn piece of artwork with this) "Before you can think outside the box, you must FIRST realize... you're in a box".
I wonder why the developer there didn't use dot net's sqlbulkcopy? Its essentially the same functionality as bulk insert and has been around since 2.0 apparently.
This occurred in (IIRC) early 2004 so .Net was definitely on the streets. I don't know if it matters because, as you know, I'm not a front end guy but the boss had insisted that it be written in Java.
--Jeff Moden
Change is inevitable... Change for the better is not.
Viewing 15 posts - 1 through 15 (of 17 total)
You must be logged in to reply to this topic. Login to reply