August 16, 2019 at 9:12 am
Jeff Moden wrote:However, while that is certainly true and very valuable to companies of that size, most companies will never come close to needing the requirements that Amazon, EBay, and other monster companies need. When it comes to those that can easily use something a bit less expensive, the core RDBMS products are a great fit and the people building those products need to pay a whole lot more attention to them without making them so complicated that smaller companies with smaller or non-existant IT departments can still use them and larger companies still find good value in them.
Huh? There are millions of people using newer data tech like Hadoop, Spark, Tensorflow, Kafka, Redis and packaged solutions like Snowflake, DataIku, DataBricks. Most of their customers are not "monster" companies and the attraction of these products is that cutting edge tech, either cloud or on-prem, is available so cheaply.
It's not cheap.
I worked for a company that had all of it's services on Amazon (EC2, S3 , SQS, DDB)
we were the second biggest spenders behind Netflix (i think we were at 250k per month but dont quote me)
we did some simple maths at my new company. It turns out that running a local on site servers are cheaper for medium sized companies
MVDBA
August 16, 2019 at 1:20 pm
... let's hope they don't make the rest of BULK INSERT slow in the process), and stop deprecating incredibly useful features, then there might not be such a need to do so much outside the core.
They will deprecate features, but there will be no more removal, at least that's what they've said publicly. They'll cease development on some features.
Changing/improving old features is hard, often because of legacy code. They fear upsetting customers with changes far more than improving functionality, at least from where I observe things.
August 16, 2019 at 1:22 pm
we did some simple maths at my new company. It turns out that running a local on site servers are cheaper for medium sized companies
Be careful of simple maths. In working with customers, we find it's incredibly difficult to decide if it's cheaper or more expensive. That's without accounting for flexibility. Setting up and maintaining on-premises has a lot of costs people fail to consider unless they really analyze well. One client of mine spends around US$90k/month. They're much happier than on-premises, both with cost and the way things work. Everything is not easier, and there are challenges, but different ones than on-premises.
August 16, 2019 at 2:22 pm
Agreed. Elasticity rather than cost is very often the clincher when it comes to cloud. But whether cloud or on-prem it seems to me that most of the interesting and cutting edge work done with data is not using SQL DBMSs these days.
August 16, 2019 at 7:51 pm
Agreed. Elasticity rather than cost is very often the clincher when it comes to cloud. But whether cloud or on-prem it seems to me that most of the interesting and cutting edge work done with data is not using SQL DBMSs these days.
So, Nova, bring me up to date. What is the cutting edge work being done without DBMSs? Anything I have seen and used that does NOT use DBMS technology has horrible performance and reliability issues. Case in point is the popular financial software called Quicken. I've used it since 1986, sadly due to its being the best I've been able to find. HOWEVER, it's performance suffers horrendously with large amounts of data because the company refuses to use a DBMS system, opting instead for a proprietary file structure of their own design. There are constant structural failures to the point that they write their own file structure correcting routines. The results are pathetic. The larger the data, the worse the structural failures become, and the worse the assumptions made to 'fix' your data. Software should NEVER alter my data for me, PERIOD.
Rick
Disaster Recovery = Backup ( Backup ( Your Backup ) )
August 16, 2019 at 8:46 pm
Hi Rick,
Did you misread me? I didn't say without DBMS, I said the innovative stuff tends not to use SQL DBMS, meaning the traditional SQL-only engines made by Oracle, Microsoft and their like. Machine learning, AI, NLP and data science generally are much more likely to be built on multi-model ("3 Vs") database tech like Hadoop, Spark, etc, sometimes with some varieties of "new SQL" as well.
There are plenty of case studies around. E.g.:
https://conferences.oreilly.com/strata/strata-ny/public/schedule/speakers
August 16, 2019 at 9:57 pm
OK, I hear you. I guess I just always assume that Relational DBMS systems are best suited for SQL. I'm definitely getting obsolete. In my days, it was the best thing going.
Rick
Disaster Recovery = Backup ( Backup ( Your Backup ) )
August 19, 2019 at 8:06 am
nova wrote:Agreed. Elasticity rather than cost is very often the clincher when it comes to cloud. But whether cloud or on-prem it seems to me that most of the interesting and cutting edge work done with data is not using SQL DBMSs these days.
So, Nova, bring me up to date. What is the cutting edge work being done without DBMSs? Anything I have seen and used that does NOT use DBMS technology has horrible performance and reliability issues. Case in point is the popular financial software called Quicken. I've used it since 1986, sadly due to its being the best I've been able to find. HOWEVER, it's performance suffers horrendously with large amounts of data because the company refuses to use a DBMS system, opting instead for a proprietary file structure of their own design. There are constant structural failures to the point that they write their own file structure correcting routines. The results are pathetic. The larger the data, the worse the structural failures become, and the worse the assumptions made to 'fix' your data. Software should NEVER alter my data for me, PERIOD.
I'm 50% with you here. I think the issue is skillset in databases. bad design in 3NF etc etc at initial design means people opt for other solutions.
give me a boyce codd design any day of the week. the other solutions have been designed because database developers haven't been good enough (and i'm one of those)
but equally NoSQL developers get it wrong too... i'm pretty sure it's a case of "screw and screwdriver" vs "nail and hammer" - both have a purpose
MVDBA
Viewing 8 posts - 31 through 37 (of 37 total)
You must be logged in to reply to this topic. Login to reply