July 14, 2014 at 10:16 pm
Comments posted to this topic are about the item Not Good Enough For Government Work
Best wishes,
Phil Factor
July 15, 2014 at 5:21 am
The moral of the story being, if your database vendor doesn't use it for their accounts, don't run your 'bank' on it.
I'm a DBA.
I'm not paid to solve problems. I'm paid to prevent them.
July 15, 2014 at 6:33 am
Like I've always said: a good developer recognizes that he can't be an expert in everything. I once had someone ask me if I was a SQL expert, I laughed and said I've been in the software world for 15 years, and I've never even met one!
July 15, 2014 at 6:33 am
If you give a mechanic a new tool, does he dump all his other tools? No, but it seems developers tend to view the latest new database widget as the replacement for all the others. Anyone remember object databases?
The more you are prepared, the less you need it.
July 15, 2014 at 6:45 am
Without reading the underlying links, I would guess at issue is the classic conflict between .net/java/whatever developers and database architechs/developers/dbas. Like many others on this forum, I have seen both sides and can understand the "whatever" developers desire to do what they want, to just use the database as a more permanent datastore. However, as you've pointed out, the rules/guidelines/best practices are there for a reason and the events revealed the gross lack of understanding/experience/(fill in blank here) of the architects.
The follow up question, as I am responding off the top of my head, are those really 2 separate issues?
1. Don't know or want to know the best practices for handling data and why DB architecture (schema and code for ACID-ity) is important.
2. Don't have the experience with these kinds of systems and the forethought to consider the soundness of the code, not just that it completes the drawn-up workflow diagram, and test for possible security issues.
Amateurs indeed.
July 15, 2014 at 7:07 am
NoSQL databases are a solution to a wide range of niche applications that for the most part didn't exist a decade ago. However, Bitcoin is a revolutionary new business model built on top of what should be a traditional stock trading database, so the concepts of ACID proof transactional processing should still be applied. What's going on is that these NoSQL database vendors are seeking venture capital, so the movement is steeped in kool-aid and guerrilla marketing. A lot of organizations are getting led down the proverbial primrose path.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
July 15, 2014 at 7:31 am
From what I've read, these systems were caching in separate threads and fell victim to "lost transactions." Given the details, I'm not all that sure these guys would have faired any better with a relational database back end with coding and testing like that.
Is it impossible to have acid properties in a system using a non relational database? Using MongoDB? Using any NoSQL?
July 15, 2014 at 7:34 am
The key measure for all sorts of buggy and risky problems is whether or not they are insurable. These are process questions for the most part, not coding ones.
July 15, 2014 at 7:44 am
Yes, it is certainly possible to have a NoSQL database that handles ACID properly. It just surprises me that some don't. As I understand the Bitcoin exploit, I can't see how it would have happened with multi-user RDBMS. This is, after all, a problem that is intrinsic to any trading or banking system. The links are well-worth reading if you're interested in the details.
Best wishes,
Phil Factor
July 15, 2014 at 7:44 am
At least for as long as I can recall, and probably dating back to Sybase, SQL Server has given the developer some flexibility over isolation level. Starting with SQL Server 2014, we now have the DELAYED_DURABILITY database option.
I havn't read up on this particular new feature yet, but does anyone know if ISOLATION LEVEL READ UNCOMMITTED is now even more problematic when combined with DELAYED_DURABILITY = ALLOWED ?
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
July 15, 2014 at 8:06 am
I always cringe when someone talks about "dumping" terrabytes of "stuff" in SQL Server..., so a few analysts and contractors can chew on it for while..., just to see if it provides any useful information.
Please, dump your stuff in something like Hadoop. It was design for, and is well suited for, that specific task.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
July 15, 2014 at 8:16 am
Phil Factor (7/15/2014)
@patrickmcginnis59Yes, it is certainly possible to have a NoSQL database that handles ACID properly. It just surprises me that some don't. As I understand the Bitcoin exploit, I can't see how it would have happened with multi-user RDBMS. This is, after all, a problem that is intrinsic to any trading or banking system. The links are well-worth reading if you're interested in the details.
I've seen someone write a race condition with SQL Server, essentially they did a test on a row, then updated it based on the test, and it was not in a single statement (ie., the test didn't happen with the 'where' clause), and there was no lock to be had preventing intervening updates, and this sounds like what happened in the case of the Bitcoin thefts.
I've seen ACID properties work with non relational database technology, and I've read about banking applications (ATM's) that use eventual consistency.
July 15, 2014 at 2:16 pm
Phil Factor (7/15/2014)
@patrickmcginnis59Yes, it is certainly possible to have a NoSQL database that handles ACID properly. It just surprises me that some don't. As I understand the Bitcoin exploit, I can't see how it would have happened with multi-user RDBMS. This is, after all, a problem that is intrinsic to any trading or banking system. The links are well-worth reading if you're interested in the details.
"We handle all that in the app using the ORM because business rules shouldn't be in the database layer and just use it as a dumb datastore, <<IT'S BEST PRACTICE>>" perhaps?
I'm a DBA.
I'm not paid to solve problems. I'm paid to prevent them.
July 15, 2014 at 2:25 pm
SQL2219 (7/15/2014)
Like I've always said: a good developer recognizes that he can't be an expert in everything. I once had someone ask me if I was a SQL expert, I laughed and said I've been in the software world for 15 years, and I've never even met one!
I've met a few. One was sooooo good he could spot immediately that the optimiser was wrong when it showed a crap plan and stats for some of his work, and the vastly improved ones for an alternative method. At a glance - that's awesome! He was even kind enough to point out his obvious technical superiority, given that he "know what his code did, so didn't need that" and I had to use "stuff like that".
He was definitely an expert. Must have been, he kept saying so.
I'm a DBA.
I'm not paid to solve problems. I'm paid to prevent them.
July 15, 2014 at 2:30 pm
Eric M Russell (7/15/2014)
At least for as long as I can recall, and probably dating back to Sybase, SQL Server has given the developer some flexibility over isolation level. Starting with SQL Server 2014, we now have the DELAYED_DURABILITY database option.I havn't read up on this particular new feature yet, but does anyone know if ISOLATION LEVEL READ UNCOMMITTED is now even more problematic when combined with DELAYED_DURABILITY = ALLOWED ?
Of ffs. I just shudder to think what some bloody vendors are going to do with THAT. I may chuck it all in and run away to another type of circus entirely. That reminds me, I have cluebatting to do tomorrow. 🙁
I'm a DBA.
I'm not paid to solve problems. I'm paid to prevent them.
Viewing 15 posts - 1 through 15 (of 28 total)
You must be logged in to reply to this topic. Login to reply