May 9, 2014 at 2:29 pm
patrickmcginnis59 10839 (5/9/2014)
David.Poole (5/8/2014)
Google the "Hype Cycle".NOSQL is such a broad term that it is next to meaningless.
Not all NOSQL solutions deliver "No single point of failure".
BASE has made people realise how much they miss ACID. Eventually consistent doesn't really work.
Whats wrong with eventual consistency? Its not designed to be a drop in replacement for the 'C' in ACID right? Its more like the 'C' in CAP from what I remember.
As usual, the problem is not in the designed functionality but the assumed characteristics. A perfect example being the BitCoin theft from the exchange that got ripped off recently. The balance was not consistent after each withdrawal which allowed multiple withdrawals of the balance which eventually became consistent to a debt e.g. with a balance of $100 you can withdraw $100 and with eventual consistency you might be able to repeat this 10,001 times before it becomes consistent thereby creating a balance of -$1,000,000 so with an outlay of $100 you have made one million dollars clear.
Nice.
The problem was not with the database but with the assumed characteristics. The coder, whether they knew it or not, were relying on nonexistent ACID properties.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
May 9, 2014 at 2:35 pm
David.Poole (5/9/2014)
Whats wrong with eventual consistency? Its not designed to be a drop in replacement for the 'C' in ACID right? Its more like the 'C' in CAP from what I remember.
It seems that the "eventual" thing is a bit unreliable. It isn't guaranteed and people have started to notice.
I am sure it is not what you meant but it sounds like how most people think. It isn't unreliable in the sense of being defective but it is non-transactional and doesn't have the same levels of repeatability.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
May 10, 2014 at 10:20 am
Gary Varga (5/9/2014)
David.Poole (5/9/2014)
Whats wrong with eventual consistency? Its not designed to be a drop in replacement for the 'C' in ACID right? Its more like the 'C' in CAP from what I remember.
It seems that the "eventual" thing is a bit unreliable. It isn't guaranteed and people have started to notice.
I am sure it is not what you meant but it sounds like how most people think. It isn't unreliable in the sense of being defective but it is non-transactional and doesn't have the same levels of repeatability.
Except that's an impression. It often is as fast and reliable as most other RDBMSes. Most of us don't work in high transaction environments, and the ability to scale out cheaply, even at relatively low workloads, is interesting.
It's similar to what Azure is doing with the PaaS stuff. Trying to get people to think smaller databases, that are ACID compliant internally, but rollups, reports, aggregates across your shards are not necessarily consistent. Fascinating stuff.
May 12, 2014 at 6:52 am
David.Poole (5/9/2014)
Whats wrong with eventual consistency? Its not designed to be a drop in replacement for the 'C' in ACID right? Its more like the 'C' in CAP from what I remember.
It seems that the "eventual" thing is a bit unreliable. It isn't guaranteed and people have started to notice.
Are you trying to say that there is no algorithm available today to get the differences in any given list of data shared between two computers connected by a wire?
May 12, 2014 at 7:04 am
patrickmcginnis59 10839 (5/12/2014)
David.Poole (5/9/2014)
Whats wrong with eventual consistency? Its not designed to be a drop in replacement for the 'C' in ACID right? Its more like the 'C' in CAP from what I remember.
It seems that the "eventual" thing is a bit unreliable. It isn't guaranteed and people have started to notice.
Are you trying to say that there is no algorithm available today to get the differences in any given list of data shared between two computers connected by a wire?
There are but have been deemed too expensive and unnecessary for what NOSQL was originally intended for. Basically, it is a big misunderstanding with the people applying NOSQL paying too much attention to marketing and not enough to technical documentation.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
May 12, 2014 at 10:58 am
Gary Varga (5/12/2014)
There are but have been deemed too expensive and unnecessary for what NOSQL was originally intended for. Basically, it is a big misunderstanding with the people applying NOSQL paying too much attention to marketing and not enough to technical documentation.
It's not that they are too expensive or unnecessary, though that may be the decision at times.
It's that there is no way to do it at scale. If you get the volume of data that a Facebook or Twitter, or Google get, there is no way to get those machines in sync, physically. The laws of physics would be challenged, even if you were writing that data to memory.
When NoSQL says eventually consistent, it's not that they mean the data won't be the same until tomorrow, or an hour from now. They mean that not this ms. In practice, the consistency often is there sub-second, though it can be minutes when loads are high. It's the same with Service Broker. It doesn't make transactions instantaneous, but often it's so close that it might as well be instantaneous, in a practical sense, for most applications.
May 13, 2014 at 1:40 am
Steve Jones - SSC Editor (5/12/2014)
Gary Varga (5/12/2014)
There are but have been deemed too expensive and unnecessary for what NOSQL was originally intended for. Basically, it is a big misunderstanding with the people applying NOSQL paying too much attention to marketing and not enough to technical documentation.
It's not that they are too expensive or unnecessary, though that may be the decision at times.
It's that there is no way to do it at scale. If you get the volume of data that a Facebook or Twitter, or Google get, there is no way to get those machines in sync, physically. The laws of physics would be challenged, even if you were writing that data to memory.
When NoSQL says eventually consistent, it's not that they mean the data won't be the same until tomorrow, or an hour from now. They mean that not this ms. In practice, the consistency often is there sub-second, though it can be minutes when loads are high. It's the same with Service Broker. It doesn't make transactions instantaneous, but often it's so close that it might as well be instantaneous, in a practical sense, for most applications.
I agree but with regards to "for most applications" is where the professionals should know the difference. Clearly they did not at the BitCoin exchange that got ripped off.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
May 13, 2014 at 6:38 am
Gary Varga (5/13/2014)
Steve Jones - SSC Editor (5/12/2014)
Gary Varga (5/12/2014)
There are but have been deemed too expensive and unnecessary for what NOSQL was originally intended for. Basically, it is a big misunderstanding with the people applying NOSQL paying too much attention to marketing and not enough to technical documentation.
It's not that they are too expensive or unnecessary, though that may be the decision at times.
It's that there is no way to do it at scale. If you get the volume of data that a Facebook or Twitter, or Google get, there is no way to get those machines in sync, physically. The laws of physics would be challenged, even if you were writing that data to memory.
When NoSQL says eventually consistent, it's not that they mean the data won't be the same until tomorrow, or an hour from now. They mean that not this ms. In practice, the consistency often is there sub-second, though it can be minutes when loads are high. It's the same with Service Broker. It doesn't make transactions instantaneous, but often it's so close that it might as well be instantaneous, in a practical sense, for most applications.
I agree but with regards to "for most applications" is where the professionals should know the difference. Clearly they did not at the BitCoin exchange that got ripped off.
MtGox's failures didn't involve what database technology they used. There was a documented shortcoming in the bitcoin protocol that the company ignored. When they cashed out bitcoins, their erroneous use of the protocol meant that in certain cases, they cashed them out more than once. Essentially the txid (transaction id) is able to be modified by third parties, but MtGox didn't allow for this despite being a known issue since 2011, they instead relied on it to actually be immutable and trustyworthy. Developers were told not to rely on this but MtGox did and they got caught out by it.
This mistake could have been made using sql server for all that matters.
May 13, 2014 at 7:19 am
patrickmcginnis59 10839 (5/13/2014)
Gary Varga (5/13/2014)
Steve Jones - SSC Editor (5/12/2014)
Gary Varga (5/12/2014)
There are but have been deemed too expensive and unnecessary for what NOSQL was originally intended for. Basically, it is a big misunderstanding with the people applying NOSQL paying too much attention to marketing and not enough to technical documentation.
It's not that they are too expensive or unnecessary, though that may be the decision at times.
It's that there is no way to do it at scale. If you get the volume of data that a Facebook or Twitter, or Google get, there is no way to get those machines in sync, physically. The laws of physics would be challenged, even if you were writing that data to memory.
When NoSQL says eventually consistent, it's not that they mean the data won't be the same until tomorrow, or an hour from now. They mean that not this ms. In practice, the consistency often is there sub-second, though it can be minutes when loads are high. It's the same with Service Broker. It doesn't make transactions instantaneous, but often it's so close that it might as well be instantaneous, in a practical sense, for most applications.
I agree but with regards to "for most applications" is where the professionals should know the difference. Clearly they did not at the BitCoin exchange that got ripped off.
MtGox's failures didn't involve what database technology they used. There was a documented shortcoming in the bitcoin protocol that the company ignored. When they cashed out bitcoins, their erroneous use of the protocol meant that in certain cases, they cashed them out more than once. Essentially the txid (transaction id) is able to be modified by third parties, but MtGox didn't allow for this despite being a known issue since 2011, they instead relied on it to actually be immutable and trustyworthy. Developers were told not to rely on this but MtGox did and they got caught out by it.
This mistake could have been made using sql server for all that matters.
Agghhhh. Grey sources!!! I must find the article that blamed NoSQL. Either it is wrong and needs highlighting or it is right and I should let you know. I am now assuming that it is wrong.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
May 13, 2014 at 7:46 am
patrickmcginnis59 10839 (5/13/2014)
Gary Varga (5/13/2014)
Steve Jones - SSC Editor (5/12/2014)
Gary Varga (5/12/2014)
There are but have been deemed too expensive and unnecessary for what NOSQL was originally intended for. Basically, it is a big misunderstanding with the people applying NOSQL paying too much attention to marketing and not enough to technical documentation.
It's not that they are too expensive or unnecessary, though that may be the decision at times.
It's that there is no way to do it at scale. If you get the volume of data that a Facebook or Twitter, or Google get, there is no way to get those machines in sync, physically. The laws of physics would be challenged, even if you were writing that data to memory.
When NoSQL says eventually consistent, it's not that they mean the data won't be the same until tomorrow, or an hour from now. They mean that not this ms. In practice, the consistency often is there sub-second, though it can be minutes when loads are high. It's the same with Service Broker. It doesn't make transactions instantaneous, but often it's so close that it might as well be instantaneous, in a practical sense, for most applications.
I agree but with regards to "for most applications" is where the professionals should know the difference. Clearly they did not at the BitCoin exchange that got ripped off.
MtGox's failures didn't involve what database technology they used. There was a documented shortcoming in the bitcoin protocol that the company ignored. When they cashed out bitcoins, their erroneous use of the protocol meant that in certain cases, they cashed them out more than once. Essentially the txid (transaction id) is able to be modified by third parties, but MtGox didn't allow for this despite being a known issue since 2011, they instead relied on it to actually be immutable and trustyworthy. Developers were told not to rely on this but MtGox did and they got caught out by it.
This mistake could have been made using sql server for all that matters.
...When they cashed out bitcoins, their erroneous use of the protocol meant that in certain cases, they cashed them out more than once...
I know next to nothing about the technical implementation or bitcoin or even how it work conceptually, but relational databases typically use unique key constraints to prevent events like "cashing out the same bitcoin twice". Each bitcoin should have a unique id, and a table inserted with one row when it is debited for cash.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
May 13, 2014 at 8:11 am
I know next to nothing about the technical implementation or bitcoin or even how it work conceptually, but relational databases typically use unique key constraints to prevent events like "cashing out the same bitcoin twice". Each bitcoin should have a unique id, and a table inserted with one row when it is debited for cash.
You do at least know that there is no centralized database for bitcoins right? (Reading your comment, it seems this might not be the case, I keep reading folks who think that bitcoin has some sort of "bank" or federal reserve or something issuing them.)
May 13, 2014 at 8:21 am
patrickmcginnis59 10839 (5/13/2014)
I know next to nothing about the technical implementation or bitcoin or even how it work conceptually, but relational databases typically use unique key constraints to prevent events like "cashing out the same bitcoin twice". Each bitcoin should have a unique id, and a table inserted with one row when it is debited for cash.
You do at least know that there is no centralized database for bitcoins right? (Reading your comment, it seems this might not be the case, I keep reading folks who think that bitcoin has some sort of "bank" or federal reserve or something issuing them.)
Yes, I know at least that much about bitcoin, but the distributed nature of the architecture doesn't necessarily excuse the same coin from being cashed out twice. There is latency when it comes to distributed transactions, but did both cash out transactions occur within MtGox's own node?
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
May 13, 2014 at 8:35 am
Eric M Russell (5/13/2014)
patrickmcginnis59 10839 (5/13/2014)
I know next to nothing about the technical implementation or bitcoin or even how it work conceptually, but relational databases typically use unique key constraints to prevent events like "cashing out the same bitcoin twice". Each bitcoin should have a unique id, and a table inserted with one row when it is debited for cash.
You do at least know that there is no centralized database for bitcoins right? (Reading your comment, it seems this might not be the case, I keep reading folks who think that bitcoin has some sort of "bank" or federal reserve or something issuing them.)
Yes, I know at least that much about bitcoin, but the distributed nature of the architecture doesn't necessarily excuse the same coin from being cashed out twice. There is latency when it comes to distributed transactions, but did both cash out transactions occur within MtGox's own node?
Yes. Same node. It was supposedly repeats of the same transaction. A quick search online provides plenty of evidence that suggests that in the public domain no one can be certain what occurred.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
May 13, 2014 at 9:26 am
Gary Varga (5/13/2014)
Eric M Russell (5/13/2014)
patrickmcginnis59 10839 (5/13/2014)
I know next to nothing about the technical implementation or bitcoin or even how it work conceptually, but relational databases typically use unique key constraints to prevent events like "cashing out the same bitcoin twice". Each bitcoin should have a unique id, and a table inserted with one row when it is debited for cash.
You do at least know that there is no centralized database for bitcoins right? (Reading your comment, it seems this might not be the case, I keep reading folks who think that bitcoin has some sort of "bank" or federal reserve or something issuing them.)
Yes, I know at least that much about bitcoin, but the distributed nature of the architecture doesn't necessarily excuse the same coin from being cashed out twice. There is latency when it comes to distributed transactions, but did both cash out transactions occur within MtGox's own node?
Yes. Same node. It was supposedly repeats of the same transaction. A quick search online provides plenty of evidence that suggests that in the public domain no one can be certain what occurred.
The reason they were able to be repeated is because of the transaction id getting changed by a third party, and it subsequently looks like a different transaction because MtGox based their system on interpreting it like such, despite it being advised not to do so. This was on the same node, and nosql doesn't seem to have much to do with it, because only one database node is required for this bug to take place, and furthermore, MtGox is (was) an exchange, not an issuer of bitcoin, the actual issuing of bitcoin is through the distributed block chain. Obviously we don't even know this for sure, but other than the mutable transaction id, bitcoin as such doesn't seem to be broken.
May 13, 2014 at 9:47 am
patrickmcginnis59 wrote:
... other than the mutable transaction id, bitcoin as such doesn't seem to be broken.
Perhaps not in the technical sense, but it is still is subject to the restrictions placed upon it by national governments, which may decide bitcoins are not convertible into the national currency, or (in the case of Silk Road) may shut down entire marketplaces which use bitcoins as an exchange mechanism.
Something tells me technical issues will be the least problem standing in the way of bitcoin's general acceptance as a 'currency'.
Viewing 15 posts - 16 through 29 (of 29 total)
You must be logged in to reply to this topic. Login to reply