The Cloud as a Foundation

Question

The Cloud as a Foundation

Steve Jones - SSC Editor

SSC Guru

Points: 734449
More actions
February 9, 2022 at 12:00 am

#3980799

Comments posted to this topic are about the item The Cloud as a Foundation

Viewing 5 posts - 1 through 4 (of 4 total)

You must be logged in to reply to this topic. Login to reply

Jeff Moden SSC Guru Points: 1003863 More actions · Answer 1

I think that the cloud has some fantastic capabilities and opportunities. Despite some of my posts, my big concern isn't so much about the cloud, it's about the "shiny new object" syndrome that a lot of people seem to suffer.

For one company that I work for, they had a developer make a fascinating tool using Python and a couple of other "hooks" to import data data into SQL Server. They worked on it for quite a while and with a little more work (moth or two?), it'll be done. What's my concern there? Basically, the rewrote BULK INSERT in Python. <headdesk><facepalm> 😀

This reply was modified 2 years, 2 months ago by Steve Jones - SSC Editor.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

skeleton567 SSCertifiable Points: 5672 More actions · Answer 2

I think I would have reservations regarding entrusting my data to a platform called 'Snowflake'. And maybe even more so to a platform called 'Cockroach'.

Rick
Disaster Recovery = Backup ( Backup ( Your Backup ) )

Eric M Russell SSC Guru Points: 125522 More actions · Answer 3

".. Snowflake is something that a company can run on AWS, Amazon, or GCP .."

Maybe you meant to say Azure instead of "Amazon", but Snowflake can be hosted in Azure as well. We're actually researching Snowflake as a potential alternative to our current EDW in SQL Server (and maybe Azure SQL Analytics is an option as well). I'll be involved in a proof of concept implementation this quarter.

When comparing database platforms, especially NoSQL or cloud native options, one thing to consider is that most are optimized for specific data access patterns, and the underlying architecture excels for some use cases - while being very unsuited for others.

For example, Snowflake doesn't create indexes. Instead it splits up the data into 10 MB "micro-partitions" and clusters the data horizontally across distributed nodes. This is all done auto-magically, and that is it's big selling point. Little to no database administration as we know it (physical data modeling, configuration, etc.) is required.

I'm just now getting my feet wet, but apparently it's optimized for extremely fast bulk ELT and OLAP or data science workloads using ANSI SQL or Spark, but not recommended for application (ie: select for customer 123) type queries. So, the trade off is that it's not as flexible as SQL Server. My assumption at this point is that if Snowflake is used as the core EDW, then it will serve as a hub, and various application centric datamarts will need to spoke off from it. The datamart or microservices layer, that is where SQL Server, Azure SQL, or Azure CosmosDB would be better suited.

"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho

Steve Jones - SSC Editor SSC Guru Points: 734449 More actions · Answer 4

Eric M Russell wrote:

".. Snowflake is something that a company can run on AWS, Amazon, or GCP .."
Maybe you meant to say Azure instead of "Amazon", but Snowflake can be hosted in Azure as well. We're actually researching Snowflake as a potential alternative to our current EDW in SQL Server (and maybe Azure SQL Analytics is an option as well). I'll be involved in a proof of concept implementation this quarter.

Good catch. Yes, I meant Azure and edited the piece. Snowflake works everywhere.