February 9, 2022 at 12:00 am
Comments posted to this topic are about the item The Cloud as a Foundation
Follow me on Twitter: http://www.twitter.com/way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
February 9, 2022 at 3:19 am
I think that the cloud has some fantastic capabilities and opportunities. Despite some of my posts, my big concern isn't so much about the cloud, it's about the "shiny new object" syndrome that a lot of people seem to suffer.
For one company that I work for, they had a developer make a fascinating tool using Python and a couple of other "hooks" to import data data into SQL Server. They worked on it for quite a while and with a little more work (moth or two?), it'll be done. What's my concern there? Basically, the rewrote BULK INSERT in Python. <headdesk><facepalm> 😀
--Jeff Moden
February 9, 2022 at 2:49 pm
I think I would have reservations regarding entrusting my data to a platform called 'Snowflake'. And maybe even more so to a platform called 'Cockroach'.
Rick
One of the best days of my IT career was the day I told my boss if the problem was so simple he should go fix it himself.
February 9, 2022 at 6:44 pm
".. Snowflake is something that a company can run on AWS, Amazon, or GCP .."
Maybe you meant to say Azure instead of "Amazon", but Snowflake can be hosted in Azure as well. We're actually researching Snowflake as a potential alternative to our current EDW in SQL Server (and maybe Azure SQL Analytics is an option as well). I'll be involved in a proof of concept implementation this quarter.
When comparing database platforms, especially NoSQL or cloud native options, one thing to consider is that most are optimized for specific data access patterns, and the underlying architecture excels for some use cases - while being very unsuited for others.
For example, Snowflake doesn't create indexes. Instead it splits up the data into 10 MB "micro-partitions" and clusters the data horizontally across distributed nodes. This is all done auto-magically, and that is it's big selling point. Little to no database administration as we know it (physical data modeling, configuration, etc.) is required.
I'm just now getting my feet wet, but apparently it's optimized for extremely fast bulk ELT and OLAP or data science workloads using ANSI SQL or Spark, but not recommended for application (ie: select for customer 123) type queries. So, the trade off is that it's not as flexible as SQL Server. My assumption at this point is that if Snowflake is used as the core EDW, then it will serve as a hub, and various application centric datamarts will need to spoke off from it. The datamart or microservices layer, that is where SQL Server, Azure SQL, or Azure CosmosDB would be better suited.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
February 9, 2022 at 6:59 pm
".. Snowflake is something that a company can run on AWS, Amazon, or GCP .."
Maybe you meant to say Azure instead of "Amazon", but Snowflake can be hosted in Azure as well. We're actually researching Snowflake as a potential alternative to our current EDW in SQL Server (and maybe Azure SQL Analytics is an option as well). I'll be involved in a proof of concept implementation this quarter.
Good catch. Yes, I meant Azure and edited the piece. Snowflake works everywhere.
Follow me on Twitter: http://www.twitter.com/way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply