The Next Data Decade

  • Comments posted to this topic are about the item The Next Data Decade

  • I'm seeing a mixture of what you're saying, Steve. At my job we have standardized upon 1 VCS, build tools, dev tools, database technologies, etc. However, due to old methods of project management there's a really huge number of applications written by Shadow IT that trying to reign them in is very hard. I work on a team that's trying to replace them, but it's incredibly slow going, due to our having to adhere to old project management practices. At this rate I'm convinced I'll never see these shadow IT apps completely replaced and standardized.

    At this point we've only chosen one cloud vendor. I'm not sure why (above my pay grade), but that's the way we've gone.

    Your point about showing "... a lot of value to whoever pays you" is good. I can see you. However, I also see that once you start management sees you as good at X, they'll often only think of you as good at X. I'm sure that's true almost everywhere; indeed I'd like to be proven wrong.

    I am continuing to learn. In fact this additional time I've been able to add a new skill to myself - Python. I've finished a course on Python, on LinkedIn Learning. And I've also started working with Jupyter Notebooks, something I've never done before. I would really appreciate learning more about Jupyter Notebooks. Was there a stairway series on Jupyter Notebooks here on SSC?


  • I think it's really hard to justify replacing many shadow IT apps. They exist for a reason, some need wasn't met (fast enough, or at all) and once there, it is hard to spend more to make it official. More, what I've tried to do often is teach people that build these to use better practices where I can. That makes them build better software.


    I've never had a problem being the person doing a variety of tasks. I show someone I'm good at X, but I later show them I can do Y as well, or Z. Adding to your skills should mean showing others that you are adding. Sometimes they won't care or be impressed, but the continued variety of skills I've built have continued to help me.


    No Stairway series. I did a couple articles on ADS with notebooks. They're useful, and nice for doing something in a series without having to use some separate IDE. I like that they keep results. If you have more requests on things that seem hard, let me know. Or better yet,  figure it out and write me an article.

  • Thank you Steve, for your reply. Here's my response in turn. 🙂

    Your point of view about Shadow IT is interesting. I've never thought of that. I certainly believe the majority of these apps were written because IT couldn't get to them, for whatever reason. For us, the problem, which I didn't mention before, is all of these Shadow IT apps are written using old versions of Microsoft Access. All dependent upon Windows 7, or older. We've got to move away from Windows 7, thus dooming those apps all to need replacing or some other solution (VM's anyone?)

    I guess, for me, it's knowing when to bring up something like, "Hey boss, I'd like to do....". I'm poor at judging that time and that is my problem. Thank you very much for suggesting it.


  • Will be interesting to see what happens in the data warehousing space in the next decade. With the rise of Snowflake versus Azure versus AWS versus whatever, it will be interesting to see if Snowflake can out perform or even out invest the competition or if others will rise from the ashes to boot.

    It will also be interesting to see what happens in the visualization side of things too. Salesforce acquiring Tableau and other platforms like Datorama. What will Power BI, Data Studio, Amazon and other older platforms like MicroStrategy all do in the coming years too.


  • Older apps are always an issue. That's not a shadow IT issue. I've seen many from IT departments. Heck, we see the issues with New Jersey's Cobol system. Not real interest in replacing it. Same for many apps. VMs are great ways to extend the life of things here, especially with older host tech.

    For tackling other things. sometimes I ask, sometimes I work on it in my own time and show I can make something work better.

  • For the DW space, I do wonder what happens. I think what Snowflake does is great, and they've solved some of the dev/test/copy issues with size/space for multiple users. I expect different companies to adopt similar things (or license this).

    I'd also be surprised if Snowflake goes too long without being purchased. Looking at what happened with Docker, I'd think they'd be wary of trying to take the large vendors on for too many years.


Viewing 7 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic. Login to reply