Shadow IT

  • I am currently a Central IT guy....but in the last 4 years I have been a Shadow IT guy as well. The Shadow IT areas are usually so much more responsive and agile that they get tasked with critical needs first, taking on more and more tasks until they require their own instance of SQL Server, then their own hardware, then a request management system and a web portal, reporting servers etc etc...until WHAMO! your shadow IT guys are now Central IT guys, buried in beaurocracy and process to the point of being too slow to be of any real use to the business. The business will start quietly sending requests to a new shadow IT guy or team, who will get more and more work because he/she can really crank it out fast until they need their own instance of SQL Server, their own hardware.....and repeat.

    I've been through this cycle twice in the last four years, I work at a large (250k+ employees) global company.



    SQL Tips and Scripts
    SQLWorks Blog

  • Been central IT all of the years I have worked. I have seen the shadow IT employee at work, and at times they are the best thing in the world to assist the user. At other times they have been cowboys off on a journey all their own.

    Are they still around, yes and as long as they have tools to do anything they will do it. It is the nature of the beast.

    Not all gray hairs are Dinosaurs!

  • I have to agree with Anthony...I've seen this same cycle many times in my organization. Though for some reason I keep being held back from migrating to IT as "a valuable resource that we do not want to relinquish to IT"...which sounds fantastic, but makes me cringe as someone who looks forward to the scructure of an established IT department!

  • Are they still around, yes and as long as they have tools to do anything they will do it. It is the nature of the beast.

    Agree...Users will do whatever is technically possible, if you dont want it done, remove the access. Anything short of making it impossible might as well be nothing at all.



    SQL Tips and Scripts
    SQLWorks Blog

  • I'm with the others on this, Steve. Shadow IT is not the problem. The centralization of *all* IT is.

    IT is still too heavily lumped together. Hang on, I've got my soapbox around here somewhere... *blows the dust off it*.

    A year or two ago I put together that article about the division of skillsets JUST in the database world. Admins, developers, reporting people, architects, etc.

    Split that out to programming, web design, computer artists, AI specialists, etc... oh gods. I have no idea why anyone would think that the central IT department 12 states (or a country) to the left will have any idea how YOUR department does its workflow, makes its revenue, or enjoys its 8 AM happy hour. Central IT needs so much information just to get started, it's completely stifled. The devil's in the details.

    Those details are local. Instead of 'Shadow IT', let's call it 'Site IT', or better, 'OnSite IT'. IT is a monolith. It's like saying someone works in 'finance'. That covers accountants, stock brokers, oil traders, prediction analysts, interest expectation consultants, etc... Errrr, would we expect THEM to handle the books for the ENTIRE company, or do we expect departments to work with someone local, then transfer the results up?

    So, what pros does a Central IT team have? Well, what they do is standardize the working environment. Administrate it, keep it up to date, make sure some baseline practices are involved (backups, etc). They should be available to consult with Site IT to make sure those baselines are kept in practice to help protect Site IT from themselves, particularly the noob with the excel sheet trying to make it available to his department and that database idea thingie sounds good. After that... LET THEM LOOSE.

    If a process, app, or structure is company wide, it should be centrally maintained as that's where the majority of the knowledge base for the application is. If something is department wide, it should be locally maintained (if lightly administrated centrally) by local assets, as that's where the majority of the knowledgebase for the application is. There's a flipside to that, of course, and that's duplication of work-effort. There should be a central IT person (or two) always availble for 'quick-consults' to any department to do company-wide research to make sure that it doesn't have to occur. That's expensive, though, for a 'non-work producing employee'. Besides, we all know OUR way of doing business isn't THEIR way so we need OUR customizations! Sometimes it's just keeping the work flowing, without interrupting 40-50 employees in the way they do their work by forcing them to switch the local workflow. Sometimes it's accurate, they require special customizations that would have to be done locally anyway.

    In the end, any argument I hear for 'centralize everything' sounds to me like 'I don't trust anyone else to do their job well enough'. Well, good on you, don't. It just means that responsibility has to come with the authority. Heinlein said this best, at one point in the *BOOK* Starship Troopers.

    Paraphrased: Handing responsibility to someone without giving them authority is similar to asking a person to stop a train with just their body. They have no reasonable access to be able to fix an issue. Vice Versa is an equivalent issue, where unchecked authority with no responsibilty of the repercussions creates unreasonable expectation states. He used some political financial systems as reference which I'll leave out of this, they'll derail us horribly.

    Central IT needs to have both responsibility and authority to keep the backbone running and to implement the standards of the backbone. Apps and software that are corporate wide need to be in their purview, and they are the 'goto' people. They should also be available to assist local site projects. Site IT should be responsible and authorized (YES, that means you give away near-SA rights to THEIR app equipment) to handle their equipment from there with assistance as required from central IT. Perhaps not on production due to audit requirements, but they need to be able to handle 95% of their own responsibilities with authority.

    Anything less and a company is trying to enter an a$$ kicking contest with one leg tied to their shoulders. It needs to be balanced in larger corporations, or the company will suffer in one way or another.


    - Craig Farrell

    Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.

    For better assistance in answering your questions[/url] | Forum Netiquette
    For index/tuning help, follow these directions.[/url] |Tally Tables[/url]

    Twitter: @AnyWayDBA

  • We are "central IT" but the teams are split around the various business sites so also "local IT" but all part of the IT department and we have developed one nice "compromise" to shadow IT (non IT department) by building and training a team of report writers from the business areas in SQL and SSRS. They are not part of IT but work closely with IT who manage the reporting databases and servers, data imports etc and provide the tools. IT developers are freed up for application development and SSIS work whilst the report writers know the workings of the business better and are now keeping us on our toes with their SQL skills!

    For one section of the company that was merged in and who are really shadow IT, we supply and top level manage the server leaving them to do the rest. We do backups and patches and regularly have to restore a database when something gets messed up, but it's being able to do those restores that gives them the confidence in and need for central IT as their SQL skills don't go as far as admin.

    The key is to work with shadow IT to build a relationship that gives them a degree of freedom whilst retaining central IT's overall management.

  • Evil Kraig F (8/28/2012)


    In the end, any argument I hear for 'centralize everything' sounds to me like 'I don't trust anyone else to do their job well enough'. Well, good on you, don't. It just means that responsibility has to come with the authority.

    Often this argument is the result of various Shadow IT (SIT) people of the "Blind Spot" type creating horrific, undocumented monstrosities of the "manually change X, Y, Z, X1, Y1, Z1, run that, manually change X2, Y2, Z2, X3, Q1, Z3, unless it's the second tuesday, in which case (unless the second tuesday is the 8th, in which case [unless it's the first tuesday of the second quarter], in which case)..." type, and then that SIT person moves on internally or externally... and the department screams at Central IT (CIT) to take over the monstrosity. This almost always happens without that department providing CIT with the budget the department had been spending on that SIT person, much less with the amount of budget CIT might actually need to do the task given the organizational, legal, and regulatory rules CIT follows.

    This often leads to the conclusion that either all SIT positions need to be formalized (i.e. if department A loses a SIT person, department A needs to hire/obtain another one), or any SIT work that could be shunted to CIT if the writer/maintainer leaves needs to be in CIT to begin with.

  • Nadrek (8/29/2012)


    Evil Kraig F (8/28/2012)


    ...creating horrific, undocumented monstrosities of the "manually change X, Y, Z, X1, Y1, Z1, run that, manually change X2, Y2, Z2, X3, Q1, Z3, unless it's the second tuesday, in which case (unless the second tuesday is the 8th, in which case [unless it's the first tuesday of the second quarter], in which case)..." type...

    Have we been working on the same projects?!?

    But then, I'm just a "Dark Knight" stalking the shadows of CIT keeping an eye out for Blind Spots to mitigate their impact. I tend to be one of those SIT guys that roots/takes out shoddy Blind Spot activity or mentors Blind Spots toward Dark Knight status if they have the aptitude.

  • Nadrek (8/29/2012)


    Evil Kraig F (8/28/2012)


    In the end, any argument I hear for 'centralize everything' sounds to me like 'I don't trust anyone else to do their job well enough'. Well, good on you, don't. It just means that responsibility has to come with the authority.

    Often this argument is the result of various Shadow IT (SIT) people of the "Blind Spot" type creating horrific, undocumented monstrosities of the "manually change X, Y, Z, X1, Y1, Z1, run that, manually change X2, Y2, Z2, X3, Q1, Z3, unless it's the second tuesday, in which case (unless the second tuesday is the 8th, in which case [unless it's the first tuesday of the second quarter], in which case)..." type, and then that SIT person moves on internally or externally... and the department screams at Central IT (CIT) to take over the monstrosity. This almost always happens without that department providing CIT with the budget the department had been spending on that SIT person, much less with the amount of budget CIT might actually need to do the task given the organizational, legal, and regulatory rules CIT follows.

    This often leads to the conclusion that either all SIT positions need to be formalized (i.e. if department A loses a SIT person, department A needs to hire/obtain another one), or any SIT work that could be shunted to CIT if the writer/maintainer leaves needs to be in CIT to begin with.

    That's a bit of a broadcloth generalization. I've been in the field long enough to see noodle code generated from either version of IT professionals. Lately I've been seeing it a lot more from CIT, since they centralize everything but still don't have the appropriate controls in place to actually figure out or care what it is they're centralizing. That said - I will NOT toss either side under the bus.

    Some organizations just simply won't function under a centralized IT department. In a previous healthcare provider organization, the shadow IT departments were the only way individual functional departments could get anything done, simply because the arguments for IT resources never seemed to measure up to healthcare equipment expenditures (physician-led organizations will ALWAYS pick better diagnostic gear over better IT systems it seems, even if the IT system might actually have a larger impact on patient care).

    That said - neither side should put on the blinders that the other be present.

    ----------------------------------------------------------------------------------
    Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?

  • Nadrek, I agree that poor hiring practices for local departments can get unqualified or semi-qualified people into local IT positions (for example, what does a doctor really know about a database or programming professional?) and can lead to a centralized concept.

    I believe local/Site IT should be vetted by the central core, so that these blind spots don't occur. I just disagree with the idea that everything can be done centrally without an extraoradinary volume of work needing to occur, to the point that it can stifle the company.


    - Craig Farrell

    Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.

    For better assistance in answering your questions[/url] | Forum Netiquette
    For index/tuning help, follow these directions.[/url] |Tally Tables[/url]

    Twitter: @AnyWayDBA

  • The vast majority of individuals I've run into in my organization would be catagorized as SIT fell into the position by chance. They were part of the business unit trying to solve a problem, came of with a technical solution and owned it from that point on. As their local reputation increased along with their self taught skills they were tasked with more projects until they may have earned the label of Shadow IT.

    It's certainly how I got my start...

  • I don't have a problem with either approach but problems comes when there is overlap between the two.

    In effect you end up with Central and Shadow operating as if they work for separate companies. This is fine until a business reorganisation takes place and you get all the problems you face for any merger/acquisition.

    The worst case scenario is when you get the two with autonomous budgeting powers and no single oversight resulting in incompatible systems or failure to capitalise on economies of scale.

    Shadow IT tends to apeal to business units who want agility at all costs. In effect they run an adhocracy. Great for a young entrepreneurial start up and it allows "capture the flag" operation. Where shadow IT starts to fail is where you reach the limits of an adhocracy and the bureaucracy becomes a necessity rather than a painful overhead.

    I have seen DBAs run a shadow IT department producing software that the business loved but the net result was an IT civil war.

    To get the best out of shadow IT you do need some sort of agreed standards for development otherwise you end up with little silos of incompatible solutions.

  • I got a call yesterday from one of the "Shadow IT" folks at the company I work at. They had done a "copy" from all but 4 rows of the entire spreadsheet and couldn't figure out why it was producing an error about there being more than 4000 types of formats in the "paste" to another spreadsheet. The spreadsheet had just 4 rows less than a million and lord knows how many columns.

    Having been a "Shadow IT" in many of my roles, I'm the first to embrace what "Shadow IT" does and the value they bring to the table. They know their work far better than anyone in IT and have been known to come up with some incredibly clever ways to do things especially with data.

    The problem is that many of these people take their adopted role a little too literally and never bring anything to "THE table". The people they try to help think of them as infallible "data gods". I'm not sure if it's pride, arrogance, ignorance, shear stupidity, or a combination of many factors but a lot of the "Shadow IT" don't realize when they need to ask for help (like did you even think about a backup for that mission critical spreadsheet you just screwed up???) or take things to the next logical level (a database table or two to drive a report generator really will be better than a million row spreadsheet).

    There are, of course, those who know when to get other folks involved and those are the people who are the truly beneficial "Shadow IT" that I embrace.

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

Viewing 13 posts - 16 through 27 (of 27 total)

You must be logged in to reply to this topic. Login to reply