≡ Menu

Why IT Magic is Never Good

I recently spoke at a conference attended by senior business and IT people from accounting firms. I described my usual view on IT Magic: that when Information Technology gets too complex, it’s perceived as magic; then there are lots of problems for IT because business people develop unrealistic expectations. In the Q&A with attendees, one of the audience members asked an interesting question, “Is there ever a situation when IT magic is good?”

In the conversation that followed, it became apparent that this particular questioner — the head of IT for an accounting firm — is so trusted by his business partners that they blindly turn all of the IT direction over to him. He doesn’t bother to explain to them why certain technologies are chosen, why he uses the level of infrastructure security that he does, or how his software and outsourcing choices contribute to the success of the firm. He is apparently given carte blanche to do whatever he thinks needs to be done, and he lets his IT work become magical to his business partners. Now he was describing his situation and asking me, “What’s wrong with that?” And you may feel the same way.

Trust is Good
Let me separate out two different things that are going on with this IT leader. First, he has been highly successful in getting the trust of his business partners. That is undoubtedly a good thing. Trust means that he is respected by the business leaders of his firm, and they know that he has their best interests at heart. It probably also means — at least I hope so — that he participates in high-level strategy discussions on the direction of the firm, the firm’s weaknesses and its plans to strengthen them, and the firm’s strengths and its plans to leverage them. All this high-level communication puts him in a good position to put together a complete and appropriate IT strategy for the business, and to align IT direction with that strategy.

So far, so good. I applaud his ability to gain and earn business trust. I’ve been in that position myself a few times over the years, and I know how nice it is to be trusted enough that you don’t have to plead for project money.

But Magic is Bad
On the other hand, I’m very concerned about his emphasis on magic and even wizardry (the perception that IT people are wizards because of the magic they can provide). I feel that with trust comes obligation — not just the obligation to act in a trustworthy manner, but also the obligation to be transparent.

Transparency is kicked around a lot lately in politics, but to me transparency means that people have a certain level of visibility into what you’re doing with your time and with the money that you’ve been allocated. It doesn’t mean that they have to watch over your actions on a day-by-day basis — it just means that they can see traceable evidence of your progress, and they constantly get positive reassurance that you’re doing the right things.

Transparency of Complexity is Important Too
But there’s another aspect of information technology transparency that people tend to forget.  Business processes that involve information are complex, and information systems are often put in place to simplify things for the business user.  But the original complexity of the business process doesn’t disappear — it’s just passed from the business user to the IT organization.  This follows Larry Tesler’s Law of Conservation of Complexity:

“Every application must have an inherent amount of irreducible complexity. The only question is who will have to deal with it. “

You may remember that Canon used a similar slogan a few years ago in advertising their cameras.  They said that a Canon camera was “so advanced, it’s simple.”  In essence, Canon was claiming that because the Canon engineers successfully dealt with the complexity of taking digital photographs, the camera user shouldn’t have to deal with that complexity.

Often the business forgets how much inherent complexity has been passed from the business user to IT.  And if the IT organization isn’t careful, that forgotten complexity turns into the perception of magic, including the feeling that the business user is getting something for nothing.  The ongoing effort required in IT to support the hidden complexity is forgotten and taken for granted, and business people begin to wonder why the Information Technology organization is spending so much.

Magic versus Transparency
To me, magic and transparency are opposites. Magic means that things happen without a good explanation.  Transparency means that things happen, but that you’re aware of how they’re being done, and you can appreciate both the things themselves and the work that went into them. Transparency shows effort, progress and result. Magic shows only result, and makes you wonder how it was accomplished.

Now again you may be wondering, “Why is it bad to hide the effort and progress? Why is transparency a good idea?”

Why Transparency is Good
Here’s why: Because sooner or later, the business is going to experience financial obstacles which cause it to reevaluate its expenditures. Or maybe the business will be acquired, and new management will question whether to keep the old IT or replace it with IT from the parent company. In those situations, all areas of business expense will be reevaluated. Budget cut-backs may ensue, or whole departments may be eliminated, combined or outsourced. If your department has relied on magic for its good reputation, then you’ve got a good chance that your department will be cut back. Senior management will be looking for cuts which will maximize their savings-to-loss ratio. And if your IT department hasn’t been transparent about where business investment has gone, then it will be very difficult for you to explain how budget cuts will affect the business.

On the other hand, if you’ve been transparent, then it ought to be easy to say something like this:

“Yes, we can cut the IT budget 10%. It will have the following impact on the business: These projects will be delayed [and you name them]. The support levels for these applications will decline [and you give specifics]. These key business systems will run more slowly because of decreased network capacity [and you name the systems and give your estimate of slower performance]. The probability of a system outage or data security breach will climb from x% to y% [and you give specifics and provide the logic behind your calculations]. And the impact of all this on the business will be [and you give specifics on how these changes will affect the business].”

With transparency, you can do that. With transparency, you can easily show the impact of budget changes (either down or up) because you are accounting for the resource you’ve been allocated.

And Why Magic is Bad
But if you’ve chosen to use magic instead — if you’ve deliberately hidden your hard work and taken credit as a wizard who can do anything — then woe be unto you. Because wizards who can create something great out of minimal resources are expected to be able to reduce their resources without impacting their great results.

And that’s what I told the questioner who asked about good magic: IT magic is never good, and if you insist on using magic — on deliberately avoiding transparency in your IT work — then sooner or later it will come back and bite you in the butt.  Because sooner or later the perception of magic won’t be enough; you’ll need to actually be able to tie specific parts of the IT budget to specific deliverables and services.  And if you can’t, then your days as a successful IT magician or wizard will be over.

Comments on this entry are closed.

  • Chris September 14, 2010, 1:20 pm

    Well, aside for the regard for perceived level of effort, too much “magic” too many business rules can be detrimental to the usability of the application.

    You know there’s too much business logic when regular users start claiming there’s a “ghost in the machine” or that the application “has a mind of its own”. My philosophy now is to not create interfaces that “outsmart” users, but instead guide them.

    For me, I want transparency in the interface because I want the end user to be aware of the entirety of the process. If creating an object creates other objects the user should know. For example, creating a user creates a profile and a profile image. By enforcing transparency the end users can make better decisions about how to extend functionality, and also generally provide avenues within the interface to bypass business logic in situations where it shouldn’t apply.

    • Harwell September 14, 2010, 1:42 pm

      I agree. There’s a story I tell in my book about a Material Requirements Planning (MRP) system that was so complicated that some of its users basically gave up trying to understand how it worked. They just accepted the results whether they were right or not, and that got them into a lot of trouble. We solved the problem by using paper-based training where we took the users through the process used by the MRP system, having the users fill out pieces of paper that corresponded to the various screens and reports from the system. By using the paper-based exercise we were able to show the users that the system wasn’t doing anything magic; it was doing the same calculations that they would do — just doing them faster.
      This is the kind of system transparency you want: the users understand what the computer system is doing, and therefore know how to sanity check the result.
      And as you say, Chris, this is often made much easier by using simpler systems without the feature bloat that just gets in the way of user understanding of the system.
      Microsoft, please pay attention! You crossed the line of too much complexity a long time ago.

  • HonestNick September 14, 2010, 2:06 pm

    Yeah, that would all be well and nice, if it wasn’t for the fact that people are fucking clueless. And you “transparency” is a short for the reason why the whole industry of IT/IS is a fucking mess. We invited clueless dumb fucks into our room and allowed them to share their opinions on how our room should be decorated.

    • Harwell September 14, 2010, 5:57 pm

      You’re certainly entitled to your opinion, Nick. But if I understand you correctly, you’re saying that IT is “our room” and no one else should have a say.
      If that’s what you mean, then I should point out that “our room” is part of their house (i.e., IT is part of the business). And the whole reason for IT is to provide strategic advantage to the business, as well as basic infrastructure services that could easily be outsourced.
      I absolutely agree that far too many business people try to get IT to implement “solutions” the business has already chosen instead of working with IT on the best way to solve a business problem. But IT organizations cause just as many problems by refusing to participate in business strategic planning and turning the IT organization into a bunch of people who just follow orders from the business (see my article, “IT Governance is Like Pushing a Rope” or my article, “Driving Information Technology — Is the CIO Just a Chauffeur?“).
      The bottom line is that an adversarial relationship between IT and the business does no one any good. And arbitrary criticism of business people is not a good way to build a better relationship.

      • HonestNick September 15, 2010, 2:57 pm

        And see, this is where all of a sudden we talk politics rather than tech.
        You immediately see “mine vs. yours” as a matter of property. I see it as a matter of expertise.

        No, people outside the IT doesn’t understand the environment. They may have strong opinions about how a web server should be configured. But they don’t know how their wishes factor into the other 99.9% of the organization. But stating this very obvious point means we aren’t team players. So in order for us to become team players we have to listen to and take into account the arguments and logic of technical retards.

        • Harwell September 15, 2010, 5:40 pm

          Thanks for your comment, Nick. You’re obviously frustrated, and I’m sorry. I’m still not sure I understand your original comment or the follow-up comment. It sounds like my reply wasn’t addressing your real concern, so please help me get a better understanding of your problem.
          It sounds like you’ve got people outside the IT organization who are pushing specific IT solutions on you (e.g., how to configure a web server). This sounds like the roles of IT and the customer have been blurred, and your “business” people are crossing the line.
          One of the points I make in my book (and in many of the articles on this web site) is that the business people need to define their needs in business terms and then let the technology experts in IT define exactly how those needs will be met. But this doesn’t sound like it’s happening in your IT organization, so the key question is “why?”
          Do you not have responsibilities clearly defined on who is supposed to do what? Is your manager or CIO not supportive on maintaining clarity of responsibilities? Where is the process breaking down?
          Give me more information about where the problem is, and I may be able to steer you toward a solution. And it sounds like my book will help. It’s only $9.99 when you buy it through Amazon Kindle, and you don’t even need a Kindle to read it — you can use free Kindle reader software for PC, Mac, Android, iPhone, etc. I’m not pushing my book on you as a marketing thing. I honestly think some of the information in it might help you, especially if you can get your manager and the manager of the offending business users to read it.
          But whether you choose to get the book or not, please clarify your problem in a follow-up comment, and I’ll try to help you. I’d respond by email but you didn’t leave an email address, but if you prefer to do this via email, send me email via info at makingitclear.com

  • Bill September 14, 2010, 3:19 pm

    I am starting a company called IT Magic, are you saying I am doomed? :|

    • Harwell September 14, 2010, 3:44 pm

      Sorry, Bill, but in my opinion, yes, a company called “IT Magic” makes as much sense as a company called “Perfect Used Cars” or one called “Free Energy.” The phrase “IT Magic” raises false expectations.

  • Bob September 15, 2010, 1:06 am

    I agree with his take on transparency… but professionals in a highly technical field should not feel obligated to ‘dumb down’ what is in many cases a highly complex undertaking. This leads down the road of ‘well it’s not that hard’ or ‘Just do the bare minimum’ or ‘I’m sure a bunch of junior developers from India can do just as good a job’.

    It’s not complicated… it’s complex – so take our word for it and please quit trying to understand the details.

  • Piter September 15, 2010, 1:07 am

    Please quit trying to understand the details.

    He’s not advocating that you need to expose the complex details. You just need to show at the level of resource allocation where those resources are going, because it’s just a hard fact of corporate life that sometimes a budgeting decision will get mandated onto you whether it’s by a self-serving executive trying to boost his bonus by reducing costs or out of necessity by the harsh realities of an economic downturn.

    Wouldn’t you rather whoever made that decision that you now get a lower headcount or a lower operating budget understand the consequences of making it? Hell, if you’re transparent enough and can justify your resources you might even manage to avoid losing some of them in the first place; or at the very least you’ll be freed from some obligations at the same time your budget is lowered.

  • Serra September 15, 2010, 1:07 am

    I’ve found that the difficulty comes from being as informative as possible while attempting to not sound condescending. What makes this rather harrowing at times is that the definition of “condescending” varies from person to person, so unless you’ve worked with someone before, you can’t be sure whether you’re unintentionally stepping on their toes.

  • Markos September 15, 2010, 1:15 am

    Good points. Magic can be good in securing your job. You can’t be easily replaced by someone cheaper if you provide good magic. :-)

  • Stephan Schmidt September 15, 2010, 1:23 am

    I’ve written a blog post on a similar topic, because I agree with you.

    “Be careful with magical code”
    http://codemonkeyism.com/beware-magical-code/

    Best
    Stephan

  • Adam September 15, 2010, 4:13 am

    Circular arguments.
    Why marketing Magic is Never Good
    Why financial Magic is Never Good
    Why production Magic is Never Good
    Do we have to know everything? No, we need clear results from other participants of the process. And they need clear expectations from us. We cant tell anyone how she should work but only what we need.

  • Harwell September 15, 2010, 6:53 am

    More than 50 additional comments on Reddit