Joris talks to an expert in banking systems about the competitive spirit – and the time when he thought he was doing good
• This monologue is part of a series in which people across the financial sector speak about their working lives
“We need to talk about the huge risks banks are running with their IT operations,” he wrote in to the blog. An inconspicuous man in his early 40s, this former IT salesman for systems used by banks has an easy laugh. He orders a cappuccino.
“Here’s a true horror story. Global corporation X is storing all of its crucial data in encrypted form. To unlock the encryption you need special keys, which are stored in one central place, on one computer.
“We sold them that system assuring them it was safe. One Friday afternoon, a new employee at X needs some extra capacity and notices this one computer doing nothing. He thinks it’s idling so he copies its contents to a temporary file, and lets the computer run whatever he needs it for. Then he tries to copy the contents back, which is impossible with encrypted files and this is how he discovers what he’s done: he has effectively erased the system that underpins all of X’s global operations. Their data are still there, but encrypted and the keys are lost.
“Panic ensues, and my colleagues who installed the software jump on the first plane to X’s headquarters. There they discover just how lucky company X has been. The installation was relatively recent, so our people had a good grasp of its details. They went through the system and thank God, the switches had not yet been reset, meaning the keys could be retrieved. If X had rebooted its systems all would have been lost.
“That was a really good outcome, but it demonstrates the risks with new advances in computer technology. Imagine our people had moved jobs or company X had rebooted the system. It sounds unbelievable but imagine a bank lost all of its data. It would make the panic during a bank run look innocent.
“There were several reasons for me to quit at my software company, but in the end this clinched it; as a sales guy I was supposed to push an encryption system while I couldn’t name even one company with the processes and procedures in place to use such complex technology safely. Sure, you have teams as well as individual employees in companies able to manage the technology, but these tech-savvy people are never high enough in the company’s hierarchy. What happens time and again is that those who have the expertise get overruled by those who don’t.
I just couldn’t bring myself to sell this product, knowing what I did. As I said, the vulnerabilities arise not from the system itself but from the organisational reality it will be used in. It’s like selling a car to a 12-year-old.
“There were other reasons I quit, and most of them actually feature on your blog. Going over the interviews I was struck by how similar the world of IT and finance are. The incentive structure, insane hours and warped world view, the machismo, zero job security and the sense of a bubble. Pay is quite similar with finance too, at least for those under 35.
“The competitive spirit does something to you. There was the time I came out of a sales pitch to find a colleague from a competitor weeping in the parking lot. ‘Please don’t steal this client or I’ll get fired’ he begged me and all I remember is thinking ‘yes, yes, yes we’re gonna win this account’. And we did. It’s amazing how you can get caught up in that bubble, I understand entirely how the same happens to bankers.
“At some point the entire global sales force was flown to Las Vegas. The best performers would be invited to come on to the podium to be honoured. But first we were shown about 10 minutes of a war movie, when this guy stepped forward and began shouting: ‘We are gonna rip the skin off competitor X!’ And we’d be expected to go, fists pumping in the air, ‘yeah!’ Next he shouted: ‘we are gonna fuck over competitor Y’ and we’d go ‘yeah!’ The world of finance is not alone in its excesses. But IT hasn’t had its comeuppance, not yet.
“In part I blame the American mindset that took hold in my company after 2000. It became all about targets, every problem was a ‘challenge’ and criticism became taboo. We are a team, remember, we are gonna rip the competitors’ skin off! We’d be working on a system due to go live in a week and discovered a serious flaw. But if you flagged it up, you’d be blamed for lack of motivation, for ‘not thinking in solutions but in problems’ blah, blah. Why this matters is that systems have become so unwieldy. Once you turn a system on, it becomes part of the IT ecosystem, as we call it, and the interdependencies are tremendously complex. It becomes next to impossible to turn the system off again. You are stuck with it, and changing things is like replacing an engine mid-flight while crossing the Atlantic.
“I am sure your readers would be shocked if they realised just how crap IT has been organised in many banks as well as corporations and government ministries. Sometimes we get a glimpse, when a company is unavailable for days due to ‘computer problems’. Have you noticed these cases always take longer than expected? This is not because repairs take long. Finding out what the problem is in the first place – ‘root cause analysis’ – that’s nearly always the most time-consuming. Nobody has a complete and in-depth overview any longer.
“Years ago management in major banks and corporations decided that they could outsource vital IT functions to companies such as IBM, Tata, HP and Atos Origin T-Systems. The idea was that if you describe the processes you require adequately, it’s safe to delegate their execution to outsiders. But the first contract goes to IBM, two years later a contract for another part of the infrastructure is awarded to HP, then Cisco gets to manage the network … Now, who is responsible for the overall system? All systems need patches frequently. But before you install one, you need to make sure it doesn’t do things you didn’t expect. This requires cooperation between these suppliers, who will of course charge for these things. Often the various suppliers’ support contracts don’t match. Also, there is little continuity within those suppliers; operations are restructured, people get replaced or moved around. Vital expertise gets lost. Major suppliers do collaborate on patch management. But each has to ‘certify’ every upgrade, and since each works with a different ‘software map’, they come to different decisions on different timetables regarding certification. Often the latest software cannot be installed because one of the suppliers has not certified it.
“Are so-called chief information officers, the top executives responsible for IT, aware of this? I very much doubt if they are and if they care. They are managers, skilled in office politics, not technical experts. Most CIOs rarely stay in their post more than a few years. I worked for one of the major software companies in the world. It took my boss a year and a half of begging and pleading with the secretary to get a meeting with the CIO of a major client. CEOs are worse. They are afraid of looking stupid or ignorant, and actively avoid their IT people. It’s a huge source of frustration on the part of technologists; particularly banks see technology as ‘support’, as a side show you delegate.
“When I went into IT in the mid-90s, there was this idea that we were making a positive difference in the world. Info-sharing was bringing down the chances of another world war, we thought, eliminating the scope for misunderstandings. I no longer believe this. Things have become too complex. I know people in IT who retain paper copies of all their internet banking statements.”
• Follow Comment is free on Twitter @commentisfree
Need a Loan? Visit Secured Loans Broker.
View full post on Free Secured Loans Advice