How to measure the success of a new IT system

Does the new software work well enough to pass all its quality-control tests? Can you convince people to install it? Once installed, do they like it enough to pay for it (either in money or with their time)? Does it continue to be used beyond an initial “kicking of tyres” phase to see what it can do? That question leads me neatly onto my second criterion for measuring success – user adoption.

User adoption

Introducing a new computer system is a change to the status quo. Many people are reluctant to embrace change, while some are actively hostile – “but we’ve always done it this way” is the oft-heard refrain. Ideally, a new computer system should make life so much better for users that they embrace it whole-heartedly, but unfortunately this can’t always be achieved. Sometimes the new system is necessarily more complicated than the old one (or rather the old way was too lax and “flexible”, and the new system imposes more order).

In any case, users may resent the imposition of structure on their work. It may, according to the managers, represent “standardising best practice”, but some users may still see it as the computer telling them what to do.

Often you’ll have to convince people that a new system is better than the one it is replacing even though it’s more complicated for them to use

Often you’ll have to convince people that a new system is better than the one it is replacing even though it’s more complicated for them to use. It may be that you’re asking them to enter more data than before, which effectively is telling the computer more about their working style. Nowhere is this more apparent than when implementing a new customer relationship management (CRM) or enterprise resource planning (ERP) system. Their data used to be held in several Excel workbooks spread across different computers, or even just in their head; now they’re expected to enter all the data into one monolithic system. It’s hardly surprising there can be resistance.

CRM and ERP systems are by their very nature large and complex, and a single user won’t necessarily see any benefit from their use. They’ll just feel they’re being asked to put all their data into this beast without getting much, if anything, in return. This notion that it’s somehow “their” data is one of your biggest problems. It isn’t. It’s the company’s data. The company benefits from having all its data in one place and made visible to those who need it. If an individual employee is ill, or falls under a bus, someone else can pick up where they left off with minimal disruption to the business processes or customers.

Some users don’t like this idea at all: they want to be indispensable, because they think that will guarantee their job security. It won’t. It may, however, mean they’ll never be promoted. If only they know how to do a particular job and they hide their work inside strangely named files or keep it all in their head, I’d say they’re a liability rather than an asset. No-one in a company should be indispensable, not even the CEO.

So, how do you get people to use a new computer system? The first thing is to ensure you have “buy-in” from the senior management. They should know all of the benefits it brings to the company, themselves and the workforce. They should also be made accountable for enthusiastically communicating these benefits to the people in their division, department, group or team and ensuring those people can and do use the system.

If user adoption is to be one of your measures of success, you must define exactly how you’re going to measure it. Is it the number of users in any department who log onto the system in any week? Or the total time they spend logged in? The number of transactions they made on the system every week?