Ask Slashdot: Good Metrics For a Small IT Team? 315
First time accepted submitter shibbyj writes "I'm a member of a small 3 person IT team for a medium sized business (approximately 300-350 employees) that has multiple locations internationally. I have been tasked with logging our performance using the statistics from our ticket management system. I've also been tasked with comparing these stats and determining if we are performing above or below what is considered optimal. I'm wondering what people opinions are on what good metrics should be in regards to mttr mtbf etc. I have had trouble finding information on this."
Hahaha (Score:5, Insightful)
Re:Hahaha (Score:5, Insightful)
Also, the title is a bit redundant: all IT teams are small.
Re:Hahaha (Score:5, Insightful)
As an IT professional who was just fired from a job shortly after management started to ask for all sorts of new metrics from our ticketing system, I also fully agree with the parent statement. Get your resume together and start getting it out to recruiters pronto.
In other words, this is a sham. There ARE no standard metrics because every single company and department are different. The only real metric that matters is if your customers are happy and satisfied, and that they could find out better than you could. You're being set up.
Re:Hahaha (Score:4, Funny)
Re:Hahaha (Score:5, Informative)
Yes, this.
Management wants to eliminate someone and wants to do so in an "objective" way to hide the fact that they're firing someone while probably giving the CEO a fat Christmas bonus. You're tasked with figuring out which of the three of you gets fired and how you can cloak this in enough "objectivity" that no one can object to it. Your best bet is to make this shit up. Figure out who the weakest link besides yourself is, or who you like the least, and generate a system of metrics that's biased towards eliminating that person. Use lots of acronyms and jargon. Also, make sure no one at work reads Slashdot.
totally irrelevant CAPTCHA: forgive
Re: (Score:3, Insightful)
Re:Hahaha (Score:5, Funny)
The very core of your writing while sounding reasonable initially, did not sit perfectly with me after some time. Somewhere throughout the paragraphs you actually were able to make me a believer but just for a very short while. I however have got a problem with your leaps in logic and one would do nicely to fill in all those gaps. If you can accomplish that, I will surely end up being fascinated.
Re:Hahaha (Score:4, Insightful)
Yet another AC here.
If there were any justice in the world firings would start at the top. Fuck management and their big salaries. What the fuck are they doing that they have to be paid so much? What value do they bring to a company? They're too happy to fire you to make more money for themselves (the fact that most people are one paycheck away from financial ruin doesn't seem to bother them) but do they ever get fired even if they run the company into the ground? Fuck no. I'd love nothing more than to see some of the bosses I've had in the past be out there doing actual work.
Is this better?
Re:Hahaha (Score:4, Funny)
Re:Hahaha (Score:4, Insightful)
Do you actually believe this? Seriously? Most upper management is there because they either knew someone in the right position or had enough money/clout to force their way into it. Often time through family. Rarely these days do you see any upper corporate management that actually worked their way up.
Re: (Score:3, Insightful)
Do you actually believe this? Seriously? Most upper management is there because they either knew someone in the right position or had enough money/clout to force their way into it. Often time through family. Rarely these days do you see any upper corporate management that actually worked their way up.
Do you have statistics or data that shows this? Because my experience has been the opposite, people do become upper management because of skill. Except in companies that are about to fail.
My guess is you just made up your facts there, but if you have hard date, I'm willing (and interested!) to change my opinion.
Re: (Score:3)
You just sound like you are offended,
I am not, I like to see data. That is how I learn.
because you happen to have an upper management position or are connected to one through family or friends.
I wish I were. Alas, I am merely a lowly programmer.
The upper management doesn't need skills, since they know how to get people with skills, that is why they spend their time with golf and relaxation... at that level, that is how work is done (slowly with relationships and quickly with manipulation).
If this is your view of upper management, you're braindead.
Re: (Score:3)
Your view is optimal but its not at all reality.
Why do you think this? Based on your own un-optimal view?
I bet your own upper management has personal ties among each other that you don't realize or are ignoring to make your argument.
Being 'friends' isn't the same as being incompetent. If I'm CEO, I'm going to hire people I think will do a good job. They might be my friends, but they aren't going to be the ones who are incompetent.....those will drive the company into the ground.
This isn't something limited to upper management.....if I want programmers on my team, I'm going to first look at people I've worked with previously who I KNOW are good programmers. This is extremely com
Re: (Score:3, Insightful)
Re: (Score:3)
Management isn't about having the ability to offer an extended work knowledge base to your subordinates. Management is all about leadership and when to crack the whip
This kind of thinking has caused a number of companies to fail. That's part of management, but the most important part of senior management is providing direction (there's a hint in the job title for a lot of them...). This requires them to have a detailed understanding of the market. A successful manager needs to understand three things:
The first is what his subordinates are capable of. When an engineer says 'that's impossible' does he mean 'I can't be bothered?' Or 'we could do that, but only with f
Re: (Score:3, Informative)
First observe that no matter what, someone HAS to be in management. CEO is a position, unlike yours, which cannot by stay empty for long.
Second, all that has to happen to rise in a company is someone above you promotes you. People who bet on Skill and Hard Work taking them there are a dime a dozen and what's become essenti
Re: (Score:2)
Re: (Score:2)
Done in one (Score:5, Funny)
Seriously I just have to say that this is the single funniest comment I've ever read on Slashdot. Laughing, pointing at the screen, drug my wife over here to have her read it funny. Brutal. Absolutely brutal.
From one cynical bastard to another, I salute you.
Re: (Score:2)
Re:Done in one (Score:5, Funny)
You cynical man. For all you know, upper management have a budget flush with cash and have singled out someone in the hard working but unacknowledged IT department for a raise and a promotion.
Re:Done in one (Score:5, Informative)
You cynical man. For all you know, upper management have a budget flush with cash and have singled out someone in the hard working but unacknowledged IT department for a raise and a promotion.
Don't forget the free pony.
Re: (Score:2)
I was coming in here to say that. LOL
Re:Hahaha (Score:5, Insightful)
One of you is getting fired
On a three person team, metrics are irrelevant - personality and politics are 100% more important than technical skill.
If you've got 15-20 people and it's time for a 10% downsizing, or (less realistically), performance based bonus or advancement, then metrics start to be something worth looking at.
When I had to choose between 2, my boss asked me who was my choice, I told him, he agreed, I started to tell him why, he stopped me and said: "it doesn't matter why, I'm just glad that we came up with the same answer."
Re: (Score:3, Interesting)
Concur with the "someone is getting fired". 3 people supporting 300+ people. That's overtaxed, even school teachers only have to deal with 30 potential-assholes for 8 hours a day and have to put in unpaid overtime because you can't just leave things when the shit hits the fan 10 minutes from clocking out.
As for metrics, serious or otherwise, 3 person team, you should already know who the weakest link is (hint, -you- if you don't come up with a solution.) The boss always asks the least intimidating person, t
I just can't add anything to that (Score:5, Insightful)
Might as well close the comments now. :-)
Go look up Robert Austin's book on measurements and management. Read it and recognize that you've been given a task that is at best counterproductive and at worst impossible. Dust off your resume, because it may be more than one of you that are getting fired. ..bruce..
Re:Hahaha (Score:5, Interesting)
As I was idly paging through the comments thinking about how many of them were jumping to outlandish, unsubstantiated conclusions about why the poor submitter was being asked to come up with metrics, I also realized that pretty much nobody (in the finest tradition of Slashdot) has bothered to answer the actual question: "I'm wondering what people opinions are on what good metrics should be in regards to mttr mtbf etc." I think I misunderstood this at first as well... in light of what he does say about the goals ("...determining if we are performing above or below what is considered optimal") I don't think he's asking what metrics to measure or whether or not we think doing so is any good, but instead what reliable industry benchmarks are for those metrics, so he can tell his boss what is "optimal" or not about their operation.
Well, shibby, sorry, but I don't think there are such things, or at least none relevant enough to take back to your management team. The only way to get anything meaningful out of metrics (obviously a lot of other posters are arguing you'll get nothing meaningful out of them; I disagree, but it may not matter either way if those are your orders) is to establish baselines for your organization and track future performance against those. It will take a while and it will have to be viewed in context to be worthwhile... important points to make when you are presenting your findings to management.
I'm being optimistic and assuming genuine business goals and a desire to understand IT operations on the part of non-technical managers are the point of this request, not some haphazard effort to chop down a three-person department, but it is also worth passing along some of the critiques that are being posted here. On the other hand, if you don't already, you should understand that not all managers are buggers, and that many of the better ones have legitimate reasons for trying to understand what is going on in their IT department. We often forget how mysterious what we do looks to the un-initiated, and I have seen enough poorly run IT departments to sympathize with non-technical managers who are grasping for the tools to understand theirs. The point being, getting defensive and obstructive in the face of these requests isn't always the brightest idea; instead, you can look at it as an opportunity to present some of your perspectives and difficulties to managers who are finally prepared to hear about them (after all, they did ask!). They may not have the time or horsepower to learn everything you do in depth, but it is possible to analyze operations based on the right sort of shorthand.
The cynics may be right; only you will know. But I've seen companies run down by their IT departments as often as I've seen IT departments run down by the management team, so performance concerns on both sides are well-founded. Anyone who thinks their manager shouldn't ask for a suitably abstracted toolset for judging performance is asking for a stupid manager.
Metrics suck (Score:5, Informative)
Re: (Score:3)
Great read too. I question this:
determining if we are performing above or below what is considered optimal
How do you determine 'optimal'? How you determine that determines the metrics you use.
Re: (Score:2)
Determining optimal comes well after you start collecting metrics--usually months after. If you've never collected them, you are almost guaranteed to find things in the first month (or even the first day) that will look stunningly bad at first, but are quickly correctable. Once you get things balanced out--if you can--then you can figure out what's optimal.
Getting management to understand this may be a chore if they're new to metrics or not trying to find a reason to fire someone.
Re: (Score:3)
If you don't know what's wrong, how can you identify what to improve? You may be able to come up with a few things off the top of your head based on anecdotal evidence, such as frequent concerns about available storage or that tickets take too long to close. Those can perhaps be part of your initial focus, but metrics are there to help you identify trends, find the deviations, and act on them (preferably before they become a problem). If a server is consistently running at 50% CPU utilization, is that op
Re:Metrics suck (Score:4, Interesting)
Don't know how to program? Hit the books - your job may depend on it.
Re:Metrics suck (Score:5, Informative)
Bad metrics suck, good metrics are useful data.
As folks have already mentioned, time to answer and time to resolve are both important, and I think you have to watch for re-opened as well to curb "how fast can I shove this under the bed?" resolution games.
My favorite is average tickets per user, though. Particularly on a small team, what you really want to gear your measurements toward is preventing incidents in the first place. It is helpful to know what your overall ticket volume looks like, then, and to aim to decrease it over time in the same way you might try to decrease time to answer and time to resolve. That's important, because as the previous article suggests, if you will get what you measure... and your overall goal should not simply be to answer tickets faster and resolve them more quickly, but to not have as many in the first place*. Every issue represents a waste of somebody's time and therefore corporate resources that could be put to more productive uses. Steadily decreasing mtta and mttr are nothing to cheer about if your ticket count is increasing.
But you can keep it simple. You can drown yourself in metrics and lose sight of why you're tracking them and what you really want to accomplish. You may not really need any more than these few; better to start small and add what you need when you need it. I know there's always tension over getting a system in place that can capture what you need for historical purposes when you realize you need to know something new down the road, but resist the urge to over-collect. Half the time you won't need it all and are just wasting time getting it in the system.
* There is a caveat to this; in some organizations, I actually DO want to see an increasing ticket/user count, at least for a time. This is something I shoot for when relations between IT and users have broken down badly enough that users have stopped reporting problems to IT because they feel it's useless, and their issues are never resolved anyway. In those cases, a rising ticket count can represent an increasing trust level, which is good. You generally won't fix issues you don't know about.
Re: (Score:2)
First call resolution is almost as important as 7 day repeat rate... while you do want to look at things like mttr, you want to be careful with using that as a hire 'em and fire 'em metric, because it will encourage people to cherry pick easy problems, and can penalize the people who get stuck with the hard issues. It takes 5 minutes to configure somebody's e-mail client, but it will take much longer if you have to order parts to replace defective hardware.
Re:Metrics suck (Score:5, Informative)
Metrics is just a management word for "bad statistics".
With a distribution of 3, it's not really possible to have statistics of meaningful nature. You've got shared responsibilities and bounce things off of each other. One person may open and close more tickets, have a shorter duration for tickets, etc. - but he's only doing the actual "work". The others may be giving him all the input necessary to complete the task(s).
Ideally, your ticketing system will reflect, very vaguely, who's doing work and who is not, but even then it's not going to be well representative of what's actually going on.
People do different types of work, of different levels of difficulty. For instance, I may do one ticket on Monday, three on Tuesday, and one for Wednesday through Friday. Why? Aside from the fact that I'm bad about actually doing tickets for my work (my god, I'd not have the time for work, and then there'd be more things that aren't getting done, making us -all- look bad), there's the reality that my tickets aren't terribly easy, often requiring hours of log perusal and research to try to fix problems. Meanwhile, the guy who knocks out 40 tickets a week - malware disinfection, workstation reinstalls, etc. - has fairly wrote work of a repetitive nature, comparably. Also, he's following instructions or asking for advice on a regular basis, even if I'm not his boss.
You said so yourself: you're a member of a 3-person IT team. The only use 'metrics' have here aside from what should be plainly obvious in a group of 3 (who's fucking up, who's not getting back to people, who's not doing work) is to keep track of what amounts to customer requests and problems. X workstation needs to be reinstalled, Y server has a crashing whatever, and so on. If you're working on and sharing a ticket queue, you are all mutually responsible for all of the tickets: if something isn't getting done, it's everyone's fault (or nobody's fault). You may consider presenting your metrics in the light of this reality (like statistics, metrics can lie, too).
Re: (Score:2)
The problem with metrics is that if you look closely enough at things to actually KNOW if they're good or bad, you know enough to not need them.
Is the guy with the lowest average tickets slow or is he the guy that takes the hard ones because the rest of the team doesn't stand a chance of solving them? If you know the answer to that question, he is already evaluated, what do the metrics contribute? As you point out, a declining ticket count can be very good or very bad. If you know which it is, you already k
Re:Metrics suck (Score:5, Insightful)
Totally agree.
My small team was taking a beating and getting little support from management. Issues were piling up, we were getting randomized, and I felt like I needed another staff member. Management wouldn't budge though. I needed to prove to them that I could manage my department better. Nobody even cared about IT metrics at our company. Rather than see them as an evil thing, we embraced them to gain credibility in our organization.
We started by outlining some of our support boundaries, types, and set goals for response times. Then we customized our ticket system and added in some categories and priorities.
For Support (meaning something is broken and must be fixed)
High pri: 24 hour solution
Medium pri: 1 week
Low pri: 3 months or more, not a critical metric
We also had a "Request" category (not a fix, but doing standard tasks like toner replacement, adding e-mail alias, help train someone, etc.). These had different goals for solutions. We even had ticket categories for maintenance and projects. Though they junk up the ticket system, it allowed us to track time going into other tasks and paint a picture of a staff member's whole day of work, not just the support end.
Some of our favorite metrics:
1. Average time on support OR request tickets aggregated by our team or split down to individual staff members, and divided by priority level.
2. Total time spent on tickets by user (so I know if we're actually working or lounging around). ---this is a motivator for staff to actually enter tickets and get credit for their work. Some flexibility is necessary here because people eat lunch and go to the bathroom - you can't expect this to add up to 40 hours a week perfectly.
3. "Most troublesome user or department." I don't advertise the data on this report, but it lets me know who to focus on with either training or nudging their boss a little. Execs get interested in this once in a while, and the users that found out we keep such a log try to keep off of it. Many will joke about it, but still ask "Hey I'm not in the top 10, right?" It encourages users to be a little smarter and not lean so heavily on IT for silly things.
4. Most troublesome product - THIS IS BY FAR THE BEST METRIC. This has helped me gain support in dropping old junk software, getting better solutions, setting up training for people, or creating general awareness that we need to improve. For example when I was able to quantify how much time it took when people forgot their passwords for our cloud-based e-mail password, I got support for Office 365 and ADFS for single sign-on. In the interim, people stopped took a little more personal responsibility because they realized that password stupidity = preventing us from getting value-added projects done. Stuff like this is beautiful because it takes most of the heat off of IT staff, and informs other managers of our company-wide pain points that we should invest in.
5. On the same lines as #3, identifying how many high pri tickets are out there vs. medium or low (and possibly categorizing by customer or department) can sometimes sniff out IT abusers that need stuff now now now. Tread on this carefully as you don't want to constantly wage war with your metrics, but with some political prowess and some data to back you up, you can start solving some of these problems.
#3 and #4 are big for us. Don't just think of metrics in terms of how you will hang yourself. Think about the bigger picture and answer questions that will help motivate your team in a positive way, get credit for what they/you do, and enable you to tell a better story to management. You probably will want to develop two sets of metrics - a more detailed set for yourself so you can manage effectively and proactively identify problems (and solve them in advance before your boss gets to it), and a more general set for your boss so you can deliver a simple and concise picture without getting him too involved with the details you can handle yourse
Re: (Score:2)
Sorry, I guess I was not clear enough about this; it's a generalized number that has nothing to do with either the individual tickets or the individual users. It's measured across the board: Monthly Tickets/Total Users. It's a gauge of the department, not of specific people.
Re: (Score:2)
Re:Metrics suck (Score:4, Interesting)
it's a nightmare.
metrics and quotas will only lead to less overall return... unless, of course, you're just trying to create a jobs program.
Re: (Score:2)
Here's a few... (Score:5, Informative)
Time to answer call, time to resolve ticket, abandoned tickets (unresolved).
If you google a few of those it will bring some more, but that's a simple start.
Re:Here's a few... (Score:5, Informative)
Re: (Score:2)
These are both good. I'd mod up if I could.
call / ticket time is bad metrics (Score:2)
as not all ticket take the same time.
Do want quick password resets to count the same as doing some think that needs a more time? like setup / reload a pc?
Re:call / ticket time is bad metrics (Score:5, Informative)
I got written up once because my ticket stats were radically different than the other people on my team. 15% lower "total time on tickets" but 20% more tickets closed. I was apparently fudging numbers and closing unresolved tickets.
Fortunately, a trip to HR with a ream of printouts from closed tickets proved otherwise.
Still left the company a few months later.
Re: (Score:3)
You get what you reward, not good work (Score:4, Insightful)
Time to answer call, time to resolve ticket, abandoned tickets (unresolved).
In business school it is a common theme in various classes that you get what you reward, not what you ask for, not what is necessarily best for the organization. Here is a highly relevant Dilbert cartoon illustrating this point, http://dilbert.com/strips/comic/1995-11-13/ [dilbert.com].
The underlying problem is that metrics applied to humans leads to people working towards the metrics, not necessarily doing good work. It is a classic environment for unintended consequences. Its not even that the people are necessarily being opportunistic, there is also a certain amount of practicality. If you are being measured by some metric and keeping your job or getting a raise is dependent upon that metric you may quite rationally decide to act to that metric rather than what is necessarily in the best interest of customers.
Are you measured by resolved tickets? Then tickets will get resolved quickly. Not necessarily thoroughly, completely, or robustly resolved. Which leads to related followup tickets because of a minimal effort put into resolving the original ticket. I saw this in a programming environment where the tickets consisted of new features or bug fixes.
Are you measured by abandoned tickets? Then tickets will get resolved, even if they don't reasonably deserve to be considered resolved. You will get things unnecessarily classified as "unable to duplicate", "insufficient information", etc.
In these two examples, where is the difficulty of the task factored in? Not all task, tickets, are equivalent. Furthermore sometimes there are external dependencies, a part is being shipped, where is this factored in?
The metrics you offer are reminiscent of stats from call centers. There such metrics are a little more reasonable, not perfect but perhaps OK, given that the calls are somewhat equivalent in the amount of effort required, a small number of minutes not hours, and that they are randomly assigned. Over the period of say a month the large number of calls handled by any operator will resemble a normal curve with respect to effort required. For an IT organization the evaluation period may need to be some number of years to get to a normal curve with respect to effort required.
Re:You get what you reward, not good work (Score:4, Interesting)
Are you measured by abandoned tickets? Then tickets will get resolved, even if they don't reasonably deserve to be considered resolved. You will get things unnecessarily classified as "unable to duplicate", "insufficient information", etc.
I experienced a form of this phenomenon just this afternoon, albeit not in an IT environment...
I went to Bank of America to (proudly) close my accounts, having moved to a smaller and thoroughly more moral bank. The customer service rep figured it out right away when he saw that my six ~18-year-old accounts now all had zero balance, and zero activity for the past month. So we start closing. He is doing the keyboarding and mousing while I am watching the screen.
At a certain point in the process of closing each account, the rep is required to give the reason for account closure. In the popup list of reasons are some very relevant choices: 'Service' and 'Competition' sprang out as the correct choices. He chose 'Misc', and for the sub-reason chose 'Bank of America Consolidation', whatever the hell THAT meant.
It was then that I knew there had been a memo from headquarters, probably last month, that said "We know people are closing their accounts. Management wants to make sure that the reason is NOT people leaving in disgust, headed to our competitors." And so now the CEO can stand up and say "We've only lost 1% of our accounts to the competition!"
Just this afternoon that happened, right in front of me. I almost laughed out loud.
easy (Score:5, Insightful)
"what good metrics should be in regards to mttr mtbf etc"
Easy, there are no good metrics. Metrics don't lead to improved business outcomes, they rarely cover enough variables to tell the whole story, so all they lead to is people gaming the metrics, most likely leading to worse business outcomes.
Metrics are favoured by lazy management.
Re:easy (Score:5, Funny)
Look, using metrics doesn't indicate lazy management.
Look, using metrics that you don't have available doesn't indicate lazy management.
Look, using metrics that you don't have available so you ask your staff to measure their own metrics doesn't indicate lazy management.
Look, using metrics that you don't have available, so you ask your staff to measure their own metrics, but you don't know what metrics they should measure, so they end up asking Slashdot what some good metrics are, doesn't indicate lazy management.
Re: (Score:3)
Metrics are favoured by lazy management.
Metrics are also favored by management that does not knows squat about what the heck they are managing.
Re:easy (Score:4, Insightful)
More to the point: There is no good reason to use metrics for a 3 person team. You could sit down and observe the 3 person team for a week and get a very good idea of what's going on, and it would be easier and probably take much less time than designing, implementing, gathering, and analyzing metrics. It's easier and more accurate.
If you're going to use metrics, use it in situations where you're managing such a large group that you can't possibly know who everyone is. Don't use it in situations where you have the option of doing a simple evaluation.
Re:easy (Score:5, Insightful)
I don't buy this. I (hate to admit that I) worked at Zynga, and their entire business model is based off of metrics, both internal and customer metrics. They are behaviorists, and brought in financial industry data modeling to design games and make business decisions. It works, and it works fucking well. In the long term this Skinner box model may or may not work for creating a sustainable business, but it has worked for the last couple years. People may not like their games, but they keep playing them.
The problem is bad or incomplete or misinterpreted metrics. Metrics in and of themselves or not bad. The problem is with the people that use them.
LS
3 People? (Score:5, Insightful)
Re: (Score:3)
Doesn't mean they aren't going to try to work with 1 fewer. I remember working an understaffed job a while back and they still managed to figure out how to eliminate an extra position. It didn't work well and stressed out the employees, but they were able to cut the position.
I'm guessing they'll try that here, even if they do have to give up and hire somebody back.
Good luck (Score:2)
Re:Good luck (Score:5, Interesting)
It's not unusual for management to be clueless about what exactly it is that their IT staff does on a daily basis, nor is it unnatural that they should take an interest. Often, it's a good sign when they actually ask the guys doing the work what the metrics should be... it indicates some degree of trust, and they haven't simply read an "IT Management for Dummies" book over the weekend laying out some arbitrary system that isn't going to fit your organization.
As a more cynical commenter points out, it also provides the opportunity to create a measurement system that you can game to make you look good. But I think it isn't a terrible sign that the bosses care what their employees are up to. It may represent an opportunity to explain what you think is important that perhaps they hadn't considered previously.
Above optimal? (Score:2)
You're always below optimal, by definition of optimal.
Re: (Score:2, Insightful)
I think you've confused "peak" with "optimal."
Peak = best possible output
Optimal = most satisfactory
You can tune an engine to run within optimal specs. But if you run it at peak nonstop...
Re: (Score:2)
I can tell that you didn't actually look up those words in a dictionary or anything.
optimal? (Score:5, Informative)
talk about flow, about bottle necks. Visualize workflow. Look at Henrik Kniberg's paper on kanban as applied to IT Ops. My guess is that your ticketing systems will provide low value data on volumes on resolution time - gear up - visualize the pipeline. check http://www.infoq.com/minibooks/priming-kanban-jesper-boeg - turn the conversation around to "business value" - don't get wrapped in the ropes of volumetrics /peace.
Re: (Score:3, Informative)
+1 for this answer. Kanban is a great way to generate actually useful metrics for a team, project, or department. You'll be able to calculate things like how long it takes the average ticket to work its way through your processes, where tickets tend to get stuck (cumulative flow diagram), and where the sources of waste are in your processes.
In addition to the book mentioned above, I also like this one by David Anderson: http://www.amazon.com/Kanban-Successful-Evolutionary-Technology-Business/dp/0984521402/r [amazon.com]
No Metrics (Score:4, Insightful)
There is no metrics system that can't be gamed.
If you set it for "total tickets fixes" (higher=good): you just encourage people to report trivial problems you can fix easily.
If you set it for "total tickets" (higher=bad): you refuse to do things, add features etc, or you make it hard to contact IT to log a fault
If you set it for "time taken per ticket" (higher=bad): you end up pushing kludge solutions
If you set it for "user rated response" (higher=good): you end up blackmailing the end users to rate you 10/10 otherwise their emails/logs/dirt etc get published and sent to boss/wife/etc
Ask your manager how their performance is evaluated? Then start suggesting ways they could bust their KPIs, and they should get the drift.
Re: (Score:3)
If you set it for "total tickets" (higher=bad): you refuse to do things, add features etc, or you make it hard to contact IT to log a fault
If you set it for "time taken per ticket" (higher=bad): you end up pushing kludge solutions
If you set it for "user rated response" (higher=good): you end up blackmailing the end users to rate you 10/10 otherwise their emails/logs/dirt etc get published and sent to boss/wife/etc
So... 'user-rated response' it is, then.
Thanks!
Ahh, metrics, good. (Score:4, Insightful)
Metrics. Excellent, I hate when bosses use the Imperial system.
All jokes aside: If you care about your job in this economic climate, I suggest you do what your 2 other teammates are doing - picking through the stats that make YOU look the best. The company isn't going to look out for you. IT is an expense to be cut, remember. Boosts the temporary bottom line, promotes "growth" in this fiscal quarter, gets the investors going so the CEO can shuffle another fold into his golden parachute. If non-important metrics are selected that sacrifice your job, it's a brief victory lap straight into the unemployment line.
We can't answer your question, though. In the end, I recommend you watch a clip from "Office Space" - wherein the Bobs interview the employees:
Bob: "So tell me, what is it, exactly, that you do here?"
If you can't answer that question, you probably should be job hunting already. Or should have kept a copy of the job posting from when you applied.
Metrics (Score:2)
I have been tasked with logging our performance using the statistics from our ticket management system. I've also been tasked with comparing these stats and determining if we are performing above or below what is considered optimal.
Standard ticketing system metrics are no good. Add a post-incident survey to your customer interactions. Your metrics don't measure performance, unless you can actually measure how 'hard' the ticket is intrinsically / what skill is required, and how well the ticket is solved.
so if i build an incredibly shitty IT system (Score:2)
where people are constantly calling me asking to fix stuff, then my numbers will be awesome!
god, capitalism is awesome.
Re: (Score:2)
where people are constantly calling me asking to fix stuff, then my numbers will be awesome!
I think you missed the part where total IT service revenue for that application is divided by the number of tickets. The more tickets for a specific IT service, the less the issue resolution is worth.
The fewer total tickets for that service, the more each resolution is worth.
For a team of 3 people? (Score:2, Insightful)
Is shit broken?
Does shit get fixed fairly quickly?
Are your people busy, but not swamped?
The answer lies in the past... (Score:2)
Yesterday's Slashdot, to be exact: http://it.slashdot.org/story/11/12/14/225204/the-four-fallacies-of-it-metrics [slashdot.org]
Easy, start by choosing what you need to change (Score:3)
- Percentage of staff with ITILv3 foundations certification: zero.
I know this because of your question. Watch thes videos as a start: http://pmit.pl/en/it-management/free-itil-v3-course-collection-of-itil-v3-moviesdarmowe-szkolenie-itil-v3-zbior-filmikow-o-itil-v3/ [pmit.pl] -- and sell a formal training course to your management.
The people joking about how one of you is getting fired, or you're all getting outsourced. . . probably true. Learning ITIL is all about learning what's important to your business stakeholders, how to monitor/measure these things, and how to make sure you're always making the right decisions based on the business priorities.
If you can't convince them to pony up for you three to take the certification course, then pay for it out of your own pocket, you'll need it to find a new job.
Look for a new job (Score:2)
Exactly. (Score:2)
I found this gem in the other metrics thread the other day.
-2000 lines of code. [folklore.org]
Add something (Score:4, Informative)
Your ticketing system needs or needs to have added an automatic followup to the customers. The system sends out an email after every ticket asking "Did the problem get resolved in a reasonable amount of time? Did the IT staff respond in a way that enabled you to get back to work?" Nothing more complex than that, though you can parse things out by ticket priority (though deciding what's a higher priority than other things is, just by itself, a major undertaking).
Your goal should be to increase the percentage of positive responses.
Why this touchy-feely stuff instead of a hard metric? Easy. There are no metrics that work in your situation. It's quite easy to argue that there are no metrics that work, period.
By adding this email feedback to your ticketing system, you have met the requirement to come up with a metric derived from the ticketing system.
Selling this to management can be simple, depending on how you handle it. Something along the lines of "Given that the IT staff is so idiotically understaffed, we must be given the agility to solve problems instead of meeting random metrics. Only our customers can know if we met their needs, considering all pertinent factors. Someday, when we actually have enough people and money to divide work more rigidly, we can add metrics like timeliness of ticket closure, etc." Then you hope they never notice that you never divvy up the work rigidly. All of this requires having an IT manager who is dedicated to the inescapable truth - that their function is to keep the MBAs off your ass and let you do your job.
I've worked where my performance was measured in this way. It can be heaven.
One more thing - if your upper management doesn't already have faith in you, they'll never go for it. They need to already appreciate your contribution to go along with this. The very fact that they're asking for metrics tends to suggest they don't sufficiently appreciate you now. If that's the case, than all I can say is that I've worked under those circumstances, too, and my heart goes out to you.
Re: (Score:2)
Doomed from the start (Score:4, Insightful)
> determining if we are performing above or below what is considered optimal
Scenario 1: you are below optimal -> you are inefficient so they replace you
Scenario 2: you are above optimal -> you are overkill so they replace you
Bottom line, I would rent The Wire and learn how to "juke the stats" because that's the only way you won't get to jump on that grenade.
Been there, done that - my advice: be just under optimal so you have room to grow and show improvement, but don't be too low so they don't feel the need to consider a business case for outsourcing.
Re: (Score:2)
Customer Satisfaction? (Score:3)
I'm not with most IT management on this one, but I always thought the best metric was customer satisfaction. For instance every time I open a ticket with Cisco I get a survey at the end of like 5 questions. Was my problem resolved, was the person polite, etc.
The other metrics suggested are things to graph and look at trends. Are repair times getting worse or better? Is the average time per ticket going up or down? They are great int he aggregate. They break down quickly when divided. Only one guy on your team knows network devices, so he gets all the network devices which include the 8 hour fiber cuts, so his times always are worse than the guys fixing printer problems, as an example. You have to be very careful as you start to divide them up.
At the end of the day though you're trying to make the customer happy. Track it, and see how your staff is doing. If people are happy with their IT support, your department will be seen in a more positive light.
how to survive in the corporate environment (Score:5, Insightful)
1. make your numbers.
nobody actually cares what 'the numbers' are, or if they actually mean anything. but you have to make them.
you might ask yourself - isn't this a huge waste of time? isn't it completely counter productive? doesn't it actually decrease efficiency? aren't the metrics measuring completely the wrong thing? as the slashdot story the other day said, aren't bad metrics actually worse than no metrics, because they cause people to do inane, wasteful things to make their numbers?
well, your problem is that you are asking yourself. in a corporate environment, do not ask. just do.
just make the numbers.
hopefully, if you get good enough at 'making your numbers', you will have time left over to actually do some work.
2. but what about the theory of capitalism, the free market, efficiency, etc?
its all bullshit. just like the theory of communism was bullshit. what statistics and 'numbers' were reported to the government were just flat out garbage. people somehow managed to make the system work through personal relationships and working-around the assholse in charge. but most of the theories it was built on have no resemblance to reality. think about it - if efficiency really made for the best corporation, why would you be spending 4 hours a week filling out meaningless statistical performance reports that nobody will ever read, let alone understand?
the only difference between the soviet union and 'the west' is that 'the west' still hasnt collapsed yet.
Re: (Score:3)
the only difference between the soviet union and 'the west' is that 'the west' still hasnt collapsed yet.
It will. People looking at the inner structure have known it for a while. Now the cracks are starting to show on the surface where anyone can see them. It won't be long.
If you are brave try to look past how you're going to survive the collapse: try to plan how you will help rebuild it better.
Customer Satisfaction (Score:3)
The only truly meaningful metric is customer satisfaction. After each ticket is closed send a survey email to the user. If your team plows through enough tickets you get a statistically significant success % per tech that you can compare to the other techs.
Without knowing a lot more about the nature of tickets it is hard to give a better response. It might be very important to gauge the difficulty (trivial-hard) and novelty (common-rare) of tickets but it could also be a waste of time. Does one tech plow through dozens of tickets a day or a few? Are the techs specialized? Anything email related goes to joe, hardware issues to tom, etc.. Are the tickets auto-generated from emails, called in or do they fill out a web form?
Given the size of the team the company thinks 1) you aren't getting the job done, 2) one or more people on the team are dead weight or 3) you are overworked and need more people. I wouldn't bet on #3.
Metrics - A lazy manager's out every time. (Score:4, Insightful)
Falling back to metrics is a lazy manager's way of proving to her superiors that her drones are operating at peak efficiency. The most lazy of all will rely on utterly meaningless metrics such as the number of help tickets closed per day, per individual per day, etc. A metric such as this is completely useless as all tickets don't require an equal amount of effort to complete. Diagnosing a problem due to an intermittent hardware issue doesn't take the same amount of effort as helping a user change their password. Unfortunately these types of issues generally comprise the vast majority of tickets generated and therefore often end up being the ones that are 'measured. ' This often leads to a drop in morale and thereby negatively impacts performance; ironically the opposite of what the whole exercise is attempting to accomplish.
Trouble ticket data is primarily useful for detecting trends, thereby helping an IT team appropriately focus their human capital on issues that will enable their users to be more efficient. Going back to the password issue above, the speed and alacrity with which the IT staff help users change their passwords isn't a useful metric at all. A more meaningful metric would be the frequency of password change requests before and after the installation of a self-service password reset solution that was put in place in response to the analysis of help ticket data that showed that this was one of the most frequent issues and one that could be easily solved with little effort and financial expenditure. Measuring a sharp drop in password reset requests would show that the solution worked and was therefore beneficial to the organization by enabling users to help themselves, resulting in their having more time to concentrate on their primary tasks, and also by allowing IT staff to allocate their resources on issues that are less amenable to resolution via automation.
Unfortunately, in my experience, ticket systems get used to determine useless metrics such as the first example mentioned above, and therefore end up being the bane of IT staff, rather than a useful analytical tool.
Hmm (Score:2)
While not a standard metric, consider the amount of time you spend planning and implementing upgrades to your infrastructure, as opposed to the amount of time spent supporting it. In a healthy company, the weight is towards planning and implementing upgrades to your infrastructure, where not a day passes that a new piece of equipment isn't being delivered. Why, you ask? Because it shows that there are enough people to handle the daily problems adequately, freeing up others to plan for several months (or yea
Turnaround time (Score:2)
If you have to go down that path, I'd go with TAT. Simple to measure even over old data, hard to game consistently, and closely related to customer impact. If you want to, weigh it by customer-perceived severity of each request so that people don't boost their numbers by cherry picking easy tickets.
Tickets don't matter, users opinion do (Score:2)
The best answer is to not use tickets as your metric. I worked at a smaller company and our metric was a simple yearly user survey. We would ask what they thought of their equipment, the how the support team did (response time, niceness, knowledge, etc) and even took it so far as to ask for suggestions on what could be improved. In the end, how many tickets you close really doesn't matter. What matters is how happy your users are. If you are doing the job correctly, the majority of your users will at le
Amateurs (Score:2)
I can only imagine that most people replying with derision about metrics have never been in the position of having to justify what they are doing, and when they have been they have acted dishonestly.
People that actually work in the real world, with real companies with real budgets, and that have some self respect, honesty and pride in what they do will have to justify their salaries or rates somehow, and one of the tools used is some kind of metrics.
A professional will find metrics that are meaningful to bo
Re: (Score:2)
No, just the opposite. The people who have never been in that position probably don't understand why there's so much derision. The people who act dishonestly aren't derisive at all; they're busy writing themselves a new minivan.
Nagios + Redmine (Score:2)
Nagios [nagios.org] isn't too difficult to set up to monitor lots of things, and lots of useful uptime metrics for every service, planned and unplanned maintenance, etc. fall out quite naturally from it. And you can kinda just keep adding modules to it into it and grow it until it's full of awesome.
I haven't personally used Redmine [redmine.org] yet, but have been using Trac and everyone seems to agree that Redmine is the clear successor in terms of lightweight but capable trouble-ticket / project / task management systems.
My metrics (Score:5, Funny)
Here's what you should do... (Score:4, Informative)
First, read Robert Austin's book (Score:3, Interesting)
The points I got from the book:
1) Measuring the wrong thing or in the wrong way makes things much, much worse.
2) Good measurements are possible but take a lot of hard work.
3) Measuring things that are easy to measure is almost certainly wrong.
I also endorse BenEnglishAtHome's comment timestamped 8:55pm.
Revenue and Survey (Score:3)
I manage a department roughly the same size for a company about the same size.
I use two main metrics to see how we're doing:
1. The IT budget should be about 3% of the total company revenue. That's pretty average for companies of this size. If you're significantly under or significantly over, you need to do some soul searching to figure out why.
2. Every year we put together a survey and ask the exact same questions. Its going on 5 years now so we can compare our performance year over year. We ask about 20 questions and score them on a scale of 1-5. Things like 'hows training?' to 'how well does your cell phone work?'
Counting trouble tickets is mostly a worthless exercise. Although, you can manipulate it to your advantage. Start closing 120 tickets this month, 140 next month, etc. When you get to 240 tickets a month you can take that graph to upper management and say, "we're working twice as hard as we were 7 months ago and need to hire someone."
In the end, you have two ways to view this: as a bullshit exercise (and possibly an excuse to fire someone as others have said) or as a way to attempt to objectively evaluate your department.
Re: (Score:2)
Regarding revenue, I meant to add - do you have a revenue generating component within IT? I think there's some novel ways of creating one for most businesses. Anyway, actually generating some form of revenue and growing it can be a good thing as long as it isn't a big distraction.
Optimal (Score:2)
"...determining if we are performing above or below what is considered optimal."
optimal, adj.: most favorable or desirable; best; optimum [Webster's New World Dictionary]
How can you possibly perform above optimal?
Use it to force a new hire instead (Score:3)
There are a number of ways you can do this:
1) For the next few weeks, only deal with issues in the ticketing system that can be resolved quickly. This shows how responsive you are on the "count of problems solved" and "time to resolution".
2) Always upgrade easy problems to "Extremely Urgent", so that they get picked up first (as per above).
3) Do NOT under any circumstances touch a complicated problem that requires consideration or actual work. Find someone to outsource it to. Then blame the outsourcing costs and lack of efficiency (obviously they do not have the same fast response time as you) for the problem.
Seriously: In a 3 man team, you and your manager should KNOW who is working and who is on facebook all day. If you are all working hard, then it is not time to add more pressure by introducing metrics, it is time to hire more help. If on the other hand you are all on facebook all day - well - then good luck to you in your new job at Walmart....
watch your language (Score:3)
Twice you used the phrase "been tasked with". This is not the English you learned in school. Is your wife 'tasked with' washing the dishes?
There is a certain kind of lizard who embraces corporate speak, perhaps to kiss the ass of management or impress co-workers. This is a soulless sort of individual who is doomed to a lifetime of servitude.
If you reclaim your dignity, speak correctly and stand up straight you will either be respected or fired. Either way you will have found a better path in life.
Re: (Score:2)
Nope, that doesn't work either. By rating tech support people on user satisfaction, you're basically asking them to beg/bribe/coerce their users into giving them a high satisfaction rating in order to save their jobs.
The best example of this would be my local Audi dealer...Audi only gives bonuses to staff that gets a 5 out of 5 customer rating, so they practically beg their customers to either give them a perfect rating or to not to bother rating them at all. How are you supposed to give a honest review aft
Re: (Score:2)
That still doesn't necessarily work. My recent experience: I am a senior manager in a non-IT job. I needed access to our SAP system, so asked
Re: (Score:2)
Bingo. This and the first post are the ones to pay attention to. Either nip this shit in the bud or get out of there. You're not going to be able to come up with meaningful "metrics" for a 3 person team. If I help ten people attach documents to emails before lunch and you spend all day helping a team optimize their new production process, who did more meaningful work? With crappy metrics, I did. I've got ten closed tickets and you've got one.
You shouldn't need some complicated process to track and eva