Archive for the ‘Uncategorized’ Category

Are You Competent? (Hint: “Not Really” is a Better Answer than “Yes”)

Friday, April 30th, 2010

You may have heard the expression “the problem is, you don’t know what you don’t know” used to describe how unknowns create risk in decisions. (You may also have heard the expression “too bad stupid doesn’t hurt”…but that’s just funny).

In general though, when we hear someone speak “with authority” we assume they know what they are talking about. This assumes that they have developed confidence based on years of study, hard work, and even being wrong enough times to have “learned the hard way.”

But, that is not always a safe assumption. (Are any assumptions ever safe? Never mind, different topic.) Often people who are not that competent over-estimate their own competence. That’s not too surprising. The real surprise is that people who are very competent often under-estimate their competence. As a result, if you listen to people’s own PR, you run the risk of trusting the less-competent individual!

In 1999, Justin Kruger and David Dunning, then both of Cornell University published the results of a study in the Journal of Personality and Social Psychology. Their “Dunning-Kruger Effect” noted that, with a range of skill areas (from playing chess to driving to reading) the following are typical (emphasis added).

  1. Incompetent individuals tend to overestimate their own level of skill.
  2. Incompetent individuals fail to recognize genuine skill in others.
  3. Incompetent individuals fail to recognize the extremity of their inadequacy.
  4. If they can be trained to substantially improve their own skill level, these individuals can recognize and acknowledge their own previous lack of skill.

Another psychologist, C.F. Downing determined that this sort of reverse bias applies to intelligence, with more intelligent people believing they are less intelligent than they are and less intelligent people…well, you get it by now. It leads all the way to “depressive realism” which argues that people who are depressed actually have a more accurate view of what is going on!

What does it all mean? It just means that the more you know, the more you know there is more to know. And, that we should avoid using self-assessment when getting an accurate assessment is important.

Tags:
Posted in Performance Testing and Qualification, Pete's Comments, Uncategorized | No Comments »

What Gets Measured Gets Counted

Thursday, April 22nd, 2010

On Wednesday, March 03, 2010, the ABC station in New York ran a report about a New York City police officer who went public about quotas. Apparently, the police are given specific targets to meet for arrests and summons. The complaint was that the quotas were being enforced blindly…so officers had no choice but to miss their numbers (and be disciplined) or just arrest people indiscriminately to keep their numbers up. (The original story.)

Well, are performance measurements and quotas bad? Large organizations need to manage by the numbers to keep things fair. Don’t they?

From a human performance perspective, there is actually a lot wrong with this approach, whether it is used in law enforcement or other businesses. For one thing, the numbers need to be connected to the desired performance and they need to be under the control of the performer. In this case, the measure doesn’t track the real desired results and it isn’t in the control of the performance because it doesn’t account for situational differences.

First of all, the measure is tracking activity, not results. The number of arrests look like results but it is more like measuring the number of proposals a sales person generates. Yes, there is a relationship between arrests and crime just as there is a relationship between the number of proposals and sales. But, what we are really looking for is a measure that tracks the amount of good arrests or, ultimately, the amount of actual crime.

Counting arrests is a problem because it assumes a constant volume of crime. To establish a required number of arrests for all police officers on all shifts in all areas implies that there is a stable amount of crime and the police can reasonably be expected to solve a certain amount of it. This may be approximately true over time (but probably not) but can’t possibly be true on a daily basis. The performer cannot control their performance on this metric. Unless they cheat.

One part of the bad news is that holding performers accountable for measures that they can’t control breeds cynicism and actually harms performance.

Imaginary dialog. (Italics/parens indicate what the individual is thinking but not saying.)

Sargent: Here is your quota. (I have a pretty good idea about what goes on in his area…I’m glad I don’t have to meet these targets.)

Officer: But we’ve been patrolling heavily and crime is down. I don’t think I can hit those goals, especially during the day. (Surely he knows this isn’t reasonable.)

Sargent: I don’t want to hear about it…just hit the numbers. (I have to maintain a firm hand as a leader. Besides, the people I answer to are so far removed from the daily problems of the beat officer that they won’t hear anything I say about the quotas either.)

Officer: Yes sir. (My only hope is to cheat.)

The result is wasted time, money, effort, and also injury to innocent people. With the side benefit of misleading statistics on record.

This sounds like lots of businesses actually…the farther away from the actual work you are, the less you will be able to understand the issues behind the numbers and the more likely you are to turn it into a clear cut, simplistic question. “Did you make the numbers? No? Then start making the numbers. I insist.” It highlights the importance and the sad lack of knowledgeable management. Maybe even worse is that there is management training out there that will teach you NOT to “take on your employees problems.” Which some people translate into meaning “don’t listen to any explanation or get dragged into troubleshooting.” Which is really not helpful and not managing either.

But back to the measurement question. What should they measure then?

I’m not an expert on law enforcement but if you are trying to measure police performance, how about measures that measure what you really want to improve that are also in the control of the performer? By the way, this isn’t easy but here are a few ideas.

  • You really don’t need more arrests. You either want more convictions (as an indicator that the right person was arrested) or, ultimately, reduced crime, maybe based on reports or complaints by citizens. Some kind of ratio would be a good place to start.
    • A ratio comparing arrests with convictions (or plea bargains) to indicate the quality of the arrests.
    • Ratio of crimes reported to crimes solved.
    • An index incorporating crime per capita, arrests (or convictions), complaints, and feedback from the public.

In a business, besides looking at the performance and setting measures based on indicators that the performer can legitimately control, it is also critical to incorporate knowledgeable managers who can understand the context of the performance and make allowances where appropriate. There is a point as you progress up the “food chain” though, where you lose touch with the day-to-day issues…or even to where the management never had the know-how in the first place.  (For example, how many mayors or public commissioners are former police officers?) In those cases, the higher-ups need to listen to the people on the ground and develop enough trust to have rational discussions about performance.

If you want the measures to drive performance, it is critical to define them carefully and consider the possible unintended consequences because people will really try to make the numbers…even if it might be better if they didn’t.

Tags: , ,
Posted in Leadership, Performance Testing and Qualification, Uncategorized | No Comments »

Excellent Service

Friday, March 5th, 2010

I recently had a problem with my computer. A client I was working with said it was a D620 error. I didn’t get it at first but then realized my computer is a Dell Latitude D620. Apparently, this model has had a problem or two…my client’s company actually exchanged a bunch of them when they first came out due to problems.

My company doesn’t have that kind of leverage with Dell though and the computer was three years old when I started having the problem. In fact, I had a three-year warranty and the problem started about a month before the warranty expired. (Of course, I was too busy to deal with it until after the warranty expired but that’s my own fault…)

I was prepared for the worst…which, in this case, would be hours of troubleshooting, some new parts, some frustration trying to get the fix to work, followed eventually by the purchase of a new laptop. But, for once, that is NOT what happened.

Instead, I got a first-hand look at how Dell does service and it was impressive. Dell’s reputation can be spotty, depending on who you talk to. But they are clearly putting a focus on customer satisfaction and service.

First, I tried to chat with a service support person and right after I typed the problem symptoms and the steps I had taken to correct the problem into the chat window, the chat was ended. “I knew it…they couldn’t solve it so the ‘hung up’ on me!” I wasn’t happy but was smart enough to copy my information so I wouldn’t have to re-type it.

Figuring I would try again, I re-logged in but during the process, my phone rang. Picking up I found it was the service support rep I had been chatting with! How about that!

She did some troubleshooting on line and then had me run some diagnostics and called back to check the results. Bad news…I needed some new parts. The motherboard for one thing. (Which sounded like a big deal.)

After looking at some alternatives, she offered sell me an extended warranty which would cover on-site service/installation of the new parts and a year of accident protection for less than half the price of the motherboard (which was not the only part). My suspicions were raised “what’s the catch?” but I went for it.

Long story short…the support rep called to confirm the parts were shipped. A dispatcher called to let me know a tech would be calling me the next morning for an appointment. The next morning the service tech called to schedule a time. In the afternoon, he came to my office and installed about four separate parts (completely dis-assembling the laptop and re-assembling it in about 30 minutes). And, so far, everything works!

What was so great about that service? First of all, the results. If it didn’t work, nothing else would matter. If you are going to provide any service you have to be competent.

Secondly, the response time. I first gave up my superstitious troubleshooting methods (“maybe if I jiggle this cable while touching the monitor…”) and went online for help on a Wednesday morning. The parts were shipped that night. The tech had them the next morning and was in my office on Thursday afternoon.

Third, the value. This all cost a little more than $200! (Plus, if I drop my computer before next March, I can get it fixed again…) This is a three-year old laptop running Windows XP! But, since we have been in business (2002) we have purchased all our IT equipment from Dell (OK, not cellphones or printers). Based on this experience, we will continue to. The value to Dell is customer loyalty.

Fourth, the communication. This one is almost cliche. Communicate your commitments and then meet them. Explain the process to the customer so they know what is going to happen next. Confirm that the customer is satisfied before moving on to the next task. Fundamental customer service is not hard to learn…but it is still not easy to implement.

This level of service is actually a little bit inspiring. I found myself looking at my own business and asking “how would our service be perceived?” Are we adequate? Are we excellent? Where should we make improvements? I made some notes.

Posted in Pete's Comments, Uncategorized | No Comments »

Too Much Automation

Tuesday, August 25th, 2009
Originally posted on March 28th, 2007 by Pete

There was an article in Wall Street Journal recently about schools overusing automated phone message systems, resulting in some families getting up to five calls in the same night and some with no real information (e.g., “your child is a pleasure to have in class”)!! The desire to communicate but the ease of doing it en masse and automatically.

Even worse than automated calling out systems is the over-reliance on “knowledge bases” and voice recognition for incoming technical help call centers. There is nothing more futile than trying to get a quick answer to a system technical (usually computer-related) question. It will work like this:

1. Waste a bunch of time trying to find your problem in the “knowledge base” of pre-built questions and answers. It is great if they have your question but, if not, you better get comfortable.

2. Waste more time going on a user forum. These have a worse case of the same problem as the knowledge bases–the information is not organized so you have to use a random search and/or browsing strategy which will likely yeild either no results or the teasing promise of results just around the corner of the next click.

3. Waste more time trying to find a phone number to call an actual person. Once you find the number, you will not get a person who can answer on the first call.

You could always email for help on a non-urgent problem. The likely outcome will be an instant reply telling you to check the knowledge base!! (and probably a warning not to reply to that address…and probably a message asking you to take a survey to report how satisfied you were with how they handled your case).

The only thing worse is calling cellular directory assistance. Technology is great but, when you want an answer quickly, technology really can’t match the satisfaction of asking someone who knows. You can clarify, you can discuss other similar situations…in short, you can actually learn how something works so you can do it yourself next time. Using automation for basic, easily answered questions or for triage is fine but have a person who can help when you need it.

Posted in Uncategorized | No Comments »

Go Back Jack, Do It Again

Sunday, July 26th, 2009

Steeley Dan was a pretty cool band–you have to admire their refusal to perform live and their blantant use of jazz-influenced chord progressions. But that is not the reason for the title. Nope. It is because somehow, Yahoo managed to screw up my blog and I have to regenerate the whole thing.  I am definitely not happy but, what can I do?

This isn’t the first batch of grey hairs courtesy of Yahoo. They also bunged up my email. Dig this. I pay for business email. One day they changed it. No notice though…I just tried to log in and it didn’t work.  When I called to ask “what up?” I was told “Oh, we changed business email…now it works like a regular business email account.”  I had to ask “what do you mean?” I was told “Now you log in with your full user name (yourname@yourbusiness.com) instead of your Yahoo ID.”  Hmmm. So far, the only difference is that I get to type more letters…who even cares!?!?  One aggravation was that all the folders full of emails were no longer available from the business account. But the real problem was that whatever they did made it no longer possible to access my business mail from my cellphone!  In fact, it now works on a Blackberry but not an iPhone (which is what I use). The bottom line–I get to pay for less functionality than people get with a free account! At least now I understand why they didn’t notify users before making the change.

There is plenty I like about Yahoo so we are keeping the website here (at least for now). But they are starting to feel like an old Jaguar automobile–more time in the shop than it is worth.

Posted in Uncategorized | No Comments »