Archive for March, 2010

What Do Instructional Designers Do?

Wednesday, March 31st, 2010

Recently, I saw an ad in the March 2010 Issue of T&D  looking for instructional designers to create “innovative elearning…” It caught my eye because it was looking for volunteers to build training for kids. But it rubbed me the wrong way and reminded me of a continuing concern I have that when people think “instructional designer” they immediately think “elearning.” Maybe some ID’s like that because it gives us a niche…after all, technical know-how is a sure way to define turf and make it easy for others to figure out what you do.

But, instructional design is more than development. I see it as including

  • Analysis of performance and identifying the required capabilities
  • Designing ways of assessing for and developing those capabilities
  • Developing instructional processes and supporting materials (including assessments)
  • Evaluating the effectiveness of the solutions

(The above is really a narrow definition of ID because many of us include “performance consulting” as part of our focus.)

It seems to me that if the focus is only elearning and only development, there are a number of key decisions that the ID will not even be involved in and will only be able to influence slightly (if at all). Not good. You can create nifty interfaces for effective teaching programs that are targeted (potentially) at the wrong things. Sure it can be fun and creative but you won’t be able to reliably get results beyond “neato!”

The key benefit an ID can add to a project is really about capability. What capabilities are needed? What capabilities are there? How can we effectively install those capabilities into the target audience(s)? How can we build supporting materials that really work in the instructional process? Or, even as on-the-job references?

Ultimately, we can identify capabilities that are difficult to develop or critical to successful performance and help create plans and strategies for managing the risk. Often, we can identify ways of simplifying the performance or building tools to reduce the load on the performer, which makes a higher level of performance possible.

Somehow, we need to look beyond (and show our clients and colleagues how to look beyond) outputs and focus instead on the desired outcomes, that is, improved performance. This isn’t a new message…but apparently, everyone still hasn’t heard it.

_______________________________

For more information on the field of Human Performance Consulting, check out ISPI.

Teach to the Test

Tuesday, March 30th, 2010

One of the biggest complaints in schools is that the “No Child Left Behind” act has set up standard test hurdles to be cleared by all schools and students. The idea seems sound — set a standard and then expect everyone to meet it. It allows teachers the freedom to vary their methods but, ultimately, they would be held accountable for results.

In practice, it doesn’t work that way. In the informal conversations I have had with people in the education business, this is seen as a wrong and short-sighted idea. If there is a test, “teachers will not worry about the kids learning…they will just teach to the test.” At its most absurd, an example of this would be, instead of teaching how to add (the core capability) teachers would teach how to add “5 + 7″ because that is on the test.

I would h0pe that the actual test content is not available to allow teachers to directly teach the test answers. But is there any harm in defining in advance the subjects and type of test questions (i.e., the capabilities) to be tested? In fact, that would seem to be the best way to get standardization.

One thing that muddies the waters though, is that teachers are accustomed to having almost no oversight. They are pretty much allowed to do whatever they want in their classroom. In a business, managers who run their organization well, meeting their goals and getting good employee feedback, are often left alone as well. But someone is looking at the results. The standardized tests are the results. So, if they are well-designed tests, they would show which teachers are getting the job done and which are not. Many get concerned because results have never been tracked and reported so publicly before..and anytime you introduce testing it is perceived as threatening because the performers are being asked to 1) trust that the tests are fair and 2) that the results will be used constructively. Often, that is a big leap to expect people to take.

As it turns out, it can be argued that the tests are not completely effective. Of course, to my knowledge, we haven’t taken the step to figure out exactly what we want students to be able to do when they get out of school. If you don’t know what you are shooting for, any attempt to measure whether you have hit the mark is a futile effort.

Another problem is more practical. Tests for large numbers of students are built for electronic grading — so every question has to be multiple choice or some other easy-to-grade format. If life were only multiple choice…it would be so much easier. But figuring out which of four (intentionally unambiguous) options is the right answer isn’t the same as have the capability to do something. It doesn’t prove that you know something other than how to use the process of elimination.

Another problem is that the tests are often administered poorly. In some cases, students with learning disabilities and IEPs (individual learning plans…which means they are intentionally NOT following the same sequence and pace as the standard) are still tested. In one case, students that could not read were forced to take the test but their aide was not allowed to read them the questions. So, they looked at the tests and randomly filled in circles…

Finally, it seems that the tests are intended to measure a minimum standard. But, due to the emphasis placed on them, they are in danger of becoming the actual goal. An effective teacher who is focusing on getting real and important learning to happen should produce students that blow through the simple standardized tests like a trained athlete would ace a basic physical. Administrators and parents wouldn’t need to fret so much about hitting the numbers. (Anecdote: We probably all know at least one teacher who has had an irate parent complain about a poor grade on an elementary school test impairing their child’s chance to get into Harvard. Any society that doesn’t see that as absurd should go slap itself.)

The fix though, is not to discard the tests and go back to the good ole days. The first step should be an analysis of the performance, followed by the design of tests that test capability (not the ability to guess multiple choice answers) related to the desired performance. Then, checkpoints should be designed to allow teachers to track how well they are progressing toward the standard. And, somehow, individual differences need to be accommodated so the test really measures, rather than blindly generating meaningless numbers for administrators to gloat over or fume about.

This is not dissimilar to a standard performance-based ISD approach that we (and many others) use to develop custom training programs. It’s frustrating to see a problem continue when the means to fix it is well understood and available.

Without some kind of test, there is no verification of capability. Performance testing is the most accurate but can be difficult to administer. We still have to decide though, is making it easier to process a large number of poor tests really a better solution?

Excellent Service

Friday, March 5th, 2010

I recently had a problem with my computer. A client I was working with said it was a D620 error. I didn’t get it at first but then realized my computer is a Dell Latitude D620. Apparently, this model has had a problem or two…my client’s company actually exchanged a bunch of them when they first came out due to problems.

My company doesn’t have that kind of leverage with Dell though and the computer was three years old when I started having the problem. In fact, I had a three-year warranty and the problem started about a month before the warranty expired. (Of course, I was too busy to deal with it until after the warranty expired but that’s my own fault…)

I was prepared for the worst…which, in this case, would be hours of troubleshooting, some new parts, some frustration trying to get the fix to work, followed eventually by the purchase of a new laptop. But, for once, that is NOT what happened.

Instead, I got a first-hand look at how Dell does service and it was impressive. Dell’s reputation can be spotty, depending on who you talk to. But they are clearly putting a focus on customer satisfaction and service.

First, I tried to chat with a service support person and right after I typed the problem symptoms and the steps I had taken to correct the problem into the chat window, the chat was ended. “I knew it…they couldn’t solve it so the ‘hung up’ on me!” I wasn’t happy but was smart enough to copy my information so I wouldn’t have to re-type it.

Figuring I would try again, I re-logged in but during the process, my phone rang. Picking up I found it was the service support rep I had been chatting with! How about that!

She did some troubleshooting on line and then had me run some diagnostics and called back to check the results. Bad news…I needed some new parts. The motherboard for one thing. (Which sounded like a big deal.)

After looking at some alternatives, she offered sell me an extended warranty which would cover on-site service/installation of the new parts and a year of accident protection for less than half the price of the motherboard (which was not the only part). My suspicions were raised “what’s the catch?” but I went for it.

Long story short…the support rep called to confirm the parts were shipped. A dispatcher called to let me know a tech would be calling me the next morning for an appointment. The next morning the service tech called to schedule a time. In the afternoon, he came to my office and installed about four separate parts (completely dis-assembling the laptop and re-assembling it in about 30 minutes). And, so far, everything works!

What was so great about that service? First of all, the results. If it didn’t work, nothing else would matter. If you are going to provide any service you have to be competent.

Secondly, the response time. I first gave up my superstitious troubleshooting methods (“maybe if I jiggle this cable while touching the monitor…”) and went online for help on a Wednesday morning. The parts were shipped that night. The tech had them the next morning and was in my office on Thursday afternoon.

Third, the value. This all cost a little more than $200! (Plus, if I drop my computer before next March, I can get it fixed again…) This is a three-year old laptop running Windows XP! But, since we have been in business (2002) we have purchased all our IT equipment from Dell (OK, not cellphones or printers). Based on this experience, we will continue to. The value to Dell is customer loyalty.

Fourth, the communication. This one is almost cliche. Communicate your commitments and then meet them. Explain the process to the customer so they know what is going to happen next. Confirm that the customer is satisfied before moving on to the next task. Fundamental customer service is not hard to learn…but it is still not easy to implement.

This level of service is actually a little bit inspiring. I found myself looking at my own business and asking “how would our service be perceived?” Are we adequate? Are we excellent? Where should we make improvements? I made some notes.