EXTANT DATA ANALYSIS

EXTANT DATA ANALYSIS - Brief Description : Extant data analysis directs the trainer or personnel specialist’s attention away from what Thomas Gilbert (1978) calls “ the great cult of behavior “ towards” accomplishments”. While I admit it retaining fascination with behavior, I’ve too often seen colleagues and students capture the subtleties associated with meaningful performances linked to accomplishments. Effective TNA capture the details of optimal and actual behavior and knowledge while at the same in examining result or accomplishments attached to that knowledge or skill.

Extant data analysis is the effort training professionals make to ground front end analysis in performance and accomplishment. Just how much breakage is accurring? For how long has this been going on? How many repair call – backs are there? How many complaints? What kind of complaints are recorded? What about sales figures? What about requests for assistance? How many questions what kinds of questions what do the exit interviews say? The accident reports? The letters of appreciation? Turnover?

The bottom line?

· Extant data is sales not salesmanship

· Extant data is satisfied customers not any particular greeting

· Extant data is better mileage not any one way of tuning a car

· Extant data is fewer call-backs not a particular repair technique.

· Extant data is people in the ballpark not hot dog cooking or grounds maintenance.

· Extant data is the team’s standing in the league not batting stance or ability to scoop us a ground ball.

In the chapter we focus on where upper level management so often places its focus – on outcomes. It is hard to get a Vice President of Marketing and Sales to talk very long about closing technique : he or she wants to look at sales figures. It is difficult to fix the attention of a Board of Directors on kinds of communication skills : they want to see those skills manifested in retention rates for the engineering staff. And Hospital Directors do not want to dally over the details of equipment storage on the floors : they want approvals from safety regulatory agencies. Usually, management attention can be represented like this :

Performance Accomplishments

If, however, sales dip, engineers flee or regulators deny approvals, management will temporarily focus employee behavior on selling, communicating and safety. Then, management concern, for the moment, looks like this :

Performance Accomplishments

During TNA, the trainer must examine the problem from both perspectives, because both yield essential information. During extant data analysis. We look at outcomes and accomplishments and use them to make inferences about current employee performance. In extant data analysis, the natural products of ongoing employee effort are examined to better understand performance and performance problems.

Extant data analysis is appropriate only in those front end situations where you are looking at a performance problem. Because extant data analysis looks at what is actually happening and not happening. The study must be of an longing situation as opposed to the TNA work which accompanies the introduction of a new technology.

Purposes of Extant Data Analysis

1. Scrutinizing Results to Perceive Behavior : The major purpose of extant data analysis is to determine the OUTCOMES of employee effort and then to use them to understand employee performance. Think about the carnival hawker his sells an sings an implores, but few customers choose to visit her exhibit. Or consider flight attendant behavior. All the drinks and food are distributed and collected on time, change is made and returned to passengers, but still there are complaints that this airline’s attendants are less responsive than those of competitors. You could have watched the hawker and the attendants and not known that there was a problem with performance. By focusing on accomplishments, you know there is a problem, even if surface indicators suggest that they are doing OK.

2. Seeking Truth Through Trends : Extant data doesn’t lie. While it may not give a complete picture of what is going on, it does offer a snapshot of the results of what is actually and naturally transpiring at the carnival or in the airplane. When the training professional uses extant data analysis to seek truth, he or she is looking for trends in outcomes which shed light on trends in employee performance. One latter of complaint from a customer or one quarter of weak sales represent a kind of truth to management. The challenge for a personal or training specialist is to determine of that sliver of extant data (the letter or sales blip) is indicative of large truths about what employees are doing or not doing.

3. Using Results to Shape the TNA Effort : These large truths then enlighten TNA, becoming the basis for framing questions for latter stages of the study. Instead of queries like, “ What’s the problem with selling the ‘Salt of the earth’ account?,” you can frame questions which are grounded by reality and detail. For example, “ Third quarter sales of money market accounts, prior to the introduction of ‘Salt of the earth’ were stable, nearly identical to third quarter of last year. Since ‘Salt’ we’re down 11 percent in teller generated new accounts. Why are tellers selling so few of these accounts?” is a better question. It shows you know what is going on and it stimulates mire specific responses. Here’s another example : “Let me quote to you from some letters from passengers. Do you think they accurately reflect passenger treatment?” in the ‘Salt of the earth’ question, the trainer uses extant data to get information on CAUSE of the problem. In the airline example, extant data is grounding the search for more information about ACTUAL flight attendant performance.

4. Matching Corporate or Agency Goals : Extant data may be used the measure the company or agency against its goals or raison. Think about it this way. The nuclear power plant is really about producing energy, doing that for less than it costs to run the plant, doing it safely, and being permitted by in and of itself, is of interest to management. Employee behavior matters, in the eyes of management. For how it contributes to the achievement of organizational goals. That is why extant data analysis is such a powerful tool for negotiating with management employees an even unions. The issue it not what employees swear they know or do; it’s how it turns out in light of the goals that have been established. The issue is the results of the a great of employee effort in comparison with the reason’s that the group exists. Do employees respond properly when an alarm activates? Do passengers report having had a certain kind of experience on the airplane? Are the accounts being sold by tellers? The Human Resources group can train ad nausen, and new employee behavior might new result, but applause will come only when all that effort results in ACCOMPLISHMENTS LINKED TO GOALS.

5. Saving Money While at the Same Time Gainning Understanding of the Problem: Because extant data analysis relies upon examination of existing information within the company or agency, trainers avoid the cost of generating new opinions and data. When we use the extant data analysis technique, we gather no new information. No interviews. No interviews. No questionnaires. We seek nothing which is not already and naturally present in the organization. What we want is ACCESS to records, files, forms and print-outs, usually collected and retained for purposes other than analysis by training professionals.

6. Veryfying What You Hear During TNA : As you carry out stages of TNA, you will often hear, “They don’t know how to …..” or “We know all about that. We don’t need training. We need more … or better … “ Use extant data analysis to go back and find out if they do, in fact, know how to explain the account or fill out the form. Several years ago I got involved in a TNA for word processing training. The wood processors told us they knew all about the system, that there was no need to bother with training. If there were any problems, it was with the system and with their supervisors. What we did was examine the documents they had already created on the system for errors and frequency of kinds of uses. Inferring back from this extant data, we were able to verify their statements and to derive a clear picture odd what they could actually do as well as what they chose to do with the system.

Description of the Extant Data Technique

Extant data analysis …..

· Unearths the results of employee behavior

e.g., sales figures, accident reports, enrollments

· Makes it possible to determine the relationship between employee effort and organizational goals

e.g., matching sales with expectations

· Assures that internal. Regular, corporate data is part of the front end inquiry

e.g., accident reports or exit interviews as part of the study

· Involves cajoling and negotiating for information

· Is gathered and examined by human resources professionals but not originally generated by them

When do you use this technique ?

Early and repeatedly when confronted with performance problems in ongoing situation – usually not new systems, product or technology.

Under ideal circumstances, a supervisor or manager has used extant data to determine the details of the problem be for you are assigned to it. He or she then provides you with and initial description of the situation based on review of the records. That doesn’t always happen.

What most often happened is that one slippers of extant data ( like an accident or a letter the CEO) gets to the momentum going. “ Put together a course on this thing so we don’t have anymore accidents!” or “ do something on communication for our flight attendants.” It’s obvious they don’t know how to keep our passengers happy.” Before doing anything, make certain that you have first taken a close and careful look at all natural, related records and information.

Once you have examined records and are now involved in all the stages of TNA, you will want to keep referring back to natural records. Extant data is a reality touchstone; it doesn’t lie, especially if it is based on results of employee performance gathered over time. For example, you may be told by employees that they know all about the ‘Salt of the earth’ account. Go back to the extant data to check the records and see if there are any customer complaints about inaccurate information. Are the cards filled out correctly? Have customers been properly qualified? Match what you hear from sources during needs assessment with extant data. Pursue discrepancies.

How Is Extant Data Analysis Done?

Extant data analysis is a sleuthing technique which gets developers and trainers to outcomes of performance. Extant data analysis is unlike other front end technique because it involves use of INFERENCE, OBSERVATION and PERSUASION only. Trainers will not interviews or survey, instead they will pour over and paw through records and files. The keys to the technique are figuring out what kind of information you need, determining where it is, gaining access to it and incorporating what you find.

Here is a series of steps which will enable training professionals to carry out this technique :

Step 1 : Examine the job and its outcomes

Step 2 : Identify quantitative results of the job

Step 3 : Identify qualitative results of the job

Step 4 : Determine how to get extant data and eradicate obstacles

Step 5 : Examine the data

Step 1 : Examine the job and Its Outcomes. Examine the job, focusing attention on the duties or tasks which have been identified as problematic. Think about what employees do, might do, and the opportunities or challenges with which they are confronted. For example, look carefully at the tellers’ opportunities to sell ‘Salt’ accounts. Examine the materials that they have been given to give to customers who inquire about “Salt of the earth” accounts. Look at recent directives related to customer wait time. Has there been any recent corporate pressure to diminish the moments that customers stand on line? What else is going on in the branches now? Has there been a major ad campaign that might increase traffic for purposes other than ‘Salt?’ Are there any other new , competing products? When tellers were informed about ‘Salt,’ what else did they learn or get?

Step 2 : Identify Quantitative Results of the Job. List the tangible and possible quantitative outcomes of that portion of the job. There are en route outcomes and there are terminal outcomes. For example, an en route outcomes is the forms that the teller must fill out to initiate an account. The terminal outcome is the number of accounts sold and the size of the accounts. Word processing provides another example. En route quantitative outcomes would be the telephone and electronic mail questions logged regarding the system and its uses. A terminal outcome is the number and kind of documents generated per employee. Establishing quantitative outcomes is based on corporate or agency goals. It is en route outcomes. What the company is seeking to achieve is job stability and statifaction. If that is the case, then a more appropriate terminal outcome would be the number of request for transfer and information collected during interviews. Note that this step focuses on the kind of results which can be counted and measured objectively. When you begin to seek subjective information. For example, the kinds of aggregate feelings which might appear on exit interviews, then you are talking about the qualitative effects of employee performance. That moves us to Step 3.

Step 3 : Identify Qualitative Results of the Job. List any likely reports of qualitative impact of performance on people. What are others, like customers or users, saying? Are there letters or telephonic comments that have been collected? What about performance appraisals or exit interviews? Most companies or agencies gather information about others’ responses to t hem. What are they? Where are they? The classic example of this kind of subjective and important data is letters of complaint and appreciation. For whom? What are they saying? Remember that you are seeking the records and natural collection of opinions and responses. When thinking about qualitative outcomes, it is important to do more that count comments. You will be pressed to do a content analysis, a serious examination of the recurring and often subjectively derived themes within the extant data.

While both steps 2 and 3 involve systematic creation of lists, don’t forget the richness of what Joe Arwady calls “ eureka finds.” There are piles of interesting indicators hidden within companies and agencies : old newsletters, esit interviews, requests for transfer, union mailings, etc.

Step 4 : Determine How to Get Extant Data and Eradicate Obstacles. Now that you have a list, you need to do something with it. Where is the information? Who has it? Who else has it? Will there be resistance to your efforts to dig into files and peruse computer print-outs? Have there been any reports produced for other purposes which might relate to the problem or situation?

Not all extant data is equally accessible. Usually, you can gain access to en route outcomes more readily than the extant data which is very close to the bottom line. The company might, for example, give you copies of the account slips and cards that have been filled out by tellers. Where they will balk is at the computers print-out which present sales per teller. Per supervisor under branch.

It is not unusual for managers throughout the company to want to know “why somebody from training wants to look at accident. Breakage or cold call reports?1?” you might have to construct elaborate justifications to gain access to extant data. I remember a performance appraisal project like that. Upper level management was dissatisfied with the quality of performance appraisals filed by middle managers. They turned it over to training and to an external consultant. They wanted a course to fix the problem. Before progress could be made, a verifiable picture of actual, current middle manager performance on these appraisals was necessary. It made sense to examine randomly selected performance appraisals that had been submitted over the past 18 months. Sounds reasonable? Sure. Still, the group had to justify, implore, reason and nearly beg to get their hands on this extant data. Finally, with names appropriately masked, the training group was allowed to serutinize those appraisals and use them to infer actual middle manager skills and knowledge. If you are clear about what the extant data does for you, how it contributes to your TNA purposes, you will be more likely to be able to make a clear and compelling case to examine that data.

Step 5 : Examine the data. What are you going to do with it once you have it? Look back at the list of possible uses for extant data included at the beginning on his chapter. Which are appropriate in your situation?

In the performance appraisal example, hand in hand with a subject matter expert from the Personnel group, you would examine these randomly selected appraisals for frequently recurring problems. Where are they? On which lines? What exactly are they? Is it a lack of specificity? Failure to use behavioral statement? Failure to substantiate? The exact nature of the erros which appear on the forms must be analyzed and then summarized.

Let’s turn to the ‘Salt’ example. It is possible that extant data analysis will prove that tellers are filling the usual number of cards and that they are filling them out people who are qualified by income and circumstance to purchase the ‘Salt’ account. If that is the case, you need to ask questions about why they don’t qualify purchasers. In that example, extant data directed the instructional designer to seek very specific information on cause.

The flight attendant problem might involve very different existing records. For that case, let’s say the trainer examines all the un solicited customer letters that have been received at the home office in the past there months and analyzes them for their content. The following issuses, concerns and complaints about flight attendants were raised by letters writers. The percentage figure indicates the percentage of letters in which the comment appears.

· Not friendly enough, curt 17%

· Inadequate food and beverage service 07%

· Improper safety orientation 04%

· Unprofessional appearance 10%

· Uninformed about services, frequent flyer options 19%

· Enforced rules inappropriately 04%

· Unable to help on information about connections 11%

· Miscellaneous problems about failure to meet needs 04%

The content analysis of the letters clarifies the general areas which must be studied during this TNA. What exactly did the letters say the attendants actually did which was incourteous? What did they do which made the flight less pleasant than anticipated? Partial answers to these questions come from the letters: others must come from interviews with customers, supervisors and attendants, providing a reasonably detailed picture of what is actually transpiring on the plane. Additional information comes from observations of the attendants as they communicate with passengers. Can you look at the above list and begin to see the performance problem(s) which might be addressed through training?

Analysis of extant data provides information on what employees are actually doing, at least in the opinions of the customers who choose to write to the airlines. What needs additional confirmation? As you look at these figures, you are probably wondering WHY they are or are not doing something? What are the implications of what you have discovered?

Not too long ago, a state legislature passed a law mandating “informed patient consent” prior to surgical procedures. This meant that the medical establishment was responsible or providing patients with information that would enable patients to make intelligent, individual decisions about their medical treatment. Especially surgical treatment. Universal Hospital is expected to serve as a model enterprise in pre-surgery patient education. Imagine that you are the training director who is responsible for painlessly training 107 physicians to do this competently. Interns and residents will also participate in the program. The law has been in effect for nearly four months when the training director is tasked.

Step 1 : Examine the job and its outcomes. We are focusing on the interpersonal interaction that accompanies information about common surgical procedures. Usually it occurs in hospital room. Often with family members present, and rarely is received as a welcome event. Most patient are nervous, uncomfortable with the problem and the impending solution, frequently viewing the surgery as the lesser of evils. Some patients are in awe of the physician, doubting that they have even the vocabulary to pose good questions and stating that they don’t want to waste his/her valuable time. Physicians, although most admit to the benefits of an informed patient, were not supportive of the legislation. They raised questions about just how much information would add up to an informed patient.

Step 2 : Identify quantitative results of the job. The objective terminal outcomes of this portion of the job are : medical record/recovery f the post-surgery patient, number of malpractice suits, size of settlements, number of complaints to the hospital and legislature ; and selection of the same surgeon for other procedures. En route quantifiable outcomes are : the number of times the physician must come back to the patient to secure permission for the surgery, length of contacts, time elapsed between explanation and consent, number of times that the individual changes his/her mind about the surgical procedure, reports of additional questions to other staffers about the procedure ; and number and length of family contacts with the physician regarding the procedure.

Step 3 : Identify qualitative results of the job. The most notable and useful qualitative result of this portion of the surgeon’s job is a satisfied, informed patient who feels he/she has been treated well. Extant data indicating that kind of physician and hospital accomplishment are letters and calls of complaint and commendation from patients.

Step 4 : Determine how to get data and eradicate obstacles. Terminal, longitudinal data like deaths and repeat choice of the physician is unavailable after only four months. En route quantitave indicators have not been collected by the medical or nursing staff, although they admit such information would be useful to have.

The training director turns to qualitative extant data for a picture of how well doctors are actually handling this new and expanded responsibility. Letters to the legislature are unavailable but those to hospital administration are readily turned over for perusal. Since patients are well aware of the new legislative thrust usual. The Assistant Director of the Hospital is willing to allow the training group to examine the letters it a promise of confidentiality from the training director.

Step 5 : Examine the data. Just under half of the 24 letters were complimentary. Patients felt that their physicians eased their minds and informed them prior to the surgery.

The other half were not at all satisfied with their educational experience. The problems, when subjected to content analysis predominate as indicated below. The percentages reflect their appereanced in the 13 letters which are judged as generally negative.

· Vague and confusing information 69%

· Weak or absent post-surgical care explanation 54%

· Failure to solicit or respond to questions 38%

· Lofty and technical vocabulary 61%

· Discomfort with role of educator 23%

· Made patient more anxious 15%

· Incorrect or biased information 23%

· Got no information, took no time 0,8%

A narrow majority of the patient who chose to communicate with the hospital are not satisfied. They do not feel that the physicians are in compliance with the law, and say so in useful detail.

They also are reasonably clear about the problems they have with the doctors performance on this aspect of their jobs. Collapsing the areas of lofty vocabulary and vague and confusing information provides a clear picture of what patients think is the number one problem with their doctors as teacher. The other recurring problem is the doctors failure to provide systematic and memorable information on post-surgical hospital and home care. A distant third is the physicians omission of time for the patients to ask questions.

What are the implications of this information? The training director decides to verify these findings through follow-up interviews with patients and physicians. Given the OPTIMALS spelled out in the legislation and further specified by hospital administration, if this information on ACTUAL performance holds up after latter stages of assessment, she has clear directions for a lean training program.

However, before she trains these doctors in use of concrete and familiar words, examples and other educational techniques, the trainer must ascertain WHY they are not doing it satisfactorily. Since almost all of them are trying (only 1 out 13 letters complains of no effort) and doctors agree that educating patients is important (thought it shouldn’t be legislated), why are they doing it poorly? Don’t know what to say? Don’t know how to say it? Can’t tell appropriate from inappropriate detail? Same problem with vocabulary? Think there is no time to do it right? Try to say too much? Are they attempting to comply with the letter rather than the spirit of the law? That takes us to other TNA techniques, techniques which are appropriate to the search for the causes of performance problems.

Conclusion

Extant data analysis is the front end technique which directs attention to outcomes. Then based on those outcomes, inferences are made about employee skills and knowledge. In extant data analysis, we look first at what employees accomplish, then at what they have done to bring about those effects.

Outcomes ------ (inference) ---à Performance

A certain kind of information comes when you ask an employee about his or her ability to do the job. Another kind of information comes from watching employees do their jobs. This chapter has been about still another kind of front end technique and data. It is the powerful perspective and information which is derived from looking at outcomes and then inferring back to the nature, quality and quantity of employee performance. Extant data grounds the training professional in the real world surrounding the performance problem.