Questions For Effective Assessment

QuestionsForLiving interview with Charley Morrow, PhD. April 17, 2012 for implementing effective assessment.

QuestionsForLiving: Were there specific questions that started you on your journey to optimize employee measurement? If so, what were some of those questions?

Charley Morrow: I’ve been working with people measures for more than 25 years. Nearly every day, I see strong reactions to performance feedback, personality assessments, or potential assessments, such as IQ or EQ. Some embrace measurement as a tool for positive change, and others are nervous. Some question the measures, and others hide behind the authority of the data.

These reactions to measurement and data have always fascinated me. I started by asking myself:

• What is driving these strong reactions to measures of employee performance?
  
• What are the implications of these reactions?

Asking these questions led me to an epiphany: The reactions, and the implications of these reactions, are the keys to getting results from measurement systems. In other words, in many employment contexts, the reactions are the impact of the measures. By shaping and forming these reactions, a leader can optimize learning and value from measurement systems.

Measures are just a very precise way of communicating. This is always true.  We do not say a car “doesn’t need much gas,” we use mathematical precision to talk about miles per gallon. Imagine the reaction of shareholders and analysts if an executive talked about “a pretty good investment” as opposed to articulating a percent return on equity. 

Just like other types of communication, however, measures are denotative and connotative.  Beyond the formal denotative meaning, which is often defined mathematically, there are many connotative meanings, which may be more important to employees.  Said another way, the implications of the measures are more important than the measures themselves.  Communication is about two parties making meaning, and conversation is simply the back and forth of meaning. We need to remember this to use measures effectively.    

QFL: What are the primary questions you ask yourself as a consultant to get meaning from measures of employee performance?

CM:

1. What does this measure communicate?
Measurement is communication; its purpose is to precisely articulate something. In organizations, communication is about making and managing meanings between multiple parties to coordinate results. Without shared meaning, communication breaks down. This tenet suggests a need for reciprocal learning about the measure and its object.

Clearly, there are implications of measurement results. Too often, however, the “what is in it for me?” question answered by employees is different from the “what does this measure mean?” question answered by strategists. The answers to these questions lead to multiple meanings in the organization. This is the start of miscommunication and dysfunction.

2. What is the theory behind this measure?
People measures are based on a model or a theory; in technical terms, we talk about conceptual and operational measures. 

The conceptual measure is a theory. It could be a theory about how people tend to react to the environment—this is personality.  It could be a theory about the organization’s business model and how employees contribute to the model—this is a performance measure.  It could also be a theory about how people should relate to each other and themselves to support the organization—this is a competency model. 

The key to improving measures is understanding the theory and ensuring that it is aligned with the strategy or valued outcomes.

3. How is this measure imperfect?
There is no perfect measure—all measures are contaminated, deficient, and imprecise. Given the complexity of individuals in organizations, when measuring employees, the imperfections are even more important.

• Every measure is contaminated by factors beyond an individual’s control.  For example, an employees teammates may positively or negatively affect an individual’s productivity.

• Every measure is deficient in that it doesn’t tell the whole story. In our teamwork example, imagine an employee whose productivity is measured as an individual—say, by the number of calls in an hour, if the employee is working in a call center. If he is helping out a team member, he is contributing, but this would not show up in his productivity numbers.  Surely, helping out someone on your team is of value!

• Every measure is imprecise because it is affected by random events, like the call-center computer going down. There is always an element of error, speaking statistically.  This error is the basis of inferential statistics.

Authentic and productive two-way communication acknowledges these imperfections and, as a result, enhances learning.  Multiple measures allow triangulation to fully understand performance.

4. How will this measure affect people and the organization?
What are the intended and unintended consequences? The very act of measurement affects the object of measurement; measures have intended and positive effects as well as unintended and negative effects.

Despite the good intentions of policymakers and test developers, teachers teach to the tests. Despite the good intentions of managers and talent management departments, employees “game” performance appraisals.

By understanding and managing the context of measurement, we can anticipate and minimize negative effects.

QFL: What questions would you suggest leaders in an organization ask themselves to create and maintain a positive environment through implementing comprehensive performance assessments?

CM: Actually, it is quite close to the questions that you should ask about communication in general. 

• How is this message being received? 
• What is my audience thinking, feeling, or wanting?

At the end of the day, your employees will make meaning from the measures. The question is, how can you affect this meaning? There are a variety of tools to help you with this. Many of them can be found in communications theory.  Most of the tools have something to do with:

• Expressing an interpretation of the denotation as well as the connotation of the measure
• Having the other party talk about the same issues, from their point of view
• Trying to come to closer agreement on the meaning.

QFL: What are the primary questions you ask yourself that allow you to provide high-quality assessment services?

CM:

• What business problem is the client trying to address?
• What change in context—in the internal or external environment—is causing this problem now?

There are a lot of conversations about strategic talent management these days, reflecting a need to be better aligned with the strategic needs of the organization. Too often, assessments are a solution in search of a problem. I find it is better to start with the root cause of the problem we are trying to address. Generally, I find that the answer lies in changes in the strategic context of people in the organization—for example, the organization is going into new markets, is spending too much on employees, has high turnover, or has an engagement problem.

It sounds simplistic, but somewhere in the organization, someone is defining the quality of the assessment by how much it solves their problem. Usually this person is a senior executive who contacts a staff member and asks for an assessment of the company's talent. Often, this directive does not connect the business problem and the assessment. The root cause is the business problem.

The direction of “get an assessment” triggers a bunch of activity, and in the process, the root cause often gets lost. Sometimes, the context and root causes of the request are not even articulated or thought through—someone realizes that assessment can solve the problem but doesn’t take the time to fully understand how the problem and assessment relate.  As a consultant, I provide the best solution by first understanding the problem, and then ensuring the assessment exactly fits the need. You can only hit the bull’s eye when you know where it is.

I’ve also found that understanding the root cause allows the organization to build meaning with measurement systems. When you’re tying to develop an organization or employees, communicating meaning is the key to getting value from measurements. 

QFL: Based on your work with organizational and human performance assessment, what questions do you believe people could ask themselves to make our world a happier and healthier place to live?

CM: This is a tough one. I think a lot about relationships between people, so I would offer two questions that people in relationships—with friends, family, and co-workers—should ask themselves.

• How am I reacting to this?
We have a choice in how to react. Too often we are not aware of our own reactions. Once we take stock of our reactions, we can choose to respond more deliberately.

• How am I personally contributing to this situation, and what am I responsible for?
Too often the blame is put on someone else. In a conflict situation, it takes two to tango, as my dad taught me. You have more control over your own behaviors, thoughts, and feelings than you do over the other person’s. By getting clear on your contribution, you can resolve the situation much faster. With friends, there are misunderstandings; within families, kids react to grownups; and at work, issues are initiated by both managers and employees. Understanding how you are contributing to an issue will put you on the road to addressing it and having a happier life.

You know, in talking about this, I realize these are robust questions. There are a many dimensions and issues in the world.  As humans, we are all responsible for these issues, including environmental problems, geo-political conflicts, and local issues in our towns and neighborhoods.  All of these problems can be addressed by single individuals—as soon as we start asking hard questions of ourselves.

For further information, Dr. Charley Morrow can be contacted at: