The What/When/How/Why of Measuring Neutrals

Balance.

This critical component factors in every phase of dispute resolution. We view it as a critical component for fair, ethical dispute resolution.

The Neutral is the linchpin in dispute resolution. Qualified, capable, professional neutrals provide a valuable service to the parties, and for the dispute resolution process. Credibility depends on strong neutrals, and the parties deserve nothing less.

Balance presents some interesting questions when it’s applied to management tasks:

How do we define balance in measuring the Neutral’s performance?
When do we provide feedback?
What do we use as a measuring device?
When, and where, should this information be confidential?
When, and how, should it be shared?

Example:

Company A (A) has an agreement with Green Dispute Resolution Services (Green) to provide arbitration services. Green randomly assigns cases to its Neutrals. After a period of time, A notifies Green that it would like to expand services to include disputes in other subject matters.

Green contacts the Neutrals, advises them of the new opportunity, and asks the Neutrals to reply indicating their interest and documenting the subject area where they have expertise. A high percentage of these experience Neutrals reply and Green finds it has several times more interested Neutrals than the expansion would potentially employ.

Green conducts an internal review of the Neutrals who applied, looking at performance issues such as:

Decision made on time
Decision includes an explanation of the reasoning, includes compelling factual evidence presented
Decision indicates the Neutral conducted a thorough review of the information presented
Neutral’s performance demonstrates compliance with rules and policies
Neutral’s decisions, over time, demonstrate balance; reflecting a fluctuation or swing in the decisions. In other words, in some cases the Claimant prevails, in some cases the Respondent prevails.

Performance is evaluated as either “pass” or “fail”. Management selects several candidates from the Neutral pool who received “pass” marks in all the performance categories and assigns them to the expanded program

After the decisions were made, a Neutral (Chris) called to express disappointment at not being selected and asked for a reason. Management decided Chris deserved an explanation. Because Chris’s decisions nearly always favored one party, measurement #5 received a “fail”.

Management regretted not providing this feedback to all its Neutrals.

Initial Forum Question:

Is it appropriate/ethical for Management to keep records that allow it to measure the Netural’s Performance?

If no, why not?

If yes,

What should be measured?

What could be measured?

Moderator Bios:

Jo DeMars is President of DeMars & Associates, Ltd., which manages online and face to face dispute resolution programs for companies such as eBay, Professional Warranty Services Corp, Porsche Cars North America, Workhorse Custom Chassis, Winnebago, and Thor Industries.

DeMars is a published author and frequent speaker on topics such as customer retention, ethics and online dispute resolution. In 1995 DeMars was recognized the Wisconsin Entrepreneurial Woman of the Year and in 2005 was recognized a Woman of Distinction by the Waukesha County Foundation. She graduated summa cum laude from the University of Wisconsin-Milwaukee.

Anne Foye

______________________________________________

Return to Cyberweek 2014 Homepage

1 member likes this

▶ Reply to This

Replies to This Discussion

Permalink Reply by Dale Ann Iverson on November 9, 2014 at 12:55pm

Good afternoon! I'm sorry to be so late to this great discussion. Many, many good observations and I didn't want to pipe in until I had time to read them all, albeit not as carefully as I would have liked.

Ben's reply begins to get at some of what I've been thinking. In arbitrating on-line, I try to be a Jiminy Cricket of sorts, sitting on my own shoulder and scrutinizing my biases as I move through a stream of cases. I'm watching those numbers, alright, and I have asked myself, "What's going on here?" Like Ben, I think there are perhaps other things going on besides neutral bias when there are a number of decisions coming down on the same side. Besides the standards for the forum that Ben refers to, there may also be factors at play around the parties and how they participate in the forum. For example, where one type of party regularly fails to "tell their story" at all, or fails to tell it in ways that support their position under the standards, decisions can look slanted.

BUT, all that is to say that doing this kind of evaluation of neutrals seems to me a great step toward continuing improvement of the entire program, as long as evaluators are communicating with each other and with program directors and looking for patterns that suggest other ways for program improvement in addition to addressing neutral bias. Input from neutrals is probably also key.

▶ Reply

Permalink Reply by Katherine Newcomer on November 10, 2014 at 2:16am

Thank you, Jo, for inviting me to participate. While I think that it would be helpful to evaluate the Neutral, I am concerned about the issue of bias based on numerical information alone. I find that I agree with Ben that such a percentage would not properly consider the nature of the diverse fact patterns that the Neutral evaluates within the context of requirements of a particular forum. Beyond that concern, I think that Neutrals could derive great benefit from feedback and an evaluative process. It would provide an opportunity for self-reflection and hopefully lead to better decisions. However, this data should be used cautiously. While it should certainly be provided to the Neutrals, the idea of it being generated for investigative reporters is somewhat chilling. Furthermore, this data should not be provided to the parties in the dispute as it would more than likely be used as a pretext for rehashing disagreements.