|
Indicators of Quality in Pretrial Small Group Research
The Advantage: 12.05: V.2, 3rd edition
All focus group and mock trial research is not created equal. With the growing prevalence of focus groups and mock trials as assessment tools prior to Alternative Dispute Resolution, it is important that attorneys are aware of what the indicators of quality are in what the American Society of Trial Consultants refers to as "Small Group Research." After all, what your opponent calls a “mock trial” may have been the equivalent of a glorified dinner party with friends and family.
One of the most important questions to ask particularly when deliberations will be important is, "How did you get your jurors?" Were they garnered from want ads? Were they a group of "professional" jurors recruited from a focus group facility’s database? Were they obtained from an employment agency? Are they all college students?
Understanding indicators of quality in small group research is essential to your ability to be an educated consumer of these important practice areas within the trial consulting profession. This column provides some general guidelines for evaluating the quality of small group research and for determining the weight to ascribe its results.
Small group research (SGR) in legal settings typically takes the form of focus groups or mock trials. The terminology and methodological details used in SGR can vary greatly among practitioners. The American Society of Trial Consultants (ASTC) has adopted the following definition of SGR: “Trial consultants use SGR to study individual’ beliefs, attitudes and opinions, and behavior relevant to issues in litigation. SGR is characterized by participant interaction in a group setting.”
1. Larger sample sizes increase validity. Sample size, in terms of both the total number of participants and the number of groups/mock juries, is a key consideration in determining the weight to ascribe to research results. A larger sample size, or an increased number of groups/mock juries, increases the chances that the data observed accurately reflect the true state of affairs. This is particularly true when evaluating verdicts or damage awards. Data from only one group should be viewed with caution since group characteristics cannot be ruled out as the primary cause of the outcome.
2. Random recruiting increases participant quality. Random recruiting methods, such as random digit dialing (RDD), generally result in a more representative sample than other non-random methods. RDD provides superior coverage of the sampling frame and includes many potential participants that are unattainable in list-based sampling, such as individuals with new or unlisted numbers. SGR participants are sometimes obtained through placing an ad in the newspaper, or from a database of former research participants. These methods are particularly vulnerable to self-selection bias, and can produce results that are less generalizable to the population of interest.
3. Participant characteristics should approximate those of the actual jury pool. The closer the characteristics of the research participants approximate those expected in the actual jury, the more valid the research results will be. Ideally, participants are recruited from the trial venue, although this is not feasible in some smaller venues. The demographic characteristic of the participants should also correspond to those of the venue in terms of race/ethnicity, age, and sex. Quality further increases when participants are screened for probable hardships or other characteristics that would likely prevent them from serving as jurors at trial.
4. The results of the research should be properly reported and not overstated. The report should provide sufficient information to allow a reader to judge the quality of the research methodology. At a minimum, this information should include sample size, participant recruitment and qualifying procedures, an overview of how the project was conducted, and an explanation of the data analysis. The results of the research should not be given greater confidence than the research design and findings warrant.
5. The anonymity of research participants should be protected. Practitioners should always use their best efforts to protect the anonymity of research participants.
6. The identity of the SGR client(s) should be protected. Practitioners should use their best efforts to protect the identity of the SGR client(s).
7. Deliberation versus discussion. When considering the conclusions and decisions at which participants arrived, it is important to consider the level of facilitator or experimenter involvement. A facilitator, knowingly or unknowingly, can exert substantial influence in the conclusions of a group in subtle ways. Question wording, actively engaging or arguing with participants, or reframing testimony or evidence are tactics that can be used to influence a group. Although such tactics are not necessarily a sign of poor research quality, and can be used for certain purposes, their potential to influence outcomes should be noted and understood by anyone who is evaluating the research.
< Back
|
RSS