Nominal Group Technique Expert Committee

 

In order to develop a scientific and evidence-based evaluation framework, the Ontario CRISM Node collaborated with a number of international drug policy experts and stakeholders who formally participated in an international committee to support the development of the evaluation framework for the decriminalization of illicit substances in BC.

Purpose

The purpose of the international committee was to bring together world-renowned experts in drug policy and decriminalization to contribute to the development of the evaluation framework. As decriminalization is a complex policy that will likely have impacts on a vast array of sectors (e.g., healthcare, criminal justice, social services, etc.), it is important to draw on international experts' knowledge and experiences of decriminalization elsewhere to ensure that the evaluation design and framework capture all potentially impacted domains.

General Process

Initial recruitment for the committee relied on content experts from the Ontario CRISM Node research team who proposed members with known expertise and experience with international drug policy and decriminalization initiatives. Each committee member completed a Conflict of Interest (COI) form to identify any perceived or real COI that could create risk of bias in making decisions and drafting recommendations for the evaluation, and committed to the process of developing the evaluation design, as outlined below.

The international committee was a time-limited group established to support the development of the evaluation design and framework. The international committee was comprised of approximately 15 international experts who came together in a three-step virtual conference process to contribute to the design. The organization of the process employed a Nominal Group Technique to triangulate knowledge and reach a consensus on the evaluation design, as this specific Delphi technique has been shown to yield better results than purely interactive groups. 

The three steps occurred between November 2022 and January 2023, and are as follows:

1. Step 1 consisted of the distribution of a draft evaluation framework design which outlined the proposed evaluation and provided Likert scale questions asking each committee member to rank how effective they believe the proposed design will be for evaluating decriminalization, and asked members to provide feedback and suggestions
2. Step 2 consisted of the core research team collating and integrating committee member's responses and feedback, and revising the draft framework with these suggestions in mind
3. Step 3 consisted of a one-day virtual meeting where the revised framework was shared with committee members via the nominal group technique where committee members participated and came to a consensus and finalized the evaluation design