Policy makers and managers are increasingly concerned about the effects of disturbance on wildlife populations, and in situations where management or conservation decisions need to be made, expert elicitation (asking experts for their judgements in a careful, structured process) is a viable way to provide short-term filling of knowledge gaps (we always recommend this to guide research and that expert elicitation is replaced with data in the future!).
We’re pleased to announce the release of a new online training course to help experts. The idea of this e-learning course is for experts to get exposure to the kinds of judgements they will be required to make, before the workshop itself, and in a relaxed, self-paced way.

Elicitation requires experts to make probabilistic judgements, such as median and quartiles, that are difficult and unfamiliar tasks for most experts. As part of the interim PCoD model we use expert elicitation to to assess the potential effects of activities on marine mammals where there are no data available to feed the model. One of the modules for the ongoing PCoD+ project was to create an expert elicitation training package which we are delighted to announce is now freely available online!

For many species of marine mammals, we lack data on how changes in behaviour can result in changes in vital rates (such as survival and reproductive rates). Where we lack empirical data, we conduct structured expert elicitations where experts are asked to make probabilistic judgements on these unknown parameters. Critically, this is done in a structured way to minimise biases (e.g. anchoring, availability bias, confirmation bias and overconfidence), offering improvements over choosing an arbitrary value or asking the most senior expert you can think of! Here are some examples of how we’ve used expert elicitation work in the past:

  • The interim PCoD model used expert elicitation to estimate the effect of different levels of disturbance on individual vital rates for the five key UK species (harbour porpoise, harbour and grey seals, bottlenose dolphins and minke whales). Read the report here.
  • The PCoD Lite project used expert elicitation to expand the model to include both Blainville’s beaked whales and sperm whales on Navy ranges. Read the report here.
  • The interim PCoD model was also adapted using the results of an expert elicitation for use for the Cook Inlet beluga whale population. Read the report here.
  • We have described the expert elicitation process used in the interim PCoD in a paper here.

Expert elicitation is not an easy process and we have found that invited experts are often unfamiliar with the process and how to express uncertainty in their parameter estimates. Therefore, as part of the PCoD+ project, funded by the United States Office of Naval Research, one of the project tasks was to create a course for experts to take prior to attending an expert elicitation workshop so that they can better understand the process and get exposure to the kinds of judgements they will be required to make, before the workshop itself, and in a relaxed, self-paced way.

Take the course

The “Probabilistic Judgements for Expert Elicitation” course is an open-access, self-paced  elearning course. If you are interested in giving it a go then please follow the links below for the five modules:

If you stop in the middle of a module, you will be asked when you start it again whether you wish to continue from where you left off. Please tell us in the comments below what you think of the course, any feedback is much appreciated!

The work conducted to create this course was led by Dr Leslie New and Prof Tony O’Hagan and many thanks to Grifo Multimedia for the screen designs and programming of these modules.

Please note:

  • If you stop in the middle of a module, you will be asked when you start it again when you next return to the module.
  • Some users have experienced difficulties in the final module which seems to be related to which browser and version of flash they are using. We are still investigating this problem. If you find that the module stops running (leaving you with a blank screen with some pulsing circles that does not move on after a minute or more), then simply close the module. You will have finished the course but unfortunately will not receive feedback on your last set of judgements. You can try another browser but will have to re-start the module.
  • Minimum recommended specifications:
    • We’ve tested it on the following computer OS/browser/flash combinations:
      • Windows 8, 64 bit. Java version 8.0.1210.13
      • Chrome: version 63.0.3239.84. Latest flash (embedded).
      • Firefox: version 54.0, flash player version: