A Peer Review is generally known as an evaluation method of scientific publications. Peer reviews of operational situations are used as a tool for organisational learning, in particular during crowd events in order to foster internal reflection and mutual learning. The were "invented" at the Police Academy of the Netherlands. Initially they were used in order to evaluate the policing of international football games, meanwhile the peer review method has been used in several countries and on different occasions, they are an integral tool of organisational learning in the Netherlands and other European countries.
Important characteristics of a peer review are
the voluntary aspect - a peer review must be requested from the command or head of the local unit
the emphasis on learning - unlike methods that focus more on mistakes and responsibilities
Immediacy - as Peer Reviews take place in the event, the evaluators will be at the heart of the event, speak - if possible - directly with stakeholders, participants and citizens and observe the goings on.
The request for a peer review will be (preferably a few weeks before the event) addressed to a coordinator who is also responsible for ensuring the methodological integrity. He or she will then discuss the specific questions which the host (or requesting organisation) wants to be answered by the peer review.
The work process during a peer-review
As the events to be studied may differ in terms of duration, the planning has to be adjusted accordingly. In general a peer-review is carried out according to the following structure: The peer review team arrives on the day before the review is carried out.
Day 1 Briefing and planning
The team members will be met by the host or a representative and be briefed on the planned operation, consult relevant documents and familiarise with sites. The „Peer Review Rules“ are addressed by the coordinator, i.e.: safety first, confidentiality, informed consent, no interference with the operation and the need for an open, non-judgmental attitude and demeanour. For new team members, who have not received a training, there will be an introduction into the methodology and process. Based on the information received, the team will then develop its evaluation plan for observations and interviews, identifies what data will be gathered, in what way and by whom.
Day 2 The event
The team will start the actual data gathering according to the evaluation plan that has been agreed on. Usually, this includes attending briefings, conducting short semi-structured field interviews with police officers from different levels, participants, citizens and other stakeholders (typically between 30 and 100 field interviews are held during a peer review). Observations always include police- citizen interactions. Team members will always be free to go where the evaluation plan requires them to go. They are instructed to take notes contemporaneously and not to rely on memory.
Day 3 Analysis and report writing
The team members share the data that have been collected and discuss their experiences. The discussion is chaired by the coordinator who will ensure that the structure is being followed. The scientific assistant takes notes and summarises the discussion; this draft report is projected on the wall during the discussion so that all team members can check and agree on it.
In the weeks after the peer-review the draft report is finalised by the coordinator and the scientific assistant. It will be sent to the team members for a final check. The final version is then passed on to the host who may indicate any formal mistake. The report then becomes the property of the host.
Further reading:
Dr. Martina Schreiber | massenweise | Zur Helmahütte 8 | 59939 Olsberg
E-Mail: massenweise@posteo.de