Introduction Risk Assessment Method Threatening Letters: A Structured Approach for Disturbing and Threatening Letters

: The Risk Assessment Method Threatening Letters (RAT in brief) provides assessors such as forensic psychologists working for law enforcement and justice departments with a structured assessment method for threatening letters. On the basis of an assessment framework and guidelines, threatening letters will be analysed according to eighteen characteristics. The purpose is to arrive at a risk definition regarding the chances that a person will resort to violence, and to apply risk management to individuals who make threats. With the development of the RAT, efforts have been made regarding the identification of threat level of persons whose identity has not yet been established or who have not been detained, due to the absence of a criminal file. With an assessment form, the presence and relevance of eighteen characteristics divided into: content words (9), emotion words (5), linguistic features (3) and details (1), are examined in disturbing and threatening letters. The content analysis allows assessors to encode a text in binary units of measurement (present or not present) and is based on a combination of empirical knowledge and professional judgement. The objective concerns individuals who are posting (repeated) threatening or disturbing letters on social media and whose authorship must be reviewed to reveal background characteristics for the writer, in the case: 1. they are written anonymously; 2. they are intimidating in nature; 3. they undermine faith in social institutions. Its scope includes public figures such as representatives of the government who frequently are subject to threat or violence. However, the RAT is equally useful for the interpretation of threats aimed at other persons in whose cases the nature and seriousness of the type of threat is also of importance.


Introduction
The assessment of digital and handwritten threatening letters whose authorship must be reviewed to reveal background characteristics, requires an investment of the National Police or for example the Public Prosecution Service. How can a decision be made for further action of individuals who make threats and which words signal a greater risk that the individual may resort to actual violence? The internet invites us to communicate online, and threateners use the possibilities offered by social media too [11,19]. Victims and the people in their personal circle can experience feelings of fear or distress as a result of receiving a threat or they can feel intimidated, or experience social restrictions in what they think and what they would like to do [5]. Inappropriate letters or mails could influence the public debate when the fear of (repeated) threats obstructs an open discussion. Persons who make threats do not see representatives of the government as the solution to problems, but as part of the problem itself. Such persons are intimidated over the phone, physically threatened and their private information is made public on social media [25]. On the street they are accused of conspiring against the population and undermining faith in social institutions. Social media play a facilitating role here; people with divergent ideas can easily find confirmation for their views [25]. The early signalisation of serious threats is relevant for a structured approach of digital and handwritten threats [12].
The aim of this research was to realize a methodology that can assist assessors in better assessing the seriousness of handwritten and digital threats against people in the public sphere [29]. Behavioural experts or other assessors with, for example, the police, have to formulate an initial assessment on the basis of a minimal amount of information, such as threatening messages on the internet. Clues in texts could shed a revealing light on the intentions and/or existing psychopathology on the part of the writer, and for that reason it is important to analyse them [13,23,29]. Such knowledge could be instrumental when offering better protection to politicians, for example. And the more accurate the prediction, the better the police will be able to match security measures to the person under threat. Another reason is that risk assessment in relation to threats often makes use of the experience, knowledge, and intuition of the assessors. Aside from the usefulness of these aspects, such a -from a practical point of view often unstructured -approach also constitutes a risk, as it leaves room for divergent interpretations [7].
The introduction of this methodology, referred to here as the Risk Assessment Method Threatening Letters or RAT, is based on the PhD thesis Het Woord en de Daad [12] which looks at threatening letters against public figures whose security and unhindered functioning are in the national interest, e.g. public figures, such as politicians. The main idea of that study was twofold: firstly, to obtain new knowledge regarding the characteristics of digital and handwritten threatening letters in order to better assess the intentions of the letter writer, and secondly to investigate which of these characteristics could be linked the most to the chance that someone is arrested on suspicion of committing a violent act. 1 The results of that study were used to develop the RAT, about which more later.
There are few scientific studies that look at the assessment of texts to look for risks related to future violent action that assessors can use in practical risk assessment. An empirical instrument that is partly similar to the RAT, is the CTAP-25 or 'The Communications Threat Assessment Protocol' [18]. Both instruments are designed for use when there is very little information other than the communication itself. The CTAP-25 is aimed at problematic written, spoken or online communications, of specific risk factors associated with a higher risk of unwanted outcomes. The focus of the CTAP-25 is on the information contained in the content of the communication. It is a threat assessment instrument used to identify problematic behaviour. The CTAP-25 contains 25 items that are scored for incidence for example in the utterances of an employee, with the goal of assigning low, medium, or high priority to the case. The qualification criteria of the CTAP-25 are described and intended for risk managers and supervisors who within their field have risk management as one of their responsibilities. Another instrument is the VRAW, 'Risk Assessment of the Written Word' [6]. The VRAW was developed specifically for American universities to improve the assessment of violent incidents such as 'school shooters' [15]. Contrary to the CTAP-25, no specific qualification criteria are needed to use the VRAW. The VRAW consists of 5 factors and 5 sub-items that are coded for presence or absence. The number of factors scored determines the level of seriousness on which basis action is taken. Aspects such as rejection or seeking justification are important items in the VRAW [6]. Different from the CTAP-25 and the VRAW, the RAT is aimed at the question whether the writer, in case of repeated letters is the same author, or whether a digital or handwritten threat might actually resort to action, and which specific linguistics could be clues for this. The focus, therefore, are besides content words, especially emotion words, function words and details like micro markers as interpunctions and capital letters. These risk factors are combined together in eighteen linguistic characteristics that are assessed in terms of relevance, which results in a risk assessment for the writer of the threat. The RAT was developed for assessors such as forensic psychologists working for law enforcement and justice departments. These are digital or handwritten threatening letters whose contents could cause social unrest or be considered undermining to rule of law. The current form of the RAT was evolved since 2012 through the assessment of more than five hundred digital and handwritten threatening letters against public figures collected from the prosecutor's office of the Ministry of Justice and from the National Police.
In summary the RAT is developed by a forensic psychologist and based on new research conducted by the author and on the theoretical assessment framework of academic literature described in the PhD thesis Het Woord en de Daad [12], in which eighteen characteristics, or variables, are assessed for the extent to which they are risk-increasing for the chances that a person might resort to violence. Violence is understood to mean: assault, attempted murder, illegal possession of weapons, and vandalism. In this case the criteria for the selection of characteristics were: which characteristics of threatening letters are related to future criminal behaviour? How can they be operationalised in order to make them applicable in text assessment? Before introducing the RAT, this article will first discuss the dataset (2) and the underlying methodology and techniques (3). Then the RAT procedure is presented (4), followed by conclusions and discussion (5) and recommendations (6).

RAT Dataset
Prior to the dataset a scientific literature study was carried out regarding the relation between words and actions, which resulted in a theoretical assessment framework for threatening letters. This study assumed that there are connections between background characteristics for the writers and their use of language on the one hand, and the future actions of the writer on the other hand [12]. In order to have the theoretical framework tested, a total of 450 (digital as well as handwritten) threatening letters were collected, addressed to people such as politicians or members of the Royal Family who by reason of their public function attract a great deal of attention and are consequently the frequent victims of threats or violence.
After a formal request for access with the Public Prosecution Service, access was granted to this data, made available in part by the Ministry of General Affairs, and National Police. The dataset was collected between 1999-2015. A number of letters were doubles, due to the fact that they were processed both as a physical and as a digital item. After removing these doubles (172), 278 letters remained, written by a total of 150 individuals. 2 109 individuals (73%) wrote once and 41 persons (27%) were responsible for repeated subsequent letters. These 41 persons wrote 169 letters, an average of four letters each. Recurring writers were identified on the basis of looking for similar signatures. A large number (66%) was signed using both first and/or last name, and sometimes including an address (19%). In a number of cases some letters were anonymous, or signed with a false name (15%). Only 53 (35%) of the letter writers (150 individuals) could be verified by the police systems to see whether after writing the first letter there had been instances of arrest on suspicion of committing violence. 3 Of these 53 persons the names as well as addresses were known, which proved the deciding factor in being able to locate these persons in police systems and possibly signalling them as suspects. 4 In terms of time between the first and last registered incident 5 for which the suspect was arrested, this period turned out to be almost 27 months. This high average is caused by the fact that for some letters there was a longer period, between the threat and the suspect's arrest and the median turned out to be 18 months. To test the data, a questionnaire was developed on the basis of the assessment framework. The 278 collected texts were analysed in terms of incidence of the 18 characteristics (Table 1). For the background characteristics, the study by Smith [26] served as a precursor; this study showed that certain words could be correlated with future violent actions. For the selection of linguistic characteristics, knowledge from forensic linguistics was used [4,13,32]. Examples of the way in which characteristics were made quantifiable by the questionnaire were: threat classification (direct, conditional, indirect threat, no threat), violent offences (vandalism, possession of arms, assault), emotion words, conjunctions, details and selfreference. The characteristics in the questionnaire were coded for present (1) or absent (2). The letters were assessed by two independent assessors [3]. 7 The Cohen's kappa was used to visualise the degree of agreement between several assessors. All calculated kappas combined together, resulted in a kappa of 0.74 (Appendix). Kappa's below 0,5 were excluded from the data analysis, because this is seen as a less reliable score [3]. The characteristics to which this applied, were positive coping (seeking help) and prosocial involvement (offering help). Negative coping (personal loss) was also not included, because the inter-assessor reliability had revealed an overlap with the characteristic of powerlessness. Powerlessness had a higher kappa than the characteristic of negative coping, and was therefore also excluded in the analysis. From the eighteen characteristics, four were excluded from the analysis, either because of low kappa, or because it was not possible to establish the kappa. In total, fourteen characteristics are included in the model. To quantify linguistic domains, self-reference indicators (pronoun 'I') and conjunctions in the letters were counted. This continued on the work done in studies by Pennebaker [24] and Chapman [8]. Words such as 'I' [24], give an indication of state of mind, and self-reference. The presence whether conjunctions have been used by the writer, are relevant because they indicate whether persons are speaking the truth, which is also important in the assessment of threatening letters. Emotion words reveal emotions and desires, and negative emotions such as hatred and revenge in particular are related to aggression and actions.

Methodology
The self-references were divided between three groups to compare the number of self-references (pronoun I): the first group (1)(2)(3)(4)(5), the second group (6-10) or the third group (11)(12)(13)(14)(15). This was related to a relative frequency: the number of selfreferences per ten lines. Drafting the assessment framework, creating the questionnaire, and analysing the letters took six months in all. 8 All cases were processed in Statistical Package for the Social Sciences (SPSS). 9 In this way it was possible to apply anonymised data analysis to the letter writers. For the descriptive analysis a chi-squared test was used. 10 In order to assess which characteristics were decisive with regard to committing a violent act or the chance of repetition, logistic regression analysis [22] was applied (Table 2). 11 For the logistical regression analysis the model only included those characteristics or predictors that are described in the literature as being related to the chance that someone might carry out the actions they describe (Table 1). For repeated letters (n=133) not the entire dataset of letters was used (n=278), but instead the set with the number of people that wrote letters (n=150), so that someone who wrote more than one letter is not incorrectly counted more than once. Furthermore, only letters with a known date were selected. For repeated letters the analysis was based on the characteristics of the first letter. This resulted in a total of 17 letters, which brought the dataset for the assessment to n=133. With regard to persons arrested for a criminal offence, only those persons were selected (n=53) of whom it was possible to establish with certainty whether they have, or have not, been arrested for a criminal offence such as committing violence. Because the model consists of many independent variables (14) 12 , prior to the first regression analysis for threat and in order to make the model more testable, Cronbach's Alpha (α) was used to see whether a number of variables could together constitute a single scale. This resulted in a Cronbach's Alpha value of α=0.653 for 'hatred-revenge' and α=0,656 for 'state of mindpowerlessness' and justified reducing the number of characteristics in the model (from 14 to 12 characteristics), into two respective new scales. 13 To assess whether the contents of the first letter could also be a predictor for repeated letters, threat was added for persons who write repeatedly, so that the test model (in the second column) now contains 13 characteristics instead of 12. For the final regression analysis (third column), both the factors 'threat' and 'repeated letters' were added to the model to explore whether these characteristics would improve the model; consequently this column contained 14 characteristics. The third analysis was more exploratory in nature, using a Forward Wald selection procedure, contrary to the first two regression analyses which consisted in fairly large datasets (n=278, n=133). The reason behind this was the fact that the dataset was significantly smaller (n=53) and the results would otherwise be difficult to interpret due to multicollinearity. 14 To apply the Forward Wald 10 The chi-squared test was used to establish whether letter characteristics were interrelated or significantly different from each other. 11 Logistic regression analysis is used to establish whether there is a relationship between one dichotomous dependent variable and a number of independent variables. A dichotomous variable is a variable that can have only one of two values as output, for example 'yes' or 'no'. 12 Characteristics with a Kappa below 0,5 were excluded from the analysis, not only because they are less reliable, but also because the model would otherwise contain too many variables. 13 For the other characteristics the Cronbach's Alpha values were below 0.450 and therefore too unreliable to create new scales for as well. 14 Multicollinearity is a statistical phenomenon in which two or more explanatory variables in a regression model show strong correlation, which means that at least one of them can be predicted using the model. Multicollinearity influences the it was established in three steps which characteristics in this model were significant [22]. The independent characteristics were added one by one, and at each step it was analysed which characteristic would improve the model.
In brief, the analyses showed that in particular the characteristics hatred-revenge (p<0.05) and confusion (p<0.01) were significantly related to the chances of being arrested on suspicion of a criminal offence. The characteristics modi operandi (p<0.01), detail characteristics (p<0.05), and hatredrevenge (p<0.01) were significantly more often present in threatening letters, whereas confusion (0.05) was significantly more present in persons who write repeatedly. Contrary to the literature on the topic, the characteristics powerlessness and fixation were not significant predictors in the threatening letters (more on this below). The characteristics that were significantly risk-increasing calculation of coefficients, given that in such a case the characteristics show a partial overlap at least. 15 The characteristic of 'cause fear' was difficult to calculate because of overlap with other coefficients. For that reason it was removed from the model so that the total number of characteristics in that column is 11 instead of 12.
16 For the purpose of this analysis only microfeatures were assessed, including: uppercase, location, time, date, and numbers. 17 For repeated letters in the second column the characteristic of 'threat' was added to the regression analysis as an independent variable in order to determine whether this characteristic might be significantly associated with repeated letter writing. For arrest for criminal offence the characteristics of 'threat' and 'repeated letter writing' were added. Both characteristics turned out not to be significantly associated with repeated letter writing and arrest for a criminal offence. A Structured Approach for Disturbing and Threatening Letters for violence, according to the analyses, were included in the RAT as 'relevant' characteristics. Although the other characteristics were not shown to be significant predictors by the analyses, they were still included in the RAT because these characteristics could contribute to revealing background characteristics for the writer, which could be relevant in particular in the case of anonymous letter writers.

RAT Procedure
For the RAT, the 14 tested characteristics that were part of the assessment framework were elaborated and expanded with indicators and literature, in order to prepare the method for practical application and also because some of the abstract characteristics, such as 'fixation' or 'powerlessness', were more difficult to assess in texts. Four new characteristics were added to the model, namely: 1. Positive attitude violence, 2. Stress, 3. Cultural background, and 4. Style (Table 3).
A 'positive attitude to violence' means that someone holds convictions that support the use of violence [28]. This attitude finds expression in texts when the writer refers, for example, to video games that feature and endorse violence [14]. Causing fear in the victim by using threats is related to psychological issues such as trauma and stress [11], and 'stress' is evinced in texts by the referencing of contextual, personal, or interpersonal characteristics that have contributed to the writer's pain and suffering [31]. The 'cultural background' of a person can become visible in a written text through the use of words like 'we' and 'us' which could refer, for example, to social ties or groups of persons with whom the writer feels connected [9]. The relevance of the characteristic 'style' has been added because consistency in writing can help identify repeat authors [10]. Style is also evinced in texts through peculiarities or writer-specific phrasing, or syntactic characteristics such as sentence structure. For each characteristic of the RAT a description is given, with additional indicators and supported by scientific literature. In brief the procedure for applying the RAT consists of the following steps: Which texts were written in the past and which other information is available on the individual, such as violent acts, previous threats, domestic/personal situation [30]. What are the individual's possible intentions and against whom was the previous threat directed? How can the text be characterised and what might be the motive? The collected information is processed using a coding sheet containing eighteen linguistic characteristics, subdivided as: Content words (9), Emotion words (5), Function words (3) and Details (1). The items are scored for: 'present' (Y) (1) or 'absent' (N). The characteristics that score for present are subsequently assessed for high, medium, or low relevance. This assessment is done on the basis of scientific results [12] as well as on the basis of experience and professional expertise (Structured Professional Judgement, SPJ [17].
The characteristics that are coded as relevant will assist in the assessment of the risk definition, such as: is it possible to make connections between the written texts that could explain the cause of this behaviour? Which interventions are available to prevent possible violence and how can a decision be made for further investment to obtain more information? What new knowledge is available on the person, which could contribute to knowing the writer's background? In brief, assessors reach their judgement based on a number of risk factors and scenarios that can be used for risk management. The aim of this method is to arrive at better predictions regarding the future behaviour of writers who make threats or who have the intention of committing violence, and to obtain more insight into the motives and background characteristics of anonymous or repeat writers [16,26,28,32,33].

Conclusion and Discussion
In the first instance, the RAT is intended for assessors such as forensic and criminal psychologists working in the security domain (law enforcement and justice departments), whose responsibilities include the assessment of the contents of threatening letters addressed to public figures. More specifically, these are threateners whose identity has not yet been established, or for whom it is not (yet) possible to arrest them as there is no criminal file yet. The results of the RAT should be considered a first assessment leading into a followup investigation. In particular in the case of anonymous and repeat writers the RAT can contribute new insights into their background characteristics and intentions. An assessment form is used to scan threatening texts for the occurrence and relevance of eighteen linguistic characteristics, with the aim of arriving at risk definitions and scenarios. In addition to the assessment of digital and handwritten threats, the method is especially intended to function as a prevention instrument for the purpose of de-escalation and directed risk management.
One limitation of the RAT is the fact that it provides a snapshot in time. Circumstances can change, and internal as well as external changes can have an effect on the possibility of the threatener sending another e-mail or letter. In other words, the writing of a threatening letter is a particular moment in time, which means that also the risk assessment of a threatening letter is a snapshot in time. This means that risk assessment is dynamic, and requires a renewed assessment once a repeated letter appears [20]. For the correct use of the RAT, knowledge about violence and competency in risk assessment and clinical and personality disorders are necessary. With regard to the required qualifications demanded of users of the RAT, psychologists working for law enforcement or justice departments (forensic or clinical psychologists) are the main user group. Training in the use of the RAT is necessary if the method is to be applied consistently and assessed as a reliable instrument. Prior to the development of this methodology, for 18 characteristics the inter-assessor reliability (Cohen's kappa 0.74) was established (enclosure 1) on the basis of a standardised questionnaire based on 278 texts. However, in order to establish the consistency of the test results, repeated measurements are necessary. In addition to test reliability, validity is also a relevant safeguard. The dataset was made randomly available by the Ministry of General Affairs and the National Police and it was delineated by selecting threatening letters addressed to public figures. As far as face validity is concerned, the instrument will also have to be compared with the CTAP-25 or the VRAW in terms of completeness and clarity. Predictive validity is relevant to establishing the accuracy of predictions that someone may resort to violence, and in this case this has only been assessed for persons who are registered in police systems as suspect individuals. There is a possibility, therefore, that someone has been registered incorrectly as suspect for violence, because there had not yet been a conviction, or because the case was dismissed for lack of evidence. External validation is necessary, therefore, if the methodology is to be used for other target groups, or to establish whether the selection of relevant characteristics can be generalised to apply to other persons making threats.
The RAT's text analysis is done by hand, and codes it in binary measurement units (present or absent). One criticism of this approach may be that it is very time-consuming to assess each letter, which may seem inefficient or out of step with the times in which processes are automated as much as possible. Contrary to automated text analysis, however, the main advantage here is that an assessor can read and understand and extract more information from the text, which could in principle result in higher validity [3]. Automatic text analysis is useful for finding characteristics that can easily be operationalised in large volumes of textual sources [24] whereas manual analysis is more suitable for complex texts [1,33]. For an exploratory analysis of large datasets both types of analysis could very well complement each other, with manual analysis used to assess the nature of a text and to find answers to questions such as 'which risks are seen as most relevant with regard to the chance that a person may use violence?'. By combining manual and automatic content analysis the pros of both methods can be used.
The introduction emphasised how social media play a facilitating role for persons who send threatening messages, and it can cause people with divergent ideas to easily find confirmation for their viewpoints [25]. For example, the sense that public figures such as politicians cannot be trusted could cause people to feel powerlessness, sadness, anger, or hatred, and eventually lead to extremist views in which the use of violence comes to be perceived as the only solution. The internet can be a breeding ground for emotions which may lead to the expression of threats. For that reason it is relevant to identify and signal these emotions at an early stage, so that 'us versus them' thinking can be pre-empted and further polarisation in society avoided. Although the RAT was developed to assess the seriousness of threats to public figures, in the end only a very small group actually follows up their words with actions. Nevertheless, each threat must be taken seriously, not only because they have an impact on the person being threatened, but also because they can cause social unrest and undermine the public debate of authorities.

Recommendations
The RAT was developed to provide support in information gathering processes, or in cases of criminal investigation by the police and other organisations in the security domain where there is no information available on the writer. In principle, the methodology is limited to the assessment of a possible risk of violence, and as such it constitutes a first step towards further investigation to arrive at a considered judgement [21]. It is important, therefore, to obtain all other available information too, so that a case may be built.
The second recommendation is aimed at organizations that are concerned with detection programs, such as the development of algorithms. The RAT is intended for manual content analysis, but the operationalisations of characteristics could also be used for the development of algorithms. Our increasingly digitised society requires a different way of thinking in order to detect deviant behavior at an early stage. Communication techniques are changing, which necessitates innovative methods for the practical and efficient assessment of large amounts of data. Perhaps detection programs will not have as many options for nuance as manual, human assessment, but characteristics like hatred-revenge and confusion could also be applied to large datasets using the operationalisations that are part of the RAT.
Recommendation three: make the RAT a central part of the training of assessors (forensic psychologists, criminal psychologists) who will be responsible in their daily work for the assessment and processing of threatening letters.
The fourth recommendation is aimed at those who have A Structured Approach for Disturbing and Threatening Letters been threatened, and it regards reporting the incident to the authorities. It is important that public figures report incidents involving threats to the police, because the number of threats to public figures is many times greater than the number of reports made to the police. Monitoring threats provides insight into how often and for how long certain threateners have been making threats.
The fifth recommendation concerns further study in order to generalise the research results for use in the decentralised (civilian) domain. The RAT is based on the PhD thesis 'Het Woord en de Daad,' which concerned a specific study of threats addressed to people in the public (central) domain, such as politicians. To generalize practical results on a broader, external group, a recommendation is to repeat this study for external validation in the decentralised domain, such as local administration [2].
The sixth recommendation is aimed at mental health care and it concerns individuals who during times of uncertainty become more vulnerable and susceptible to the conspiracy theories that are propagated via social media. These are people who have or have had mental issues and who are designated as showing 'confused behaviour'. Mental health care has an important first role to fulfil in providing support to this specific category in the form of treatment or therapy. Persons exhibiting confused behaviour will, in the first instance, be noted in the street, causing the police a lot of work. By early signalisation it is possible to react preventively and prevent escalation. Contacting a local police officer and finding a connection with health care could result in providing such individuals with the help they need to redirect their social dissatisfaction away from escalation. Recognising and acknowledging emotions will contribute to de-escalation and connect with the need felt by people to be heard. * Note: the X in the table indicates that SPSS was unable to calculate the kappa, because the variable was a constant. The variable pain (hurt) for example, was not observed in the letters by either assessor 1 or assessor 2, and both scored this aspect as 'absent'. Some characteristics were operationalised as subcharacteristics in order to improve their measurability, such as cognitive distortions, modi operandi, references to other persons, negative coping, positive coping, and anger. This explains the number of characteristics. For the characteristic of 'self-reference' ('I') no kappa was calculated, instead counting the number of self-references for each ten lines of the letter. Consequently, a kappa was only calculated for nominal or categorical variables.