The Myth Of Risk Assessment

There has been a lot written and said recently about risk assessment in child protection
circles. The focus tends to be on whether model 'A' or model 'B' is the right and best model.
The two primary competing models are actuarial (statistical) and consensus (clinical). As one
reads and listens, it is easy to conclude that the two models are separate and distinct. One is right
and the other is wrong. Thoughtful professionals should either favor the actuarial model or the
consensus model. The impression is that the two models are mutually exclusive.



A closer examination of the two models and the supporting discussions yields an unexpected
finding. The two models are not separate and distinct. Rather, they are simply different
evolutionary tributaries of the same intellectual stream.



It will help to first understand that both models consist of parts or modules. In child
protection practice, there are several steps along the path from the initial report of suspected
abuse or neglect to the termination of protective services. For example, some reports are
'screened in' and others are 'screened out.' Only those reports that are screened in go to the next
step.



For those reports that are screened in, there is a 'safety assessment' that, at a minimum,
evaluates the level of current jeopardy to which the child is exposed. If that level of jeopardy is
judged to be significant, a 'family assessment' is pursued to identify and evaluate positive and
negative factors and conditions that potentially contribute to or compensate for the identified
jeopardy. Based on the outcomes of using these assessment 'tools' or modules, a 'plan' is
developed that is intended to reduce or eliminate the immediate and longer term jeopardy to
which the child is exposed.



It is important to understand that both the actuarial and consensus models include modules
and tools to accomplish the steps in the child protection process. Only a part of that process
relates specifically to 'risk assessment.'



Both models include 'safety assessment' tools that evaluate current jeopardy to which the
child is exposed. Neither model purports to have convincing evidence that demonstrates it is
more effective in identifying current jeopardy than the other model. Rather, the controversy
focuses on the extent to which either model includes 'tools' that can be used to identify parents
(and other caretakers) who are most likely to expose children to serious jeopardy in the future. To
the extent that either model can successfully identify these parents, there are two central
benefits:




  • Limited resources and services can be concentrated on the parents who most likely have the
    highest need for the services.

  • Since child protection services are frequently involuntary, services are less likely to be
    'forced on' parents that would not expose their children to jeopardy in the future with or with out
    those services.



With the 'future jeopardy' perspective in mind, it is useful to focus on children and to
consider the jeopardy inherent in being a child.



Children At Risk:



A child has needs, problems, and vulnerabilities with which he (or she) cannot successfully
and independently cope.





  • The child cannot independently recognize and meet his multiple physical, emotional, social,
    and intellectual needs.

  • The child cannot independently identify, analyze, and solve his multiple physical, emotional,
    social, and intellectual problems.

  • The child cannot independently manage the multiple physical, emotional, social, intellectual,
    and environmental vulnerabilities to which he is exposed.



To the extent that the child cannot meet his own needs, solve his individual problems, and
deal with his multiple vulnerabilities, he is at risk.



This is the most important aspect of the myth hidden within the 'risk assessment'
controversy. Each child is, by virtue of being a child, at risk:





  • He is unable to meet his needs.

  • He is unable to solve his problems.

  • He is unable to successfully manage his multiple vulnerabilities.



Although some children are more at risk than others as a result of age, developmental
deficiencies, and such, that is not the immediate issue in child protection practice. What is
referred to as 'risk assessment' is not actually assessing risk. The 'risk' is a given. All children
are at risk. It is inherent in being a child.



The Capacity To Protect:



Rather, workers are assessing capacity. More specifically, they are assessing the capacity of
specific adults to successfully recognize, respond to, cope with, and manage the needs, problems,
and vulnerabilities of individual children. When understood from this perspective, some aspects
of the risk assessment controversy become clearer.



For example, one might postulate that an adult who has failed to 'protect' a child (meet the
child's needs, solve his problems, manage his vulnerabilities) is less likely to demonstrate the
capacity to protect the child than an adult who has a good track record of protecting the
child.



One might additionally postulate that an adult who has a demonstrated capacity to meet his
(or her) own needs, solve his own problems, and successfully manage his own vulnerabilities is
more likely to demonstrate the capacity to protect a child than an adult who has not 'protected'
himself.



Stating the same propositions in an alternative way:





  • If you have failed your child in the past, you are more likely to fail him in the future than a
    parent (or other caretaker) who has never failed his child. If you have failed your child in the
    past, your capacity to protect your child is more suspect than if you have never failed your child.

  • If you have not met your own needs, solved your own problems, or have failed to manage
    your vulnerabilities, your capacity to protect your child is more suspect than would be the case if
    you had demonstrated your ability to meet your own needs, solve your own problems, and
    successfully manage your vulnerabilities. If you have not protected yourself, your capacity to
    protect your child is suspect.



A Preliminary Capacity Model:



Using the two circumstances, (failing to protect the child and failing to protect one's self) at
least two additional dimensions can be superimposed. They are 'immediacy' and 'intentionality.'
There are likely other dimensions, e.g., volatility, but these two will suffice in the present
hypothetical context.



Immediacy: The immediacy dimension suggests that the 'failure' needs to be current
or at least in the recent past. You are currently (or recently) failing to protect your child. You are
failing or have recently failed to successfully and appropriately manage your child's needs,
problems, and vulnerabilities.



Similarly, you are currently failing or have recently failed to successfully and appropriately
protect yourself. You are not currently managing or recently did not adequately manage your
needs, problems, and vulnerabilities.



Intentionality: The intentionality dimension splits into actions and omissions. On the
one hand, you knowingly do something (action) that exacerbates or increases the extent to which
your child is not protected. You knowingly increase the extent to which your child's needs are not
met, his problems are not solved, and his vulnerabilities are not managed.



On the other hand, you knowingly permit circumstances to exist or persist (omission) that
exacerbate or contribute to your child's not being protected. You permit or fail to take action to
prevent circumstances that exacerbate or contribute to your child's needs, problems, and
vulnerabilities being inadequately or inappropriately managed.



Risk revisited:



As pointed out earlier, all children are at risk. If adults do not adequately manage the
children's needs, problems, and vulnerabilities, children experience harm, either as a product of
omission or action.



In broad terms, child protection workers characterize omission as 'neglect' and action as
'abuse.' All children, then, are at risk of abuse and neglect. If a child has a parent who neither
neglects nor abuses him, he will be adequately protected from the natural or inherent risk that is
an unavoidable function of childhood. This represents the protective capacity of his parent. That
protective capacity is the focus of child protection services.



Assessment Focus:



The assessment focus is now clearer. Focus is on:





  • Current or recent neglect or abuse.



The assessment has to then answer two questions:





  • What is the likelihood that the abuse or neglect will recur in the foreseeable future?

  • If the abuse or neglect does recur, to what extent will the child be harmed as a result?



These questions can be summarized along two dimensions: likelihood of recurrence and
extent of harm. Recurrence ranges from unlikely to likely. Extent of harm ranges from minimal
to severe.



The 'variables' can then be put into a chart. The horizontal variable is 'likelihood of
recurrence,' divided into two segments: unlikely (left) and likely (right). The vertical variable is
'extent of harm,' divided into two segments: minimal (top) and severe (bottom).



The upper left quadrant of the chart is, thus, unlikely recurrence/minimal harm. The lower
right quadrant is likely recurrence/severe harm. In this context 'Q4' refers to the lower right
quadrant: likely recurrence/severe harm.



In the earlier discussion, two 'factors' were posited as associated with which quadrant would
best characterize a particular situation.





  • The parent's history of protecting/not protecting the child.

  • The parent's history of protecting/not protecting Himself.



For the present purpose, let's assume that the assessment only needs to answer one
question.





  • Can the child stay where he is without modification of that situation?



If the answer is 'Yes,' protective services are not needed. If the answer is 'No,' protective
services are required. The process also has to assist the worker in developing an intervention
plan, if one is needed. In the present context, though, attention is limited to the services/no
services decision.



Using the model postulated here, this decision is made based on the history of abuse/neglect
of the child and the history of self-protection demonstrated by the parent.



This gives focus to the essence of the 'risk assessment' controversy. What justification is
there for basing the intervention decision on history of abuse/neglect and parent self-protection?
The only justification for using those two variables in making the decision is my having
hypothesized that they are the ones to use.



The Conceptual Stream:



Reconsider the conceptual chart developed above. It focused on the likelihood of recurrence
and the extent of harm to the child. Let's limit the discussion to those situations where recurrence
is likely and harm is severe.



A central assessment issue is this. 'Likelihood' refers to a future event or condition. The
assessment is expected to anticipate a future event or condition. Let's refer to this event as a Q4
event: an event in the quadrant reflecting a high likelihood of severe harm to the child.



So far, the discussion has suggested that a history of abuse or neglect combined with a
history of inadequate parent self-protection is predictive of the high likelihood of a Q4 event.
That circumstance does not assure that the Q4 event will occur but it is likely. That likelihood
then becomes the justification for protective services intervention.



This, then, represents the core of the conceptual stream from which 'risk assessment' flows.
Factors such as prior abuse/neglect and a history of inadequate self-protection are associated with
a high likelihood of future Q4 events.



There is minimal disagreement about the validity of the conceptual stream. There are
identifiable 'factors' that are highly associated with Q4 events and are less strongly associated
with events in the other three quadrants of the chart. The stream is, itself, real and intellectually
exploitable. It is possible to identify factors that enable workers to anticipate which parents are
most likely to re-abuse/neglect their children and enable workers to anticipate the likely
harmfulness of that recurrence. Based on the confidence level associated with that prediction, the
worker may ask the court to 'order' the parent to participate in the protective services
process.



It is interesting to note that 'risk assessment' could also be pursued from an alternative
perspective. The current conceptual stream flows from focus on factors predictive of Q4 events.
'Which parents are most likely to re-abuse/neglect their children?' It would be equally
reasonable to pursue factors predictive of Q1 events: unlikely recurrence/minimal harm. 'Which
parents are least likely to re-abuse/neglect their children?' Attention returns to this point
later.



Diverging Tributaries:



The conceptual tributaries diverge along at least three vectors:





  • How does one identify factors that are potentially predictive of a Q4 event?

  • How does one determine the predictive validity of those factors?

  • How does one reliably determine the presence or absence of the factors?



The requirement is to adequately anticipate the capacity of the parent to protect the child. If
the parent protects the child, the child does not experience harm. If the parent fails to protect the
child, the child experiences harm. The requirement is not 'risk assessment.' Rather, it is
'capacity assessment.'



The responsibility of child protection services is to anticipate and then prevent Q4 events.
The focal question can thus be re-formulated:





  • How can the worker determine the likelihood of a parent's participating in a future Q4 event
    with his child?



The Consensus (clinical) Tributary:



The consensus tributary uses a well-known approach. Somewhat over-simplified, the
approach is as follows.



In order to identify the factors to be used to anticipate Q4 events, a combination of
experience and expertise is used. This includes the related literature and available research that
are combined with the experience, clinical judgments, and insights of people with experience and
expertise in child protection services. Combining these sources yields a 'body of knowledge'
about what factors are associated with Q4 events.



In the December 2003 Child Welfare Institute (CWI) newsletter, Thomas D. Morton and
Barry Salovitz comment as follows, 'Clinical risk assessment approaches employ the use of
indexes (often a list of clinical risk variables derived from etiological research on maltreatment)
to identify the specific focus of intervention and the case plan. Clinical models partially base
their validity on etiological research that has identified family, caregiver and environmental
dynamics associated with the occurrence of maltreatment in general.'



This body of knowledge is then subjected to analysis that leads to a matrix or model for use
in predicting Q4 events. The model consists of clusters of factors that are thought to be predictive
of Q4 events. In practice, the 'elements' in the model are included in forms that trained workers
use as guides for gathering relevant information about the parent.



The information gathered may be of a yes/no type or rating type, e.g., high-low, many/few,
more severe/less severe.



The validity of the elements is supported by the 'consensus' of the sources, including
correlations between specific factors and child abuse/neglect.



The decision to terminate or continue protective services is based on the information
gathered using the model and on the judgment of the worker who gathers the information.



The primary issues with the approach are first related to validity. Does the model anticipate
likely Q4 events and distinguish between parents who are likely to and parents who are not likely
to participate in future Q4 events with their children? Proponents of the approach argue that the
model has validity based on available research related to specific factors in the model and on the
collective experience and expertise inherent in the sources.



Proponents of the consensus model also point out that the model is intended to assess current
safety and to inform the development of intervention and services strategies to reduce or
eliminate the current jeopardy to which children are exposed. The model argues that reducing or
eliminating that jeopardy also reduces or eliminates future jeopardy for children.



The higher the immediate jeopardy, the higher the need for protective services. Thus, if
services need to be rationed, the situations where there is higher current jeopardy should receive
priority attention.



The second issue is the reliability of the information gathered. To what extent is the
information independent of the specific person gathering the information? Since the forms
require judgment-based 'ratings,' the reliability of the process is suspect. To the extent that
multiple workers would not gather the same information, come to the same conclusions, and
make the same decisions, the process itself is suspect. It may or may not be a predictive
improvement over relying exclusively on the judgment of the individual workers in specific
situations.



Based on the reading and discussions pursued by this author, the validity and reliability of the
consensus model remain open. Nonetheless, the model does have high face validity and is
consistent with contemporary practice.



The Actuarial (statistical) Model:



The actuarial model uses an equally well-known approach to develop its 'risk assessment'
module. (Note that 'risk assessment' is a specific module in the model and that there is not an
equivalent module in the consensus model.) Similarly over-simplified, the risk assessment
module is as follows.



The objective is to identify 'factors' that can be used to anticipate Q4 events. Once
identified, those factors are to be operationalized and included in a form that workers use to
gather information about a parent. Based on the information gathered, the worker will decide
whether child protection services will be terminated or continued.



The requirement is to identify discrete factors that are either present or not present. The
developers of the model contend that they have done this.



For example, a report in Juvenile Justice Bulletin -- July 2001 discussing the work of the
Children's Research Center (CRC) and its Structured Decision Making (SDM) model explains,
'The heart of the SDM model is its research-based risk assessment tool. Although other
components of the model are based on a general consensus (often informed by available
research) of what constitutes best practice, SDM risk assessment tools are based on the outcomes
of actual cases. To develop a risk assessment tool, CRC and agency staff jointly identify a list of
potential risk factors. These potential factors are based on literature, experience, and previous
CRC research results. CRC researchers then review a large sample of case records (e.g., 1,000)
and code them for the presence or absence of the factors, based on what was known about each
family at the time of the sampled investigation. Case records are further explored to identify
families that experienced reinvolvement with the agency after the sampled investigation. The
definition of 'reinvolvement' generally includes subsequent CPS [Child Protective Services]
referrals, subsequent substantiations of maltreatment, subsequent child injuries, and subsequent
CPS placements. The research process then examines the statistical relationship between case
characteristics and case outcomes to identify the variables that are most closely associated with
risk. The set of risk factors that most effectively divides families into three or four different risk
groups constitutes the risk assessment tool.'



The model then consists of a set of discrete factors that the worker verifies as present or not
present. The resulting data is then 'scored.' The score the parent receives is then used as the
basis of the service/no service decision. Parents with higher scores are more likely to
re-abuse/neglect their children than parents with lower scores.



It is important to note that the worker can make a decision contrary to the 'score' if, in her
(or his) judgment, there are factors that justify not accepting the score in a specific situation.
Also, some elements require worker judgment and are not simply scored as present/not present,
yes/no.



The proponents of the actuarial model report that the model has high inter-rater reliability
and demonstrated validity. The validity is based on studies that show that the model has 'higher'
association with Q4 events than the consensus model. Specifically, they argue that the 'risk
assessment' portion of the model is a better predictor of which parents are highly likely to
re-abuse/neglect their children. That knowledge then enables agencies to focus limited resources
and services on those families who most clearly need protective services.



Conclusion:



First, neither the consensus model nor the actuarial model measures or assesses risk. Rather,
they both focus on the capacity of parents to protect their children. In the absence of parent
protection, both models agree that children will experience harm.



More specifically, the proponents of both models assert that 'their' model is able to identify
parents likely to fail in protecting their children. If this is a valid assertion, both models should be
predictive in relation to parents who will protect their children. The absence of the model's
factors for a parent should be predictive of parent success. For example, using the two factors
developed here yields:





  • A parent who has no history of abusing/neglecting his child will likely protect his child in the
    future.

  • A parent who has a history of self-protection will likely protect his child in the future.



Not providing services should be an affirmative decision instead of the absence of a decision
to provide services. Unfortunately, neither the consensus nor the actuarial model is nearly strong
enough to support such an affirmative decision. To the extent this is true at this point in the
evolving process, neither model should be used as the exclusive basis for child protection
decisions.



To pursue the point, a reliable and valid 'risk assessment' model would predict parent
success. This is not an idle point since determining if a parent will be successful is a necessary
part of child protection practice. It is a critical element in selecting foster parents, adoptive
parents, and when approving relative placements. A model that has predictive value in relation to
parent failure should be expected to have equal predictive value when the need is to predict
parent success. It should tell workers which parents have a high capacity to protect children.



      Both models achieve consistent, positive
outcomes for the parents for whom services are provided. If services are provided using either
model, reductions in new reports for those parents are noted. The level of improvement appears
to be similar for both models.



The focal issue is, then, whether the factors can and do predict Q4 events. The actuarial
proponents have studies that demonstrate the predictive power of the model. The proponents of
the consensus model appear not to have such supporting data but do have the benefit of face
validity. Based on experience, the model 'seems' right. They also reference studies that suggest
that the predictive strength of the actuarial model may not be as definite as other studies have
suggested.



Which model should be used? The actuarial (statistical) model and particularly the 'risk
assessment' module within that model, is preferable, if it can be used by workers to predict Q4
events with enough reliability to depend on it as a valid alternative to worker judgment and the
consensus model.



Of course, what is called the consensus model can be just as strong if the 'data' is subjected
to rigorous analysis and if controlled studies are conducted to determine its predictive
validity.



The most important questions are not complicated.





  • Which model has the most predictive power in relation to future Q4 events, based on careful
    and thoughtful research?

  • Which model is most useful as a set of tools for workers throughout the protective services
    process?



At this point, the actuarial model appears to make the stronger case in relation to prediction
of future Q4 events, based on the research that has been done. Even so, it would be ill advised to
rely on that process to the exclusion of worker judgment and other processes, including the
consensus (clinical) approach. It is a very fast-moving environment and committing to one
approach to the exclusion of others would be premature. The immediate need is for more and
better research and continuation of the thoughtful debate.



Which model is most useful as a set of tools for workers throughout the protective services
process is not at all clear. Both models are very strong when evaluated as worker tools. It seems
likely that choosing one model over the other could reasonably be based on worker preference
and other pragmatic factors. It may be that some convergence of the two models would be as
appropriate as reliance on either to the exclusion of the other.



As the discussion continues, the central requirement is that decision-makers are not
distracted by the controversy or by the intensity of the debate. Rather, it is important to clearly
define the purpose of adopting any model. It should be done to facilitate and support the
activities of workers as they go about the business of reducing and eliminating jeopardy for
children and increasing the capacity of parents to successfully protect their children. The value of
any model should only be judged in terms of this complex set of tasks. Focusing only on one
aspect of that process, viz. 'risk,' would be very ill advised and far less than our children and
their parents deserve and certainly must have.



Suggested readings available on request. To download a PDF version of the article, Click Here.


By Gary A. Crow, Ph.D. May 4, 2017