CAFCASS and Woman’s Aid – Let’s look at the numbers…

CAFCASS recently conducted a research programme with Woman’s Aid. We cover the background to the commissioning of this research in a recent post available to read here. In the course of the research, the research team was granted access to 216 CAFCASS case files. With the first paragraph in the resulting press release stating that

Research released today by Women’s Aid and Cafcass finds that 62% of applications to the family court about where a child should live or spend time feature allegations of domestic abuse.

CAFCASS press release, July 2017

Looking at CAFCASS’s own private law demand statistics we can see that in twelve months between March 2016 and April 2017 CAFCASS was involved in 40,585 private law cases. So, the 216 cases considered in this study represents 0.53% of CAFCASS’s total annual case load.

A quick word on Sample Size, Margin of Error and Confidence Level

When conducting any broad based research that seeks to extrapolate its findings amongst a much larger population it is important that the sample taken is sufficiently large to give confidence any conclusions drawn can be taken to represent the population.

Determining the appropriate sample size is a function of 3 things:

1. Total Population size (in this case we can take the total number of CAFCASS cases in a given year or 40,585 in the last year).
2. Confidence Level – you can think of this as the probability (expressed as a decimal) that the sample reflects the population (within its margin of error). Common standards are between 0.95 and 0.99 (95% to 99%).
3. Margin of error – How far a sample will deviate from the confidence level (a measure of reliability of the result).

For those who are interested, sample size can then determined using the following equation:

Sample Size   =

We don’t need to go into the maths in great depth but if we take the population of 40,585 cases, use a confidence level of 0.95 and a margin of error of 2% then the sample size needed to give a statistically significant (relevant) result is 2,267. If we use the number of cases in the CAFCASS “base population” cited in the research whilst keeping confidence level and margin of error consistent then we get a sample size of 2,073.

Using that smaller population of 15,160 (as that is what the research cites), keeping the confidence level at 95% we would have to increase the margin of error to around 7% to approach the sample size used in the report. Alternatively, using a 2% margin of error you would have to move the confidence level to such a degree that the results would be meaningless.

Readers should note that this is not the first time we have seen surveys completed by CAFCASS using very small sample sizes. In a survey called “What Happened Next” in March 2016 CAFCASS says:

This was a small-scale study which was limited in its scope. However, overall the findings are encouraging, particularly so in light of the considerable pressures upon the courts (a rise in the number of litigants in person and professionals working in the family court being more hard-pressed being two notable factors)

The “What Happened Next” survey included responses from only 143 people. You can read the full report here.

Readers should note that CAFCASS does not routinely do any follow up work to ascertain the effectiveness of CAFCASS recommendations (or indeed if they are followed by the courts). CAFCASS therefore have an extremely limited data set on which to base any changes to their process to improve child outcomes.

So back to the study

Firstly, looking at the press release vs what the report actually said provides an interesting comparison. The CAFCASS press release (link above) says:

“62% of applications to the family court….feature allegations of domestic abuse”

The report itself actually concludes that:

“The main finding was that domestic abuse was alleged in almost two-thirds of cases (62%)…”

Of course, there is no way that CAFCASS can draw the conclusion  stated in the press release (with any degree of confidence), based on a sample size of 216 cases. It may be correct when you look at the 216 cases in isolation however that 62% featured an allegation. To get an accurate representation that you can rely on to provide a true indicator of incidence in a population requires a much larger sample size.

So, what did CAFCASS and Woman’s Aid actually look at?

Allegations vs Evidence

On the subject of allegations the report says this:

The study did not set out to record ‘evidence’ of domestic abuse, though in some cases there will have been civil or criminal court findings of abuse

A footnote also says

The data did not record if the finding of fact hearing took place, or what (if any) findings were made (Editor Note: a finding of fact hearing is a separate hearing to establish the validity of an allegation which needs to be proven to the civil burden of proof).

Data released post the report

A member of the public enquired into the distribution of the 216 case sample by CAFCASS service area.

CAFCASS responded with the following table:

The requestor asked:

Please provide the total number of cases in the sample where a) criminal findings were made b) civil findings were made (to support the allegations)

CAFCASS responded.

In order to provide a response, each case file from the study would need to be checked individually; therefore the cost of compliance would exceed the appropriate limit which for Cafcass is £450….A response to this request is therefore exempt under Section 12 of the Freedom of Information Act.

The requestor then asked

Appendix A in the research shows the various “types” of “evidence” used to support the assertion that domestic abuse occurred. Can you please confirm the number of cases within the 216 case sample where the only “evidence” was an allegation made by one of the parties. Please also break this figure down by male/female as you have done with appendix A.

CAFCASS responded with the following:

There were 40 cases where evidence was based only on an allegation recorded by one/both of the parties. This is broken down by gender as follows:

• 28 cases female alleged domestic abuse

• 10 cases both male and female alleged domestic abuse

• 2 cases male alleged domestic abuse

You will note from the data provided above that CAFCASS does not report on cases by service area in their private law demand statistics making a side by side comparison very difficult in terms of the proportion of cases from each service area used in the sample relative to real life case demand.

We do however have the detail for the number of Family Court Advisors per service area (which should in theory bear some resemblance to case demand.

Accuracy of research published by CAFCASS

Freedom of information requests looking at previous CAFCASS research and the corresponding responses by CAFCASS raise serious issues with the way that data is analysed, interpreted and reported. We have written about these in a previous article Independent Research into CAFCASS outcomes.

Please feel free to us know what you think of the research published by CAFCASS and Woman’s Aid in the comments below.

Voice of the Child is also on Facebook and Twitter.

Voice of the Child

In a Family Law system designed for combative parents there is no real allowance for the views of children and any understanding of how Family Law ultimately impacts on children most of all.

We speak for the children in Family Law so that, finally, the children have a voice.