Skip to main content
 

Data quality: challenges when collecting data to inform your mental health strategy

Data quality: challenges when collecting data to inform your mental health strategy
Click to enlarge

Hannah Winter, content and insights manager at the CMHA, explores the areas to be aware of and cautious of when collecting and assessing the quality of mental health data. 

Data, when analysed, interpreted & presented in meaningful ways can help inform your workplace mental health strategy. However, t here are several factors to be aware of and cautious of when collecting and monitoring data:

Not all data tells a clear story:

  • Due to stigma some employees may be reluctant to disclose if they take sick leave for mental health related reasons. Instead, they might declare a physical illness, use their annual leave or even work remotely. Organisations can seek to understand (for example, via staff surveys or focus groups) how comfortable employees would feel in disclosing a mental health-related sickness absence and monitor trends over time. Bringing different data sources together in this way provides a richer understanding.
  • An increase in reported mental ill health disclosure or absences may not necessarily be due to increased incidence. Rather, it may be linked to improved reporting as a result of greater awareness, reduced stigma or improved data collection methods.

Low utilisation/take up does not necessarily mean an initiative is ineffective:

  • Low uptake of an intervention could suggest that the intervention is unpopular and not providing a return on investment. However, it could be that uptake is low due to, for example, a lack of awareness (therefore more could be done around communications); a lack of understanding about the value of the initiative to particular individuals (again enhanced communications could help); reticence to be engaged due to a fear of stigma (and therefore there may be cultural and perception issues that need to be addressed).
  • Reviewing one data set in isolation may not provide enough information and it is worth considering other supplementary information that is accessible that could provide further/deeper insight.

Surveys often do not account for nuance: 

  • Many data sources group mental health conditions into one category and do not distinguish between, for example, stress, anxiety, depression, or post-traumatic stress disorder. Therefore, data may not be specific enough to effectively inform decision making as different challenges will require different interventions. Quantitative data may need to be complemented with other data such as that from qualitative sources, such as focus groups, to provide additional context.
  • Acquisition of disaggregated data may be vital in identifying patterns and underlying trends for different groups, e.g. gender, age, race, which may become crucial in informing interventions, as well as planning and tailoring mental health support that meets the needs of different groups.

Consistency over time:

  • It is important to collect data over time and consistently. This allows trends and patterns to be observed. Data collected once is only useful as a one-off snapshot.

Disclosure rates:

  • Low disclosure rates may indicate gaps in data. Individuals may be choosing not to disclose because of a lack of trust in how their data will be used, stigma or inefficient HR systems or processes that are not accessible. Be clear about how the data will be used and whether it will be anonymised.
  • Consider who you are asking to collect the data. If a line manager is responsible for logging a reason for sickness absence, an employee may not wish to disclose to their line manager the reason for their absence if related to mental health.
  • Even with high disclosure rates, there still may be groups you are missing data from. Check this disclosure rate across the different employee demographic groups within your business.

Mental health awareness: 

  • Employees may not have full awareness of a mental health condition and therefore might not report absences due to mental ill-health because of a lack of understanding.

To further address these challenges the CMHA recommends:

  1. Bringing different data sources together where feasible so that you can start to build a richer picture about what really is being observed.
  2. Always providing a narrative and context around the data reported. For example, ‘this is the first year this data has been assessed and hence is indicative and will be improved over time’ or “this data was collected in the midst of the second lockdown”. Clarity of what the data is, is vital to being able to understand the confidence one can place in the data and how to interpret it.
  3. Consider the functionality of your HR systems and the user experience of colleagues that are inputting data. Make collecting data easy and straightforward. Many organisations find they obtain richer data from sources that aim to listen to colleagues e.g. pulse surveys, focus groups, culture surveys where people are asked for their opinions, rather than from sources where employees are asked to log data for HR processes e.g. reason for sickness absence.
  4. Engage in peer support. Speak to other organisations that are already doing this well to benefit from shared learning. For example, within the CMHA member network.

The CMHA has recently released a new member resource ‘How to Use Data to Inform Your Mental Health Strategy’. The guide was created in collaboration with Frontier Economics and includes case studies from Unmind and CMHA members PwC and Standard Chartered. 

Resources

Members can access the full guide here by logging in*.

Non-members can find out more about membership here.

*Contact cmha@citymha.org.uk if you have forgotten your username or password.