Analysis of free text comments in the 2023 Women’s Reproductive Health Survey for England and production of a complete anonymised dataset

• Health care
• Health improvement

At the end of the Reproductive Health Survey for England 2023 (RSE2023), respondents were invited to leave free text comments: “If you have anything else that you would like to share, please leave comments in the box below, for example, about your reproductive health experiences or suggestions on how reproductive health care can be improved”. A benefit of comments sections at the end of surveys is that they allow respondents to elaborate on responses they have provided to closed questions and to add further information that had not been captured in the questionnaire.

Analysis of respondent comments in this survey will provide a richer understanding of the reproductive health views and experiences in the 2023 RHSE dataset, compared to the quantitative dataset on its own. Production of a complete anonymised dataset, i.e., the current quantitative dataset plus the anonymised qualitative data, will permit triangulation of data on specific topics.

• What is the context?

  The 2023 Women’s Reproductive Health Survey for England (RHSE) and was commissioned by the Department of Health and social Care (DHSC) to monitor reproductive health outcomes and access to and experiences of health services and other forms of support. It was a commitment of the 2022 Women’s Health Strategy and was run by a team based at LSHTM.

  The survey covered four broad themes over the reproductive health life-course: menstrual health, planning and preventing pregnancy, pregnancy outcomes and reproductive ill-health.

  At the end of the survey, respondents were invited to leave free text comments if there was anything further they would like to share on the topic. The survey itself had 59,332 respondents, with around 20,300 of those leaving a comment.

  The tagging of the qualitative comments in the 2023 survey enables a qualitative analysis of the comments, which will produce a deeper understanding of respondents’ views and experiences of their reproductive health in their own words. It can elaborate on the quantitative data, indicating what is important to women in relation to their health and how services have been provided.

• What did we want to know?

  The project aim was to produce an anonymized data set of qualitative (free text) comments from the 2023 RHSE, tagged by reproductive health topics and concepts, linked to participant survey responses, including socio-demographic data.

  This work had two main objectives:

  • To report differences in socio-demographic characteristics between respondents who left comments relating to their reproductive health with those who did not

  • To tag the comments by reproductive health topics and concepts and to produce a coding dictionary

• What did we do?

  Personal data (such as name or email address) were removed from responses in the comments section in the survey. Any comments irrelevant to the analysis were removed, such as ‘thank yous’ or ‘N/A’

  Statistical comparisons of those who left at least one reproductive health-related comment vs none was made by socio-demographic characteristics (age, ethnicity, gender identity, educational level, area of residence) and health status.

  We used the text mining software (WordStat) to code the free text comments.

  • Initial data preparation in this software included review and correction of spelling errors in the free text comments, review and updating of the word exclusion list to retain only the most relevant and informative words for the analysis.
  • Based on the common keywords and phrases extracted from the comments, and a priori knowledge of topic covered in the survey, we developed a categorisation dictionary.
  • The categorisations were applied to the comments and extracted linked to each individual participant identifier and merged back on to the full quantitative dataset.

• What were the key findings?

  20,064 free text comments remained for further analysis once irrelavent quotes were removed (equating to 33.8% of full RHSE sample leaving a comment suitable for further analysis)

  Respondents who most commonly left comments were those aged 30-34, those who were of Mixed or Multiple ethnicities, those living in London, those experiencing difficulties managing financially, those with a degree and those in poor health.

  Respondents who less commonly left comments were those aged 16-19, those of Black, Black British, Caribbean or African ethnicity and those living in the West Midlands.

  50 unique tags were created. Topics most commonly included in comments related to healthcare access and experiences, contraception and menstrual bleeding

• Outputs / Events

  The fully tagged dataset and the categorisation dictionary were shared with DHSC and are available on request to DHSC. The dataset has been shared with the PRU in Reproductive Health to undertake analysis of comments in their study looking at contraceptive use.