Improving the impact of menstrual health innovations in low- and middle-income countries: a theory of change and measurement framework

Over the last decade, growing attention has focused on the challenges experienced by adolescent girls and women in relation to managing their menstruation with dignity and comfort in low- and middle-income countries (LMIC). ^1–5 In numerous contexts, girls lack adequate guidance about menstruation, thereby experiencing their first menstrual blood flow with shame, fear, and insufficient understanding around how to manage their menstruation with confidence and dignity. ^6–8 Compounding this challenge is the inadequacy of safe, clean, and easily accessible water and sanitation facilities with disposal options for adolescent girls and women to manage their menstruation while at school, at work, ^9,10 in transit ¹¹ and in displacement contexts. ^12–14

Many girls and women report having insufficient supplies for managing their blood flow, and having to use tissues, toilet paper, rags or underwear alone. Lack of appropriate supplies makes walking long distances, sitting in the classroom, visiting the market or waiting in long distribution lines in displacement camps difficult to manage. ^7,11,12,15 Across many LMIC, girls and women are already disadvantaged, facing numerous hurdles related to gender equity in education and employment, ^16,17 and gendered norms within families and communities that may diminish their opportunities, such as early marriage ^18,19 or pregnancy-related school dropout. ^20–22 Widespread gender discrimination contributes to the barriers faced by adolescent girls and women in managing menstruation with dignity and comfort, and negatively impacts health and wellbeing, hinders ability to succeed, ^10,23,24 and potentially contributes to sexual and reproductive health harms. ²⁵

In response to the increasing awareness of the menstrual hygiene management (MHM) barriers facing adolescent girls and women in many low resource contexts, there has been growing attention to defining the issue and identifying potential solutions. In 2012, the Joint Monitoring Programme (JMP) led by UNICEF and the World Health Organization (WHO) created a definition of MHM (see Box 1), both for the purposes of lobbying for inclusion of MHM within the Sustainable Development Goals (SDGs) and to enable the development of indicators for monitoring the MHM challenges and identified solutions within and across countries. ²⁶ Although MHM was not specifically incorporated into the SDGs, there was mention in Goal 6 of the “unique needs of girls and women” in relation to sanitation. There are currently numerous efforts underway to identify and test solutions to address these MHM needs, with the majority targeting adolescent girls. ^25,27–31

Other efforts to address MHM include numerous non-governmental organizations (NGOs) distributing sanitary product supplies and information to girls, ^32–36 with more holistic approaches seeking to engage communities or teachers, or to improve the toilet facilities at schools. ³⁷ A small number of governments have implemented MHM or menstrual health policies, focusing primarily on the distribution of subsidized products to girls in school. ³⁸ Such policies frequently articulate the need for gender segregated toilets and the provision of MHM information, with a few including broader policies aimed at meeting the MHM needs of all women of reproductive age. ³⁹ Reinforcing the importance of going beyond stand-alone MHM policies, a recently conducted systematic review of education policies in 21 LMIC identified a significant gap in attention within Education Sector Plans (which have budgetary implications) to require gender-segregated toilets and the provision of puberty education (including MHM) within education systems. ⁴⁰ Moreover, adolescent girls’ and women’s MHM needs extend beyond schools, and the global development community continues to lag behind in addressing their menstrual health and management related needs at home, in the community, and in the workplace, with limited funding and prioritization given to identify effective and efficient solutions in these contexts. ^9,41

Menstrual health innovations frequently include the distribution of sexual and reproductive health guidance (including menstrual management) in tandem with product delivery. The underlying aim for many social entrepreneurs has been to improve the accessibility, affordability, and quality of menstrual hygiene products, including diversifying product choice and distribution mechanisms, in addition to the provision of anatomical, reproductive health, or MHM information to girls and women buying or receiving products. ^30,33,34 A secondary aim is to strengthen economic opportunities for local women through employment in the production and/or distribution of products, procurement of resources from women farmers or suppliers, and/or creation of micro-entrepreneurship opportunities for women to run independent small businesses that produce and distribute such products. ^35,42,43

Product accessibility and training on the safe and correct usage of a menstrual product are essential components for improved MHM. Although minimal large-scale data exists on the types of products adolescent girls and women are using in different country contexts (and across varying demographics and geographies within countries), qualitative data from numerous LMIC indicate a wider use of cloth than disposable sanitary pads, despite many girls and women indicating preferences for disposable pads. ^12,44–47 In some impoverished settings, adolescent girls have reported that they lack access to underwear, hindering their ability to manage monthly menstrual flow given that there is nothing on which to attach a cloth or pad. ^48,49 More recent surveys conducted by PMA2020 in India, Uganda, Kenya and Nigeria indicate only a small proportion of women have everything they need to manage their menstruation. ^50–54 However, a recent analysis of the datasets suggests that the surveys, while very rigorous, are based on samples that may be biased towards certain populations (eg, more educated, urban) and thus may not be capturing the practices of the most vulnerable girls and women. ⁵⁵

To address the gaps in affordable and accessible products, social entrepreneurs are developing a broader array of new products, such as underwear into which cotton wool or other absorbent material can be placed, reusable sanitary pads, and disposable sanitary products made out of local plant materials or other environmentally friendly materials or potentially biodegradable products. ^35,36 These entrepreneurs operate within a system often characterized by limited government guidelines on quality assurance, product standards or established monitoring and evaluation strategies. In the past, concerns were raised that some materials produced did not reach the quality standards required for comfortable management of menses. These issues need to be further evaluated and monitored to ensure this is no longer the case and that social entrepreneurs follow the established monitoring and evaluation strategies. Some studies have raised concerns that the materials produced may not reach the standards required by the target population or government entities. ⁵⁶ In addition, some recipients of new pad innovations have complained that the quality of the pads is too poor to support their menstrual needs, ⁵⁷ an issue to be further evaluated.

Another key component of improved MHM is information accessibility, and the provision of guidance and support to adolescent girls in particular about the changes happening in their bodies during puberty, the onset of menstruation (including what to expect), and how to confidently and hygienically manage their monthly menstrual blood flow. A number of efforts, primarily by NGOs but some by national governments, are underway to provide improved MHM information to girls, such as puberty books developed by various organizations. ^30,58,59 Although only small-scale evaluations of such books have been conducted, evidence is accruing that shows they improve girls’ levels of knowledge and attitudes around menstruation. ⁵⁸ However, information provision has been noted to have limited impact on facilitating MHM practices when water and toilet infrastructure is inadequate, for example, in schools. ^6,57,60 Pilots were conducted in Kenya and Uganda which sought to evaluate select components of MHM, generating some rigorous outcome measures. ^25,27,28,43 However, for aid and donor organizations that are required to justify investments with quantitative measures included in their log-frames, which often have the ultimate outcome measuring lives-improved, further technical support is needed to ensure that MHM innovators are able to document meaningful outcomes.

While making an important contribution to addressing the MHM needs of adolescent girls and women in LMIC, such global, national, and local level efforts would be strengthened if embedded into a structured model such as a Theory of Change (TOC) for menstrual health innovations and information. This would help to ensure that financial stakeholders are able to monitor and ensure the impact of their investments, and that social entrepreneurs, innovators, and even NGOs, are adopting the most effective approaches for impacting the lives of girls and women across LMIC. ⁶¹ To address this gap, an effort was undertaken in 2017 to develop a new TOC schematic with an aligned monitoring and evaluation framework to: 1) measure the intended outcomes related to the scale-up of menstrual innovations across countries, and 2) enable a more rigorous assessment process of various delivery models. This would yield the ability to determine which strategies are most effective at improving the lives of girls and women, which in turn could guide further investment in such approaches. The proposed TOC would include menstrual health innovations involved in the production and distribution (eg, rural, peri-urban, urban areas) of a variety of menstrual products (eg, disposable pads, reusable pads, menstrual cups) across various channels (eg, retail, direct distribution, through schools and/or NGOs). It would also encompass the delivery of information education and communication (IEC) materials and/or awareness campaigns (eg, MHM, sexual and reproductive health and rights, taboos and social norms on menstruation). An accompanying narrative document provides a more detailed explanation of the TOC schematic, including the importance of local adaptation as needed.

A parallel effort to develop a monitoring and evaluation (M&E) framework was intended to describe and quantifiably measure any changes resulting from the development, introduction, and use of the MHM interventions utilizing the TOC framework. This was done with a focus on identifying existing standard MHM indicators and structuring their correct and timely measurement in target populations. The framework links “theory” with “practice” and provides a means of capturing evidence of progress made towards achieving an innovator’s desired aims and goals over time. This contributes to the recommendations generated at the MHM in Ten meeting in 2014 for more rigorous measurement of standard indicators, to quantify the effects of MHM and subsequent impact of interventions. ³ This also supports the building of an evidence base on effective MHM approaches for entrepreneurs, policy makers, and funders/investors. ²⁵

As articulated in the definition of MHM (see Box 1), the role of menstrual products and related MHM education information are only two components within a more holistic definition of what is needed to assure adolescent girls and women are able to manage their menstruation in dignity and comfort. ^3,5,11 Also of critical importance is the availability of adequate numbers of safe, clean, private water and sanitation facilities, bathing and laundering spaces for girls and women to clean themselves and wash and dry used menstrual materials, underwear and clothes, as well as effective disposal options (eg, waste bins, chutes) and waste management systems for the hygienic handling of menstrual waste. ^62,63 Although essential for MHM, these aspects were not articulated in-depth in the TOC and M&E framework described here, given the specific focus of the effort on early-stage menstrual product innovations and social enterprises (product and/or education) in LMIC.

The Theory of Change

The purpose of the TOC (Figure 1) is to illustrate how activities undertaken by a given MHM innovation or social enterprise can contribute to the intended or observed outcomes and impact(s). A narrative (Appendix S1 in the Online Supplementary Document) was designed to be read in conjunction with the TOC and the Indicators Matrix (described below). The TOC recognizes that MHM innovations require the following key components to be successful and impactful:

Figure 1.Theory of Change: Path to Scale: Menstrual Health.

• Enabling Context: As visualized in the diagram (AppendixS1 in the Online Supplementary Document), the ‘enabling context’ underlies the entire TOC, and recognizes that a successful MHM innovation is largely dependent on:

  • its contextualization within local MHM practices and beliefs;

  • its ability to empower girls and women through access and education, and to some extent through employment;

  • by ensuring its accessibility to girls and women in the long term through maintaining government regulatory standards over time; and

  • use of an equity approach to improve all lives, including those of low-income and marginalized girls and women.

• Key Foundational Elements: As visualized, the “foundational elements” entail the necessary foundation from which innovations build outward to succeed, including:

  • Input and buy-in from women and girls as consumers, users, producers, suppliers, and key decision-makers.

  • Engagement and buy-in with other key stakeholders such as relevant policy makers, government ministries, and community leaders.

  • Development of an educational training plan (including for staff), educational materials and/or instructions that address product use, MHM, and prevalent menstrual beliefs and taboos.

  • Obtains all relevant licensing prior to production, assuring that all required equipment is procured, government licensing obtained and production hubs are established; mobilizes a workforce, including for production, human resources, sales and marketing; prepares a refined product model; executes effectively on marketing, sales and distribution plans; and secures sufficient investment to initiate and sustain these activities.

  • Consideration of environmental impacts, including menstrual waste disposal (and management).

• Scale up Activities: These generally include social enterprises’ business and other operational activities required to generate earned revenue and reach their end consumer / user / beneficiary and consist of: research and development, leadership and human resources, procurement and supply chain, production (the capacity to manufacture products of consistently high quality, efficiently and cost-effectively), product packaging and branding, sales and distribution (the ability to deliver product effectively through identified channels and assure consistent and appropriate delivery to users), marketing and communications, customer service, and more.

As detailed in the TOC, this applies to products (including their production and distribution), and education (including adaptation and production of materials, and dissemination).

• Outcomes: These include both immediate outcomes (eg, sustained access to quality and affordable menstrual product/education, improved knowledge, attitudes and behaviors on MHM), and intermediate outcomes (eg, increased end-user demand for menstrual products in target population, increased women’s participation in employment and livelihood opportunities and increased girls’ participation and attendance in school).

• Impact: This section describes the broader impact that improved MHM can have on the lives of girls and women, including enabling them to engage more in their school and work opportunities, and thus promoting improved educational engagement, economic and livelihood security, population health, engagement and reduced social stigma, and increased dignity of girls and women.

M&E Indicators and Framework

Improved Indicators

The development of improved indicators for monitoring MHM innovation and information dissemination were intended to have multiple uses (Appendix S2 in the Online Supplementary Document). Although a primary aim is for the indicators to inform those who are scaling up MHM innovations and social enterprises, the indicators were intentionally developed to have broader applicability to NGO and other institutions implementing MHM programming. Although some users may opt for the most rigorous recommended indicators, other organizations may prefer (or have more capacity to collect) the indicators that require fewer inputs to implement and rigorously measure. All indicators are designed to effectively capture the impact of various MHM approaches being implemented in a range of contexts in LMIC today. Adaptation to local contexts are expected.

In addition to building a robust MHM evidence base that allows MHM innovators to evaluate innovation-related change, indicators will also be of value for cross-study comparisons. For this, standardized indicators are required. To date, there has been variation in the definition, precision, and reliability of indicators used in association with MHM, with some neglected or not yet measured. ^3,25 Although universal agreement on appropriate measures for MHM is pending a future convening and review by a panel of experts, the following documents critical measures that are generally considered to be of value across programs, and have been defined in a robust standardized way, to facilitate ease of capture. Also, the below defines the methods available to successfully capture such data, to ensure its representativeness with respect to the target population (MHM products or information), and the ability of innovators to reproduce it using the same standardized approaches over time, adding to their validity.

Timeliness of reporting indicators was also considered to be of relevance; for example, having baseline measures prior to provision of the interventions to prevent ‘contamination’ of assessing impact when describing supportive education on knowledge and practices. Time also has significance for the arrangement of data collection activities, with the understanding that seasonal events may hamper either collection or bias the types of responses given. For example, surveying girls at different timepoints in the year may incur unequal measures of absence, due to their obligation to help with harvesting; or may be impacted by expectations to attend school to prepare for exams in one term but not in another. Cognizance of capacity or ability to capture such data, and the costs associated with doing so, needs to be factored into an M&E plan and the approaches used, recognizing that limited but well characterized information on changes to the population over time is of more value than highly detailed data where in-depth collection is compromised. In addition, attention is also drawn to the many biases which innovators may unwittingly be challenged by (eg, reporting biases from school registers, misclassification of migration as school dropout, observer bias reporting on handwashing, desirability bias when respondents answer positively to please those delivering the program).

The indicators that were selected relate to different temporal components of a given MHM innovation, often described as “input” (contributions going into a project); ‘process’ (number and types of activities and outputs, such as products and deliverables), and ‘outcomes’ (measures of expected change in the short intermediate or long-term). For the purposes of this framework, indicators were broken down into activities, outputs, outcomes, and impact. While some organizations limit measures on outputs to the log frame only, others include outputs as intermediate outcomes. ⁶¹ For this TOC and M&E these were considered of relevance to measure to determine progress and identify bottlenecks.

Scale up activities

Actions taken, or work performed, through which inputs, such as funds, technical assistance and other types of resources are mobilized to produce specific outputs such as products, capital goods and services which result from product development and can include changes resulting from intervention which are relevant to achieving outcomes. The outcomes are the likely or achieved short- and medium-term effects of your project outputs. Meanwhile, the impact, positive, negative, primary, and secondary long-term effects produced by the product intervention, provided directly or indirectly, includes both intended or unintended effects.

Quantitative indicators are defined as measures of quantity, and are expressed as whole numbers, decimals, percentages, rates, fractions, and as a monetary value. Qualitative indicators are descriptions, judgments or perceptions about a subject, generated from questions that indicate whether the work performed leads to the expected changes. Qualitative indicators are expressed as independent statements or in relative terms such as good, better, and best, and cover measures such as performance and satisfaction with service provision.

The M&E framework

The M&E framework links directly with the TOC, with groups of indicators identified which will chart progress of the project. The framework is divided into five sections covering indicators for innovation, foundational elements, scale-up activities, outcomes, and impact (see Table 1 for exemplars). Uniformly, the M&E framework provides brief guidance on each selected indicator to advise innovators first on the definition of the indicator including a description, its type (eg, numerical categorical or qualitative terminology), what method is used as the source of the data (eg, cross-sectional survey or a focus group discussion), what the monitoring time would be (eg, baseline, endline or routine monitoring), and the frequency of data gathering. Additionally, the M&E tool offers some explanation to innovators on guiding the comparative evaluation of the data (eg, will it compare baseline with endline, or will it compare against another benchmark). Some of the major biases which are recognized to be a challenge with the collection of such an indicator are briefly noted, one example is not to give notice of a spot-check, as the institution visited may prepare making the spot-check findings unrepresentative of day-to-day activities. Table 2 below provides some exemplars per section; to review the full scope of suggested indicators, please see Appendix S2 in the Online Supplementary document.

Lastly, prevention of some of the major biases which are recognized to be a challenge with the collection of such indicators are briefly noted. This included the documentation of assumptions in our TOC (risk mitigation), as can be read in more detail in Appendix S2 in the Online Supplementary Document.

CONCLUSIONS

As aligned with the research priorities for adolescent girls in LMIC identified in 2014, “there remain a number of important gaps…filling these gaps will require new research studies that use an expanded range of methodologies, that enable the global community to better understand…the impact of MHM interventions, and the costs incurred to implement them effectively.” ⁶⁴ The paper also called for the need to “eliminate time and money spent on duplicating efforts….” ⁶⁴ The TOC and M&E framework described here, along with the more extensive content in the two appendices, aim to enable menstrual health innovators to strengthen the quality and impact of their efforts and improve the consistency of their delivery approaches, using a common framework, while reducing the duplication of efforts. Indicators provided are based on current experience – refinement of these through expert panel evaluations and further research to examine and enhance robust measures are welcomed. Further efforts to develop a holistic TOC that encompasses the entirety of the MHM definition, or TOCs that tackle other aspects, such as water and sanitation interventions are recommended.

Women and adolescent girls are using a clean menstrual management material to absorb or collect menstrual blood, that can be changed in privacy as often as necessary for the duration of a menstrual period, using soap and water for washing the body as required, and having access to facilities to dispose of used menstrual management materials.

Acknowledgements: The submitted article expresses the authors’ viewpoints and are not an official position of the institution or funder.
Funding: Grand Challenges Canada.
Authorship contributions: MS and PAPH wrote the grant to build the new measurement model. MS, PAPH, MLS, SK, and GZ drafted and edited the manuscript. KN and LG provided technical expertise and contributed to the refinement of the measurement model. All authors read and approved the final manuscript prior to submission.
Competing interests: The authors completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available upon request from the corresponding author), and declare no conflicts of interest.
Correspondence to:
Marni Sommer, DrPH, MSN, RN
722 W. 168^th Street, Room 537
New York, New York 10032
USA
[email protected]