Diarrhea kills over half a million children each year, roughly 120,000 of them in India. Fortunately, use of oral rehydration salts (ORS) could avert nearly all of these deaths.1–5 Thus, in 1978, The Lancet lauded ORS as “potentially the most important medical advance of this century”.6
Despite its effectiveness, caregivers in India treat only about half of childhood diarrhea cases with ORS.
To reduce diarrhea deaths, India’s Ministry of Health has run an “Intensified Diarrhea Control Fortnight” annually since 2014. The Fortnight plan is for community health workers (called Accredited Social Health Activists, ASHAs) to distribute free ORS to caregivers of all young children. One of the largest health interventions in the world, the Fortnight mobilizes roughly a million ASHAs to provide services to over 100 million children each year.
The intervention has the potential to be highly cost-effective. Earlier versions of a related intervention showed good results in three large Indian states: Gujarat, Uttar Pradesh and Bihar.7 Similarly, a randomized trial in Uganda found that roughly 90% of caregivers gave ORS to a child if the child became ill in the month after receiving free ORS.8
The Ministry of Health reported that the Intensified Diarrhea Control Fortnights in 2015 reached 63 million of the 100 million children it targeted.9 Under reasonable assumptions (Appendix S1 in the Online Supplementary Document), this level of coverage would save thousands of lives and make the Fortnight one of the most cost-effective health interventions since immunizations.
These encouraging results depend on the validity of the Ministry of Health’s claim about reaching families with ORS, coupled with caregivers using the free ORS. This study examines those issues.
The Intensified Diarrhea Control Fortnight
The Intensified Diarrhea Control Fortnight provides education, services and supplies to prevent and treat.9 The objective of the Fortnight is to ensure high usage of ORS and zinc in children with diarrhea.9
The launch of the Fortnight is typically headed by the Minister of Health and Family Welfare, with participation of UNICEF, WHO, the Gates Foundation, and others.10 The Fortnight is covered by media such as the Times of India,11 India Health News,12 and many regional newspapers.
The Fortnight’s intervention manual is comprehensive. It starts with forming committees in each state and district. It prescribes partnerships with schools, local NGOs, Anganwadi preschools, medical associations, and other government departments. The manual includes forms for planning and check sheets for monitoring.
The Fortnight starts with awareness-building activities: mass media like radio and TV ads, local media such as posters and wall paintings in each village, and targeted outreach in Anganwadi preschools (government preschool). The Fortnight also includes improvements in diarrhea case management by the public healthcare system. In addition, each facility in the health care system and each Anganwadi establishes an “ORS and Zinc Corner” that provides both information and free ORS and zinc for treatment.
The key message of the Fortnight is “giving Zinc for 14 days for children suffering from diarrhoea, even if diarrhoea stops.”13 Zinc distribution is supposed to occur at the ORS-Zinc Corners in health centers, ORS-Zinc corners in Anganwadi preschools, and via the household distribution by ASHAs. The plan is for ASHAs and Anganwadi centers to receive enough zinc to last the community both during the Fortnight and for the following 2-10 weeks. There are also messages encouraging handwashing with soap and safe sanitation to prevent diarrhea.
During the Fortnight, ASHAs are supposed to visit all homes with children under age 5. ASHAs teach caregivers that ORS plus zinc is the appropriate treatment for diarrhea. Perhaps most importantly, ASHAs distribute an ORS sachet for each child. ORS is very inexpensive (a few cents per sachet) and provides most of the health benefits of using both ORS and zinc.
Then, when a child first becomes ill after the Fortnight delivery, the child’s caregivers can provide ORS from their sachet that is already in the home. If the illness occurs soon after the Fortnight, the caregivers can also retrieve ORS and zinc from their Anganwadi preschool. Further, ASHAs are supposed to have ORS and zinc regardless of the season.9
Each ASHA receives roughly Rs 100 (US$1.50) if she visits all the children in her area. This payment is below the market wage for the 10-20 hours needed to distribute all of the ORS requires (see assumptions in Appendix S1 in the Online Supplementary Document).
In addition to the ORS she distributes, the Fortnight plans for each ASHA to receive enough ORS and zinc to treat a case of diarrhea in about 30% of the children in her catchment area. Each Anganwadi preschool is to have enough ORS and zinc to treat about 10% of the children it serves. With about 2.44 diarrhea cases per child per year, the zinc that ASHAs and anganwadis have should last roughly 2 months after the Fortnight ends.
The Fortnight manual also requires that higher-level government officials 2% of homes to monitor household visits.
This intervention has enormous potential. Several studies have found that community health workers are effective in low-income countries,14,15 specifically in child health care.16–19 Randomized controlled trials in rural eastern India and Uttar Pradesh show that health workers carrying out home visits can improve children’s growth and health.20,21
As noted above, a study in Uganda found that almost all caregivers who received ORS used it if a child became ill.8 That trial also showed a challenge: community health workers did not always deliver ORS. The Uganda study had the community health workers verbally agree to distribute the ORS. It also had a larger financial incentive than ASHAs receive in India’s Fortnight. The Uganda intervention paid $5 for each health worker and promised a second $5 conditional on making the planned household visits. Nevertheless, only 60% of households reported receiving the free ORS. Many other studies of community health workers have noted the challenges of ensuring they complete their home visits and educational activities.22
The Uganda study found that community health workers reached only 60% of homes, but ORS usage still rose 19 percentage points (among homes with a sick child). This share of homes contacted in Uganda is similar to the 63% of children the Ministry of Health claims the Fortnight reached in India.
Inappropriate treatments
Although caregivers in India routinely under-use ORS and zinc when a child has diarrhea, they also frequently over-treat with inappropriate medications. Nationally, caregivers report that they treated 19.3% of child diarrhea cases with antibiotics, 5.5% with anti-motility drugs, and 12.2% with herbal remedies or other treatments.23(p257)
These forms of over-treatment are common in other poor nations. “These practices can reduce correct management of diarrheal disease in children and result in treatment failure, sustained nutritional deficits, and increased diarrhea mortality.”24 Consistent with medical evidence, the Fortnight training materials explain, “Antibiotics are not effective in treating most diarrhea: They rarely help and make some children sicker; unnecessary use of antibiotics may increase the resistance of some pathogens; they are costly.”25(p15)
The Fortnight training materials also explain, “Never give antidiarrheal drugs to children”25(p15) and that intravenous fluids are not necessary for addressing uncomplicated diarrhea. (We define uncomplicated diarrhea as having neither a fever nor blood in the stools.)
Unless the family is purchasing ORS or zinc from a private provider, private health care for uncomplicated diarrhea wastes both money and time. ORS should also reduce complications that lead to referrals to additional care.5 Thus, we also analyze whether the Fortnight reduces use of unnecessary private health care for uncomplicated diarrhea cases.
Specific Aims
Our specific aims are to test each step of the Fortnight’s theory of change. Specifically, we test whether in the months following the Fortnight, caregivers were more likely to have met with an ASHA or other caregiver and whether they were more likely to report familiarity with ORS. For children with diarrhea in the 2 weeks prior to the survey, we look for increases in treatment with ORS and with zinc. For children with uncomplicated diarrhea, we also look for inappropriate care: antibiotics, other inappropriate care, and costly private care.
METHODS
Data sources
We analyzed India’s National Family Health Survey of 2015-2016.9 The National Family Health Survey (NFHS) surveys a representative sample of roughly 600,000 Indian households. It targets the mother or other female caregiver to answer survey questions on child health. The National Family Health Survey uses multiple measures to ensure high data quality.23
Sample
In 2015, the Fortnight took place July 27 to August 8, and in 2016 it was July 11 to July 23.23 Because the NFHS takes 2 years to survey the entire nation, about a fourth of states have data collection both in April through June (just before the Fortnight) and in August and September (just after the Fortnight). Figure 1 is a map of the states we study, by year of the NFHS (2015 or 2016). Appendix S2 in the Online Supplementary Document gives the sample size for each state.
The selection of states depended on the rollout of the NFHS data collection. While not random, the states span a fair portion of India.
Data collection and case definitions
We analyze respondents caring for a child under 5 years of age.
The NFHS survey asks whether the respondent met with an anganwadi worker, ASHA or other community health worker in the last 3 months. If the respondent answers “Yes”, the surveyor asks the job title of the most recent community health worker the respondent met with. We are most interested in whether an ASHA has visited the home in the last 3 months. At the same time, because the NFHS survey asks only about the most recent contact, it is possible that an increase in non-ASHA health workers masks a true increase in ASHA visits. Thus, we also examine contact with any community health worker. We then test if self-reported familiarity with ORS rises after the Fortnight.
At homes with a child under five years of age, the enumerator first asks if any under-five child experienced diarrhea in the last 2 weeks. Our remaining outcomes examine children who had diarrhea in the past 2 weeks.
The survey asks about treatment for diarrhea. Our primary outcome measure is use of ORS. We also look at a secondary outcome, usage of zinc.
We look at two definitions of unnecessary care for uncomplicated diarrhea. Our narrow definition includes only antibiotics. Our broader definition of inappropriate care also includes antidiarrheal drugs, herbal drugs, unknown medicines, and injections. Antibiotics are appropriate if there is cholera, blood in the diarrhea, or a separate bacterial infection.25(p5) Thus, for the analysis of inappropriate care we drop observations when the child has blood in the stools or has a fever (as the fever may be due to a bacterial infection).
When measuring unnecessary private care, we define a private provider as a private hospital, doctor/clinic, paramedic, traditional healer, or other private health care provider. A family may have purchased the appropriate treatments of ORS and/or zinc at a private pharmacy. Thus, we include private pharmacies in the list of inappropriate treatment providers only if a child received neither ORS nor zinc.
Data analysis
We examine the Intensified Diarrhea Control Fortnights that took place July 27 to August 8, 2015 and July 11-23, 2016.23 Our core specifications compare the three months (April, May and June) prior to the Fortnight’s month (largely July) each year to the 2 months after (August and September). We discuss implications of part of the 2015 Fortnight having taken place in August below.
Cross-fit partialing out Lasso Logit regressions
The timing of the NFHS data collection was arbitrary, but not random. Thus, omitted variables might correlate with both the timing of the NFHS and outcomes such as ORS usage. Fortunately, the NFHS is a rich dataset with hundreds of potential control variables. Unfortunately, having hundreds of potential control variables leads to the familiar risk of overfitting, so our estimates will fit well in sample but have no predictive power. Also, unfortunately, if we can select which subset of potential control variables to include, we might unconsciously select covariates that lead to desired estimates.
To avoid these problems, we use a machine-learning method: cross-fit partialing out Lasso.26,27 This method uses a Lasso regression to identify covariates that predict either which observations are in the treatment group (in our setting, the period following the Fortnight) or that predict the outcome (ORS usage, etc.).
A Lasso regression fits the data while also penalizing the number of covariates. This penalty drives small effect sizes to zero. To eliminate a degree of freedom for data mining, we use the default options in Stata’s xpologit command, including three-fold cross-validation and plugin estimators to determine how strong the penalty on the number of covariates should be. Each regression also includes a separate intercept for each state, which we force to be included. Both outcomes and treatment are binary, so we estimate Lasso logistic regressions.
In a second stage, for each outcome, our estimating equation partials out all covariates that predict either being in the post-Fortnight period or that predict the outcome. Sampling error is a problem if we use the same data to choose what variables to retain as to estimate the final equation. Thus, the cross-fit method splits the sample to separate the selection of what variables to include and how they predict treatment or outcomes.
This method yields unbiased estimates if we have sufficient control variables so that either the error in the treatment equation is (conditional on the covariates) uncorrelated with outcomes or the error in the outcomes equation (conditional on the covariates) is uncorrelated with treatment. The estimated standard errors are robust to nonnormality of the error and to heteroscedasticity.
We started with 50 potential control variables that measure household characteristics such as wealth and decision-making patterns in the household; respondent characteristics such as occupation; and husband characteristics such as education and occupation (Appendix S3 in the Online Supplementary Document). We chose these control variables because they may capture knowledge, isolation, and ability to pay that we hypothesize could affect usage of ORS and zinc.
We recorded missing values at the mean for continuous variables and at the mode for categorical variables. We added a separate dummy for common missing values. We recode categorical variables into a set of dummies. To capture potential non-linearities, we include interactions of all main effects and the squares of all continuous variables, leading to 17,607 potential control variables. Due to computational constraints, we have opted to remove variables that exhibit collinearity within the sample of families with a child who had experienced diarrhea within the last two weeks. It is important to note that the removal of variables was not conducted on a per-regression basis, but rather on the sample as a whole. As a result of this process, a total of 13,984 variables were eliminated due to either collinearity with another variable or a lack of variation.
Double difference with treatment for fever
It is possible that a shock affects caregiving for ill children in general during the period from just before the Fortnight to just after. For example, there might be a seasonal effect in accessing the public health service. As a placebo test, we examine treatment for a cough or fever (in cases without diarrhea). Our hypothesis is that ORS usage will increase more rapidly than treatment for cough or fever after the Fortnight. In a double-difference specification, we look at the effect of the Fortnight on ORS versus its effect on treating a cough or fever.
Characteristics of the sample
Most of our sample is disadvantaged. Looking at the pre-Fortnight subsample, 80% live in rural areas and over half have primary education or less. 10.6% report diarrhea for a child under 5 years old in the 2 weeks prior to the survey (Table 1).
If the timing of the survey (just before versus just after the Fortnight) were random, then the characteristics of respondents should be similar in the three months prior to the Fortnight and the two months after. We tabulate that difference in Table 1. The later period has a lower share with no formal education (39% early, 24% late) and a correspondingly higher share of secondary education (39% vs. 53%). The difference in rural (80% early, 75% late) and wealth quintile (0.3 quintiles lower in the early period) also show the early period is less advantaged than the later period. Thus, as noted above, we control for these respondent and child characteristics in the analyses.
About half of the differences in characteristics between the pre- and post-Fortnight samples is because the pre-Fortnight sample has more respondents from very poor states. To be more precise, the pre-Fortnight sample comprises 63% of respondents from the three Indian states with the lowest rankings on the Human Development Index (Bihar, Uttar Pradesh, and Jharkhand), while the post-Fortnight sample only includes 40% from these states. The Human Development Index (HDI) takes into account factors such as life expectancy, average years of education, and per capita income. As noted above, we control for state fixed effects in the analyses to remove the effects of this imbalance.
RESULTS
We analyze each step of the theory of change: seeing an ASHA or other health care worker and familiarity with ORS for all caregivers, as well as ORS usage, zinc usage, and inappropriate care for caregivers of a child with recent diarrhea.
Health care and AHSA visits
At baseline, only 40% of respondents said they had met with any health worker (ASHA, Anganwadi worker, or other community health worker) in the last 3 months. Of those with any contact, about 70% said an ASHA was the most recent worker they had met with (that is, 28% both said they had met with a health worker in the last 3 months and in a follow-up question said an ASHA was the most recent health worker).
There is no evidence the Fortnight increased visits by ASHAs to homes (dP/dx=-1.38 percentage points, odds ratio, OR=0.946, 95% confidence interval, CI=0.878-1.019, P=0.142, Table 2, col. 1). We cannot reject the null hypothesis that there is no relationship between the Fortnight and the visits by ASHAs to homes. Recall the Fortnight’s plan was for 100% of homes with a child under age 5 to receive an ASHA visit. Reaching that goal would have required at least a 60 percentage point increase in AHSA contact, not the near-zero effect we estimate.
There is also no evidence that the Fortnight increased overall access to the community health care system, defined as any contact with an Anganwadi worker, ASHA or other community health worker (dP/dx =-0.15 percentage points, odds ratio 0.9940, CI[0.936,1.056], P=0.848, Table 2, col. 2).
Familiarity with ORS
The baseline level of familiarity with ORS is high: 80% of respondents who had a child with diarrhea in the pre-Fortnight period reported they were familiar with ORS.
In spite of the high baseline, familiarity with ORS rose 3.24 percentage points (OR=1.139, CI=1.050-1.235, P<0.01, Table 2, col. 3) after the Fortnight.
ORS usage
The core results of the paper examine if ORS usage increased after the Fortnight. The sample is now a child under 5 years with diarrhea in the last 2 weeks. At baseline, 46% of respondents reported using ORS to treat the child’s recent or ongoing diarrhea.
The estimated effect is 1.89 percentage points (Table 3, col. 1). This estimate is not statistically significantly different from zero (OR=1.079, CI=0.899-1.294, P=0.415). Note that the actual impact of the intervention may be marginally higher or lower than the estimated effect. (Appendix S6 in the Online Supplementary Document)
Treatment with zinc
At baseline 18% of caregivers of a child with diarrhea reported they administered zinc. In the cross-fit partialing-out lasso logistic regression, zinc usage is 1.74 percentage points higher after the fortnight (August and September) than in the 3 months prior to the fortnight (OR=1.072, CI=0.859-1.339, P=0.538, Table 3, col. 2).
Inappropriate care
We analyze three measures of inappropriate care for uncomplicated diarrhea: antibiotic use, a broader definition of inappropriate care that also includes care such as herbal care, and private care. These analyses drop observations with blood in the stool or a fever.
At baseline, 18% of caregivers of a child who had diarrhea with no fever or blood in the stool use antibiotics to treat uncomplicated diarrhea. The antibiotic use decreases after the Fortnight by 2.01 percentage points (CI for odds ratio=0.651,1.308, P=0.651, Table 4, col. 1), but the change is nowhere close to statistically significant.
Adding in other inappropriate treatments (antidiarrheal drugs, herbal drugs, unknown medicines, and injections), the share of uncomplicated diarrhea cases treated inappropriately at baseline rises to 35%. The decrease in inappropriate care after the Fortnight is slightly larger than for antibiotics alone, 2.88 percentage points, and remains statistically insignificant (CI for odds ratio=0.676,1.174, P=0.413, Table 4, col. 2).
Our third measure of inappropriate care is private care. Recall we do not classify private pharmacies as unnecessary if the child received ORS or zinc. At baseline, caregivers visit private providers in 50% of uncomplicated diarrhea cases that did not receive ORS or zinc. In the two months following the Fortnight, use of private care for ORS cases fell by a small and statistically insignificant 5.14 percentage points (CI for odds ratio=0.620,1.068, P=0.138, Table 4, col. 3).
In short, most uncomplicated diarrhea cases in India receive inappropriate and/or needlessly expensive care. There is no consistent reduction in inappropriate care from just before to just after the Fortnight.
Robustness checks
The higher use of ORS after the Fortnight might be a general change in the propensity to access health care. Thus, we estimate the double difference looking at changes in trends in ORS versus change in trends in treatment for a fever or cough (in cases without diarrhea). This equation has the full set of interactions between each state and a variable indicating whether the observation belongs to the ORS sample. The point estimate on treatment with ORS post-Fortnight rises slightly compared to the results in Table 3 (dP/dx=1.89 percentage points, OR=1.079, CI=0.899,1.294, P=0.415), although the rise is not close to statistically significant. In short, there is no evidence that the effects we estimate are influenced by other factors that affect willingness to treat common child illnesses.
Bihar contributes a fourth of the pre-Fortnight observations and only 2% of the post-Fortnight families with a child under 5 years who had diarrhea in the last 2 weeks. Although we include state fixed effects, it is possible this disproportionate sample from one large state biases our estimates. Results on ORS and zinc usage are almost unchanged if we drop Bihar (results not shown). More generally, results are similar if we dropped any individual state that has a disproportionate sample.
More extensive treatment or private care may be appropriate for a child who is very malnourished. When we dropped children with BMI more than 3 standard deviations below their age norm, results on inappropriate care and private care were almost identical (results not shown).
To avoid the problem of biased specification search, we also ran all regressions using control variables that mimic those in Wagner, et al.8 As that project chose what measures to collect several years prior to starting this project, using the Wagner, et al.8 specification avoids data mining. These variables included the child’s age and sex and the respondent’s age and education.
Results with this specification are similar to those in the text. A modest exception is that the effect of the Fortnight on ORS usage. In the logit regression with predetermined control variables, ORS usage is 4.9 percentage points higher after the Fortnight (August and September) than in the 3 months prior to the Fortnight (CI=0.8-9.1%, P<0.05, OR=1.24, Table S2 in the Online Supplementary Document, col. 1). The point estimate is slightly larger than estimated by the machine learning method in Table 3, col. 1, though the change is nowhere near statistically significant.
ASHAs are more active in rural areas, on average. Thus, we ran the analysis separately for urban and rural areas. Results were similar in each.
DISCUSSION
We can rule out the hypothesis that ASHA visits increased substantially after the Fortnight. It is plausible that Rs 100 (US$1.50) is not a strong enough incentive to encourage ASHAs to visit all the children (or that some ASHAs never received instructions to participate in the Fortnight).
Awareness of ORS rose after the Fortnight. Given the small increase in contact with the health care system, these results suggest that mass media campaigns and/or the ORS-Zinc Corners in anganwadi preschools may have increased awareness.
There is not much evidence it also increased ORS usage, though the point estimates are positive and we cannot rule out small increases. On the one hand, we had dozens of potential control variables and used a powerful machine learning method to reduce confounds. On the other hand, it is possible that omitted variables still bias these estimates. Rates of diarrheal disease historically have increased during the monsoon months (roughly June through September). That pattern does not show up in the 2015/16 NFHS (results not shown). Nevertheless, there is no plausible bias that overturns our main conclusion: these estimates rule out that the Fortnight reached most young children (as claimed by the Ministry of Health and Family Welfare23). That is, this nation-wide program is much less successful than smaller programs in India7 or other nations.8
Policy implications
It would be easier to track the progress of the Fortnight and identify barriers if each level of government posted expenditures and monitoring data on the Web. Mobile phones with cameras and GPS can also help ASHAs document their distribution. Having national data would extend the limited coverage of this study.
A follow-up evaluation should identify the barriers to the Fortnight’s success. Some states may never have allocated funds. Where states allocated funds, if ASHAs never receive their ORS packets, then improving the supply chain is key. If ASHAs’ low compensation is an important barrier, then it might be cost-effective to increase payments to ASHAs. Higher compensation may be most important for ASHAs serving communities where travel time is high (dispersed homes, poor roads, etc.). If usage is low among caregivers who receive ORS, then better education for caregivers is probably helpful.
There is no evidence that the Fortnight reduces antibiotics or other inappropriate care for uncomplicated diarrhea. It probably is cost-effective to provide more messages on not overusing antibiotics and other inappropriate forms of care.
The wholesale cost of ORS is only about US$0.03 per packet.23(p7) Thus, repeating free distribution of ORS throughout the year might increase benefits substantially with only modest increases in cost. For example, Anganwadi preschools could distribute free ORS a few times a year. Anganwadi workers can also give ORS to caregivers when they notice a child has diarrhea and when a child returns to the anganwadi after an illness (if the caregiver reports diarrhea). Also, health centers could distribute ORS at all immunizations and routine medical visits for young children.
More generally, ASHA programs in general and the many interventions that ASHAs carry out deserve more careful monitoring and evaluation. Rigorous evaluations have found that some interventions using community health workers are highly effective.18 At the same time, many evaluations find that community health workers are not effective on average, largely because they are not always very active.28
In short, India suffers from many thousands of deaths that ORS could prevent. On paper, the Intensified Diarrhea Control Fortnight is one of the largest public health interventions in history. Unfortunately, it is not working. These results highlight the opportunity to improve the Fortnight and other means to distribute free ORS to caregivers.
While distributing free products is sometimes frowned upon,29 this intervention has the potential to be almost as cost-effective as immunizations.30 Thus, all regions with a high burden of diarrheal disease should consider scaling up free distribution of ORS prior to illness.
CONCLUSIONS
This is the first evaluation of the enormous Diarrhea Control Fortnight. Unfortunately, there is no consistent evidence that the Fortnight increased ASHA visits. Familiarity with ORS rises 3.24 percentage points after the Fortnight.
Most importantly, ORS usage rises by 1.9 percentage points (compared to the pre-Fortnight rate of 46%, P=0.415). Zinc use rises 1.74 percentage points, compared to a baseline of 18 percent (P=0.538). Neither increase is statistically significant at conventional levels. The Fortnight does not consistently reduce measures of unnecessary care (antibiotics, etc.).
Acknowledgements
Discussions with Zach Wagner were very helpful. Jeseo Park helped review the literature. Ethical approval. Because this study examines anonymized archival data, there was no ethical review.
Funding
IRLE at UC Berkeley funded this research, and had no role in the design, analysis or write-up of the study.
Competing interests
The authors declare that there are no conflict interests regarding the publication of this paper. The authors completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available upon request from the corresponding author), and declare no conflicts of interest.
Correspondence to:
Xinyu Ren
Industrial Engineering and Operations Research
University of California, Berkeley,
California, US
[email protected]