• MONDAY, JULY 22, 2019
  • ISSN 2399-1623
Mary Carol Jennings, Dunstan R. Bishanga, Sheena Currie, Barbara Rawlins, Gaudiosa Tibaijuka, Ahmad Makuwani, Jim Ricca, John George, Filbert Mpogoro, Stella Abwao, Lusekelo Njonge, Jeremie Zougrana, and Marya Plotkin

Abstract

Background

Globally, an estimated 2.7 million neonates died in 2015 during the neonatal period, including approximately 0.7 million intrapartum deaths. In Tanzania, 46,000 neonates died in 2016. Nearly two-thirds of these deaths could have been prevented through improvements in quality of care at the time of delivery. Scaling up the use of Doppler for fetal assessment may support such improvements. We describe and assess the inputs and processes used to integrate Doppler into the workflow of maternal triage upon admission to Tanzanian labor and delivery wards.

Methods

We conducted a mixed-methods study on the inputs and processes used to integrate Doppler use into the workflow of maternal triage upon admission to maternity services. Our study took place in 10 health facilities in Kagera Region, Tanzania, from November 2016 to April 2017. We trained providers on using the Doppler and assessed the logistical and resource inputs needed to integrate the Doppler into routine care. To assess the transfer of knowledge and skills, we administered pre- and post-assessments at the time of training and then observed providers in practice 6 months after the training. Results We trained 163 providers to use the Doppler and record initial fetal heart rates in the facility registry; 87 providers underwent observation while triaging 112 women for admission to maternity services. Providers’ mean knowledge score increased significantly from pre-test to post-test. Twelve percent of the providers failed the 15-question, objective structured clinical examination on their first try, but all passed upon coaching and remediation. The average admission took 30.6 minutes, 4.1 minutes of which were used to assess fetal heart rate using the Doppler (including locating and using the device and storing and recording the results).

Conclusions

The training sessions effectively conveyed knowledge and skills for integrating the Doppler into routine triage and admission workflows. This bodes well for the potential scale-up of Doppler use in labor and delivery services. Findings on structural needs indicated that minimal resource inputs will likely be required to maintain the use of the device in practice.


In 2015, 2.7 million neonates died around the world. Intrapartum-related events were the third leading cause of these deaths, accounting for an estimated 700,000 (1). A majority of these newborn deaths occurred in low- and middle-income settings (2), such as Tanzania. This country is among the 10 that account for 60% of global deaths in children under 5 years of age, with disproportionate rates of newborn death (1).

Although the neonatal death rate in Tanzania has dropped dramatically from 41 deaths per 1,000 live births in 1990 to 22 deaths per 1,000 live births in 2016 (3), mirroring a continued decrease in neonatal deaths from 47,000 in 2015 to 46,000 in 2016 (4), the rates of perinatal and neonatal mortality remain unacceptably high. In a comparison of modeled and observed neonatal mortality rates in six low- and middle-income countries (LMIC), Tanzania had an extraordinarily high level of very early neonatal deaths (deaths which occur within 24 hours of delivery) (65.5%) (5). Also in 2015, the stillbirth rate in Tanzania was 22 deaths per 1,000 live births (6), vastly exceeding World Health Organization’s target of 12 or fewer stillbirths per 1,000 live births (2).

Interventions that focus on the prevention of stillbirths and very early neonatal deaths are well studied (2, 7), but their scale-up, from a pilot or trial to an intervention that benefits the broader population, has faced challenges in many settings, especially sub-Saharan Africa (8). Reaching the United Nation’s Sustainable Development Goals targets for maternal mortality and preventable newborn death will require concerted scale-up of interventions that improve care at the time of delivery (9). However, the literature on scaling up interventions for disease prevention and health promotion focuses little attention on how to support adoption, coverage, and standardized delivery of new interventions at scale (10).

The World Health Organization recommends auscultation using a Doppler or Pinard stethoscope to assess fetal well-being upon a pregnant woman’s admission to the labor and delivery ward (11), reinforcing additional international and Tanzanian standards of care that practitioners measure and record fetal heart rates (FHR) upon admission (12, 13). The practice of FHR assessment directly impacts a facility’s ability to provide quality of care. A study at Tanzania’s Muhimbili National Referral Hospital attributed 40% of audited perinatal deaths to poor fetal heart monitoring—a critical issue being poor documentation (14). However, the Tanzanian Health Management Information System (HMIS) does not include a field to collect FHR data. This gap is significant because a large proportion of the very early neonatal death that occurs in a facility may be responsive to improvements in the quality of delivery care. But facilities lack an indicator to track improvements.

The World Health Organization’s guidance does not specify a preferred method of FHR auscultation (11). Although Pinard stethoscopes are commonly used to monitor FHRs in Tanzania, there are benefits to using a Doppler rather than a Pinard stethoscope, and multiple studies in Tanzania have explored the practicalities of doing so (1518). Most commonly noted reasons for using Doppler include increased rate of detection of abnormal FHRs (16, 17, 19) and client preference (20, 21).

This study is part of a parent study that validated an indicator of perinatal mortality that occurs after admission to health facilities (22).The objective of this mixed-methods study was to describe the inputs and processes used to integrate Doppler use into the workflow of maternal triage upon admission to Tanzanian maternity wards. This study specifically focused on describing and assessing the provider training that transferred knowledge and skills to operate and maintain the Doppler; the study also estimated the provider time and facility logistics needed to support the continued use of Doppler. Inputs were described in terms of training, knowledge transfer, physical materials (i.e. ultrasound gel) and facility resources (i.e. provider time), while processes were described in terms of the sequence and nature of actions that providers performed in order to use the Doppler in practice. Findings will be useful to facility managers and program planners as they make policy and implementation decisions on scaling up Doppler use for maternity care.

METHODS

Study design

This mixed-methods study integrated Doppler use and standardized FHR data collection into the process of admitting women in labor to maternity services. The study took place in 10 health facilities in Kagera Region, Tanzania, from November 2016 to April 2017 and was part of a parent study that demonstrated the validity of an indicator to track perinatal deaths that occurred after admission to the maternity ward. The parent study has been previously described (22).

In the study described here, a team of 6 experts trained midwives to use a handheld Doppler to assess FHR when triaging a pregnant woman for admission to the facility, and to accurately and consistently record the measured data in the HMIS register. The training included a simple refresher on how to diagnose, classify, and manage common labor and delivery clinical outcomes, with a focus on perinatal mortality outcomes. The training team assessed didactic clinical knowledge transfer with simple pre- and post-training assessments, and assessed transfer of clinical skills with an objective structured clinical examination (OSCE).

After an initial period of 3-6 weeks, a team of 5 research assistants observed providers in their labor and delivery triage workstations. We employed a provider-observation tool to collect data on how well providers retained their knowledge and skills from the training, and to examine the time needed to perform steps associated with triaging and admitting a patient when using the Doppler. We conducted a detailed assessment of physical and logistical facility inputs associated with continued use and maintenance of the device.

Selecting study region and sampling health facilities

This study was conducted in facilities supported by the Maternal and Child Survival Program (MCSP), which is funded by the United States Agency for International Development. The facilities were located throughout Kagera Region, a region targeted for development efforts because of its elevated maternal and infant mortality rates. At the time of study design, this region had Tanzania’s highest infant mortality rate of 62 deaths per 1,000 live births, compared to the national rate of 46 deaths per 1,000 live births (23). Out of eight district councils and six districts in the Kagera Region, we selected four that were within a feasible (1-day) travel distance from the region’s central office, were considered by locally experienced staff to be representative of the region as a whole, and had health facilities which had received MCSP support. The four districts were Bukoba, Bukoba Municipal Council, Muleba, and Karagwe.

At the time of the study, MCSP-supported health facilities in these districts had received more than 2 years of technical assistance to improve maternal and newborn care. Facilities were eligible for inclusion if they had a) at least 360 deliveries annually and b) five or more skilled birth attendants listed on their labor and delivery ward rosters. Ten facilities -- three health centers, six district hospitals, and one regional referral hospital – met the criteria and were included. Annual delivery volume in the study sites ranged from 385 to 4,388 births in 2015, at the time of study design.

Our study utilized the Moyo Fetal Heart Rate Monitor, a handheld Doppler device manufactured by Laerdal Global Health and specifically designed for the LMIC setting (https://laerdalglobalhealth.com/products/moyo-fetal-heart-rate-monitor/).

Sampling health care providers

We invited all eligible health care providers at the 10 study facilities to participate in a half-day of training. Providers were eligible for inclusion if they were skilled birth attendants (e.g., nurse, nurse-midwife, or physician) who admitted women to the labor and delivery ward of a study facility. At the time of study design, facility staff records listed 201 eligible providers, 81% (163 providers) of whom were enrolled in the study and trained to assess FHR using the Doppler. Training sessions were between 3 to 4 hours in duration and were conducted on site at each facility during clinical duty hours. Each provider took part in the one training held at their facility. The number of providers at each training varied with the population of the health facility staff.

Study intervention: training health care providers

A mix of six regional and national trainers facilitated each training. Facilitators were seasoned health professions educators skilled in the provision of maternal and newborn health care services, with global and locally specific expertise in obstetric practice, research, and program design, implementation, and evaluation. Regional health managers and facility in-charges were briefed before and after the training through three orientation sessions that communicated study objectives and scope—specifically obtaining their buy-in for the inclusion of a field for FHR data in each study facility’s HMIS register (called MTUHA Book 12).

The training consisted of a didactic session followed by a practicum component. Content was derived from the World Health Organization’s International Statistical Classification of Diseases and Related Problems (tenth revision) for perinatal deaths (24), Tanzania’s Maternal and Perinatal Death Surveillance and Response Guidelines (25), and user instructions and specifications in the training materials for Laerdal’s Moyo Fetal Heart Rate Monitor (26).

Training assessment

The initial didactic component provided a refresher on the proper classification of stillbirth and newborn deaths; a refresher on the core clinical steps in triaging and assessing a woman in labor for admission; instructions on using the Doppler to assess FHR; and guidance on the clinical importance of not only conducting these assessments but completely and accurately recording the data in the register. An eight-question knowledge assessment was developed to test providers on the specific content conveyed during the training; it was administered before the training and a reordered version was administered after the training.

In the practicum component, study trainers were individually assigned to assess each trainee using an OSCE based on the Moyo Doppler user guide (26). The OSCE was specifically developed for the study by clinical education experts based within and outside Tanzania; they used other OSCEs as models (specifically, an OSCE on respectful maternal care). During the OSCE, the assigned study trainer assessed and coached each provider through a hands-on clinical scenario on using the Doppler to triage and admit a standardized patient and then to properly record the fetal and maternal assessments in the register. Because the study focused on use of Doppler for admission to labor and delivery services and maternal triage, a longer and more detailed training on the clinical steps necessary if an abnormal FHR was detected was outside the scope of this training.

Assessment of Doppler adoption into labor and delivery admission

Study research assistants subsequently used the provider workflow observation tool to collect data on provider actions as the provider admitted the woman into the labor and delivery ward. The tool used a time-motion format to document the sequence and timing of steps provided upon admission, as well as the facility infrastructure used to maintain and store the Doppler (charging station, cleaning supplies, etc.). The observation tool was designed based on quality improvement instruments, including those of Lean Six Sigma (27). The design used techniques of time measurement, process mapping, and value-stream mapping, adapted to conform to World Health Organization –recommended practice upon admission (11) and guidance from the Tanzania Ministry of Health, Community Development, Gender, Elderly and Children (28). Among other factors, data on how much time a health provider uses in employing a new intervention helps health systems planners to assess the demands that the new intervention places on the health workforce (29); we measured the timing (30) of key tasks that needed to be performed to integrate the Doppler into the maternal triage and admission clinical workflow. Study staff provided memos and comments on observations to augment the time-motion findings.

We divided provider observations a priori into five categories: intake and booking, admission procedures, initial physical assessment, activities and data collection to assess and record fetal status using the Doppler, and final assessment and recording of data in the register. Content of these categories included gaining maternal permission from and checking maternal understanding of the use of the Doppler; correctly using the Doppler; recording the FHR assessment into the register; and conducting additional routine clinical care steps to triage a patient for admission to maternity services.

We also evaluated Doppler use in relation to the facility’s physical infrastructure, along with other material resource inputs involved in implementing the new workflow.

Data collection tools

The study utilized three data collection forms: the provider knowledge assessment, completed before and after each training session; the OSCE assessment, completed at the end of each training session; and a provider workflow observation tool, which was used within 6 months following the training sessions, to directly observe health care providers as they triaged laboring pregnant women for admission to labor and delivery services.

Pre- and post-training knowledge assessments

Study trainers administered the eight-question knowledge assessments on site after the providers had registered and again immediately following the training session. Providers could not refer to notes or sources while completing the assessment. The post-training questionnaire asked the same questions as the pre-test questionnaire, but in a different order.

OSCE

Study trainers administered the OSCE individually to each provider at the conclusion of the didactic and demonstration components of the training. The OSCE aimed to evaluate each provider’s skills, knowledge, and competency in using the Doppler. Each OSCE required 30–60 minutes to complete. Providers (or trainees) were deemed competent at the end of the training session if they correctly performed 80% (12 of the 15 steps) of the steps in the OSCE. Trainees who did not obtain a passing score on the OSCE underwent remediation and then reassessment until each was evaluated as competent. Healthy pregnant women from the facility, who were not in active labor, volunteered as standardized patients for the OSCE assessments.

Observing the triage of a woman in labor for admission to maternity services

The provider workflow observation tool used a time-motion format in which the trained observer evaluated health care providers as they triaged pregnant women for admission. Start and stop times for workflow steps and related details were recorded using this tool.

Data collection procedures

Research assistants – all of whom were nurses or nurse-midwives—received a 4-day training on research ethics and study tools and procedures. During the data collection period, a regional study coordinator supervised the research assistants both on site and remotely. Each research assistant was assigned to two facilities and visited the facilities on a rotating basis.

During October 2016, 163 health care providers were trained at their facilities to use the Doppler and correctly classify perinatal mortality outcomes. Data on training outcomes were collected using paper-based observation checklists as part of OSCEs. Observations of health care providers admitting women to the labor and delivery ward were made using a tablet.

Observations of 87 providers were conducted between November 2016 and April 2017. Over the study period, a research assistant attempted to conduct at least one observation of each trained health care provider (some health care providers who attended the training session did not work in the ward afterwards, due to rotation, leave, or illness). The research assistant obtained written informed consent from the provider and verbal informed assent from the patient, observed the admission, and entered data using a tablet computer.

Study staff regularly assessed the condition of the Doppler at the facilities, addressed any technical problems, and encouraged facility providers to use the Doppler for every admission.

Data management

The paper forms for the OSCE and training were transferred to Dar es Salaam, entered in Microsoft Excel, and imported to STATA (https://www.stata.com/) (31). CommCare software (32) – a mobile web-based program – was used for data capture of the provider observations. Completed forms of provider observations were uploaded to a secure server at the end of each day.

Analysis

Descriptive statistics from the assessment of knowledge and skills and from observations of the clinical workflow were calculated in STATA. The statistical significance of the change in provider knowledge was assessed using paired Student’s t-test and Fisher’s exact test, while the Shapiro–Wilk test was used to test normality distribution of the providers’ performance scores before and after assessments. Data on timing of clinical steps, obtained from the clinical workflow observation tool, were analyzed in STATA and then used to map the process workflow. We qualitatively analyzed the free-response data from the clinical workflow observation tool to describe barriers to and facilitators of using the Doppler.

Ethical oversight

This study was submitted to the Johns Hopkins Bloomberg School of Public Health Institutional Review Board (IRB) and was determined to be exempt (JHU IRB #00007059). The study was approved by the Tanzania National Institute of Medical Research (NIMR) IRB (NIMR/HQ/R.8a/vol.IX/2219).

Consent and confidentiality

Health care providers signed a written informed consent statement before being observed conducting facility admissions. Oral assent was provided by all mothers whose admissions were observed. Pregnant women who required immediate emergency care or referral to a higher level of care upon admission (before the assessment of FHR) were excluded from the study, ensuring uninterrupted provision of urgent and/or emergent care.

RESULTS

Characteristics of training participants

Across the 10 study facilities, 163 clinical providers completed the training and were certified as competent (Figure 1). Of the trained providers, 72% were female, 85% were nurse-midwives, and 15% were physicians. Annually, the study facilities delivered 385 to 4,388 births, and the ratio of provider to every 100 deliveries ranged from 0.37 at the regional referral hospital to 2.13 at the local health center with the lowest volume of deliveries (Table 1).

Figure 1.  Timing of key steps in observed clinical workflow, over an average 9.5 minutes, for admission to maternity services.
joghr-03-e2019040-f1


Table 1.  Distribution and characteristics of providers trained across study site facilities
Site Providers Cadre*(n=161) Gender(n=158) 2015 Annual deliveries Provider per 100 deliveries
n % of total Nurse/nurse-midwife Physicians Male Female n % of total‡
Regional referral hospital
Bukoba Regional Referral Hospital 16 10% 11 5 8 8 4,300 24% 0.37
District hospitals:
Mugana District Designated Hospital 15 9% 12 3 2 13 1,035 6% 1.45
Nyakahanga District Designated Hospital 28 17% 27 1 2 26 3,959 22% 0.71
Izimbya Hospital 20 12% 16 2 8 10 1,007 6% 1.99
Kagondo Hospital 19 12% 16 3 7 12 1,402 8% 1.36
Ndolage Hospital 12 7% 9 3 7 5 1,078 6% 1.11
Rubya Hospital 14 9% 11 3 3 11 2,665 15% 0.53
Health centers:
Kaigara Health Center 20 12% 19 1 2 17 1,382 8% 1.45
Katoro Health Center 8 5% 7 1 4 4 375 2% 2.13
Kayanga Health Center 11 7% 9 2 1 8 608 3% 1.81
Total 163 100% 137 (85%) 24 (15%) 44 (28%) 114 (72%) 17,811 100% 0.37

*Two providers did not report their cadre.

†Five providers did not report their sex.

‡Percent of all deliveries in study facilities.



Assessment of provider knowledge and skill

Knowledge

The mean score in the pre-training assessment was relatively high at 6.65 on eight questions (mode 7, range 3-8), and the mean score on the post-training assessment was 7.13 on eight questions (mode 8, range 3-8). The change in knowledge from pre-test to post-test was a 7% increase. Mean post-training scores were significantly higher than pre-training scores (7.13, 95% confidence interval (CI)=6.97-7.29) (P<0.001) (Table 2).

Table 2.  Provider performance on training assessments
Population test performance Total (n) Mean 95% CI† Mode Range SD
Pre-training assessment score 160 6.65* 6.46–6.48 7 3–8 1.21
Post-training assessment score 160 7.13* 6.97–7.29 8 3–8 1.05
Difference between pre and post 160 0.48* 0.31-0.65
OSCE skills assessment score 163 13.34 13.08–13.59 15 6–15 1.66

CI – confidence interval, OSCE – observed structured clinical examination, SD – standard deviation

*Significant difference in mean test scores, Paired student t-test P< 0.001.

†95% CI of the mean test score.



Provider OSCE and knowledge assessment performance immediately post-training

After the training session, the mean OSCE score was 13.3 out of 15 scored steps (mode 15, range 6-15) (Table 2). Twelve percent (n=19) of the trainees failed the OSCE on their first try. Those who failed were coached and then reassessed until they attained passing scores.

In addition to categorizing OSCE performance as fail/pass, knowledge assessment scores for the 160 providers were further categorized as low (≤6) or high (7, 8) at the fiftieth percentile of each knowledge assessment’s score (ie, a score of 7). The overall mean difference between pre- and post-training knowledge assessment scores was 0.48 points (95% CI=0.31-0.65, P<0.001) (Table 2). The difference in pre- and post-training knowledge assessment scores did not vary by individual site, level of health care facility, cadre, OSCE score, or provider’s sex. OSCE pass rates were not correlated with individual pre- or post-training knowledge assessment scores (data not shown). There was a significant increase in the proportion of providers attaining higher scores in the post-training knowledge assessment, compared to the pre-training assessment (P<0.05): 82.5% obtained a high pre-training score, and 92% obtained a high post-training score (Table 3). Those who scored high on the pre-training assessment had greater odds of scoring high on the post-training assessment (OR=15.16, 95% CI=4.25-54.11) than those who scored low in the pre-training assessment (Fisher’s exact test, P<0.001) (results not shown).

Table 3.  Provider knowledge and skills assessment outcomes and performance by study facility
OSCE skills assessment (n=163)
Site Change in knowledge (n=160) Pre-training high scores (n=160) Post-training high scores (n=160) Overall Categories
mean (SD) range High (≥7) High (≥7) mean (SD) range Fail Pass
n % site n % site n % site n % site
Regional referral hospital:
Bukoba Regional Referral Hospital 0.8 (1.26) -2, 3 12 75% 14 93% 13.6 (0.96) 12, 15 0 0% 16 100%
District hospitals:
Mugana District Designated Hospital 0.3 (0.72) -1, 2 13 87% 14 93% 13.8 (1.15) 11, 15 1 7% 14 93%
Nyakahanga District Designated Hospital 0.4 (0.93) -2, 2 25 89% 26 96% 14.0 (1.14) 12, 15 0 0% 28 100%
Izimbya Hospital 0.4 (1.14) -2, 2 17 85% 20 100% 14.0 (1.05) 12, 15 0 0% 20 100%
Kagondo Hospital 1.0 (1.33) -2, 3 12 63% 17 89% 12.6 (1.54) 10, 15 5 26% 14 74%
Ndolage Hospital -0.1 (1.31) -3, 2 11 92% 9 75% 12.6 (1.73) 9, 14 3 25% 9 75%
Rubya Hospital 0.1 (0.86) -2, 1 9 64% 10 71% 13.4 (1.60) 10, 15 2 14% 12 86%
Health centers:
Kaigara Health Center 0.4 (0.68) -1, 2 17 85% 19 95% 11.9 (2.73) 5, 8 8 40% 12 60%
Katoro Health Center 1.0 (1.31) -1, 3 7 88% 8 100% 13.1 (0.64) 12, 14 0 0% 8 100%
Kayanga Health Center 0.7 (0.95) 0, 3 9 82% 10 100% 14.2 (1.25) 12, 15 0 0% 11 100%
Total 0.48 (1.07) -3, 3 132 82.5% 147 92% 13.3 (1.66) 6, 15 19 12% 144 88%

OSCE – observed structured clinical examination (OSCE), SD – standard deviation



Observation of admissions to the labor and delivery ward

Out of the 163 providers trained and included in study activities, 87 providers (53% overall, ranging from 33% to 82% of eligible providers across the study sites) were observed using the Doppler to admit 112 women to maternity services within the 6 months following the training. Factors contributing to this relatively low rate of observation included substantial human resource-related constraints on the part of the providers, such as clinical assignment rotation, illness, leave, or facility transfer. Of the 87 providers, 65 were observed only once, 19 were observed for two admissions, and three were observed for three admissions (Table 4). Nearly all (97%) observed providers were either nurses or nurse-midwives (data not shown). Of the 24 physicians trained, three were observed using the Doppler when admitting pregnant women in labor to maternity services. Physicians in these facilities rarely spend substantial time in triage admitting pregnant women. Through 112 provider-client observations, we assessed provider tasks and processes associated with the admission of pregnant women in labor to maternity services (Table 5). In all observed admissions, providers performed an abdominal exam and measured the FHR using the Doppler. In 93% of the admissions, providers checked maternal understanding of the purpose for assessing the FHR with the Doppler. In 9% of the admissions, providers augmented the use of the Doppler with either another handheld Doppler at the facility or a Pinard stethoscope. In the vast majority (96%) of admissions, FHR was recorded in the HMIS register, and the provider initiated a partograph to monitor labor in 85% of the admissions. (The partograph used in Tanzania defines the transition between latent phase and active labor as reaching cervical dilation of 3 cm.) In 82% of the admissions, the provider measured maternal blood pressure; in 63% of the admissions, the woman was assessed for signs of pre-eclampsia; and in roughly one-quarter of the admissions (22%), the provider requested urine and tested it for the presence of protein. In 67% of the admissions, providers asked women about other danger signs, such as heavy vaginal bleeding or intense, sudden onset of abdominal pain.

Table 4.  Number and proportion of trained providers and admissions undergoing observation
Site Trained providers (n=163) Providers observed (n=87) Admissions observed (n=112)
n n % of trained providers observed by site % of total providers observed n (site total) % of total observed admissions
Regional referral hospital:
Bukoba Regional Referral Hospital 16 7 44% 8% 13 11%
District hospitals:
Mugana District Designated Hospital 15 5 33% 6% 7 6%
Nyakahanga District Designated Hospital 28 15 54% 17% 21 18%
Izimbya Hospital 20 6 30% 7% 9 8%
Kagondo Hospital 19 14 74% 16% 19 16%
Ndolage Hospital 12 6 50% 7% 6 5%
Rubya Hospital 14 10 71% 11% 11 9%
Health centers:
Kaigara Health Center 20 10 50% 11% 10 8%
Katoro Health Center 8 5 63% 6% 5 4%
Kayanga Health Center 11 9 82% 10% 11 9%


Table 5.  Workflow processes for admission to labor and delivery services
Process Observed as correct Illustrative qualitative observations
n=112 observations n %
Time woman waits between arrival and admission assessment:
Checks maternal understanding of purpose of device assessment 104 93%
Examines maternal abdomen 112 100%
Measures FHR with study Doppler 112 100%
Measures FHR with other handheld Doppler 4 4%
Measures FHR with Pinard 6 5%
Records FHR in HMIS register 108 96%
Measures maternal blood pressure 92 82%
Assesses for warning signs of pre-eclampsia/eclampsia 71 63%
Assesses other danger signs 75 67%
Requests urine protein test 25 22%
Initiates use of partograph 95 85%
Structural factors for use and storage of Doppler:
Doppler immediately available 110 98% Was locked in the cupboard; was in admission room rather than in triage
If no, observations on why not 2 2%
Storage of Doppler:
Within 1 m of triage admission bed 98 88%
Admission room farther than 1 m from bed 11 10%
Other (describe) 3 3% In admission room; admission room far from triage room; provider had to search for device, which was in the same room but in a congested space
Probe was clean 112 100%
Device battery charged sufficiently 112 100%
Gel present in sufficient quantity 85 76%
Wipes available to clean probe 99 88%
Designated location to charge device in ward 111 99%
Personnel designated to be in charge of device 112 100%
Challenges to Doppler use:
Provider expressed confusion on proper device use 7 6% Comparing maternal pulse with FHR, cleaning the device; poor arrangement of admission area; provider did not turn on device until after placing transducer
Provider made mistake using device 7 6%

CI – confidence interval, FHR – fetal heart rate, HMIS – Health Management Information System, IRB – institutional review board, LMIC – low- and middle-income countries, MCSP – Maternal and Child Survival Program, OSCE – observed structured clinical examination, SD – standard deviation



Structural factors related to Integrating the Doppler into clinical practice

The clinical workflow observation tool also assessed structural factors associated with storage, placement, and use of the Doppler. In 98% of the 112 client-provider observations, the device was immediately available to the provider (Table 5). For the two observations in which the device was not available, the device was either locked in a cupboard or in a different room. For 88% of the admissions, the device was stored within a short distance (<1 m) from the bed used for triage and admission assessment. In all admissions, the device probe was noted to be clean and the battery sufficiently charged. In 99% of the admissions, a designated location was noted for charging the device. An accountable staff member – typically the maternity ward’s in-charge – was designated for storing, charging, and maintaining the device at each study facility. The most notable deficiency was in the availability of the ultrasound gel for the Doppler’s probe: in 24% of the observed admissions, an insufficient amount of ultrasound gel was in stock. In these cases, per study protocol, providers used sterile water or other liquid to ensure that heart sound waves transmitted from the fetus to the device. In 88% of the admissions, wipes were available to clean the probe after its use.

Research assistants provided free-response data for all 112 observed admissions. In 6% of the observed admissions, research assistants noted that providers expressed some confusion or demonstrated process issues in the proper use of the Doppler. These issues included failure to clean the device after use; powering on the device after, rather than before, the transducer was placed on the abdomen; and general poor organization of materials and workspace needed to use the device. In 33% of the observed admissions, qualitative contextual memos provided by the research assistants made particular mention of the shortage or lack of ultrasound gel (n=12 admissions) and provider’s failure to perform an essential component of an adequate maternal health assessment (n=9), ie, focused physical exam, vital sign assessment, standard urine test for protein, and assessment of common obstetric danger signs. Additional issues that the research assistants emphasized included the following: provider needing more practice (n=7); poor arrangement of the assessment area (n=4); improper use of the device or its components (n=3), ie, the probe or the on/off switch; provider’s failure to properly clean device components after use (n=3); lack of use, or failure to initiate use, of indicated partograph (n=2); failure to use normal saline or other liquid when ultrasound gel was lacking (n=1); lack of wipes to clean the device (n=1); and provider failure to sufficiently educate the mother on the purpose and use of the device (n=1).

Timing of clinical workflow steps

Clinical workflow observation data was analyzed for the 75 of the 112 admissions that had complete data. The overall mean length of time for the admission process was 30.6 minutes (with a high of 95 minutes and a low of 8 minutes). The timing of the five categories of the admission process is detailed in Figure 1. Providers used a mean time of 4.1 minutes for fetal assessment and device usage. This step included locating, powering on, and positioning the device; obtaining a measurement of the FHR with the device and additional devices as preferred; answering the woman’s questions; and cleaning and storing the device. Intake and booking required the longest time; this step included the time a woman spent in the waiting room before the admission process began.

DISCUSSION

In Tanzania and globally, successful efforts to reduce rates of stillbirth have long recognized the importance of early detection of fetal distress to improve outcomes (14). A study conducted close to three decades ago in the Southern Highlands of Tanzania focused on early detection of fetal distress as its key intervention, demonstrating close to a 50% reduction in perinatal mortality (33). Despite the established importance of enabling providers to detect fetal distress early and take action, perinatal mortality continues to be high. A significant challenge facing Tanzanian health care providers and public health planners lies in being able to accurately map the inputs and processes needed to scale up new interventions, such as integrating Doppler use into the routine triage and admission workflow. Multiple studies have looked at the use of Doppler to auscultate FHRs upon admission or intermittently throughout labor in LMICs (including in Zimbabwe (34), Uganda (19), and Tanzania (15, 16)). Recent studies of the Moyo Doppler in Tanzania have demonstrated its effectiveness in detecting an abnormal FHR (18), its superior performance compared to Pinard (16), and its ability to improve FHR documentation (17).

The current study described inputs and processes involved in integrating Doppler use into the workflow of maternal triage upon admission to maternity wards. We showed a change in provider knowledge and skills before and after the study training, and we identified a number of inputs that merit attention and potential improvement to sustain and scale up the use of Doppler. Our assessment of clinical workflow provides insight to better understand the nature and sequence of processes that providers need to follow in order to sustain the adoption of Doppler use for triage and admission.

Implications for scale-up of training on Doppler for maternity triage and admission

We found that providers’ knowledge of Doppler use, which was quite high at baseline, showed a statistically significant increase of 7% after their participation in the half-day training session. The majority of providers (88%) also demonstrated rapid skill uptake in using the Doppler, thereby demonstrating competency on their first try. The strong baseline knowledge (mean of 6.65 out of 8) as well as relatively high post-training scores bodes well for the scale-up of training to incorporate the use of Doppler into admission workflow. In comparison, in 2015, 279 Tanzanian health care providers similarly assessed with OSCEs to demonstrate competency in Helping Babies Breathe had a passing rate of 63% (35).

Whereas many studies on the Doppler address intermittent monitoring throughout labor, the training assessed in this study restricted its focus on Doppler use to triage pregnant women in labor for admission to maternity services. In assessing the use of the Doppler, we found that providers omitted some of the other key steps for admission, such as initiating partograph use and assessing for signs of pre-eclampsia and other obstetric emergencies. This suggests that the training materials should be expanded to include a refresher on all key steps of triaging women in labor for admission to maternity services. Additionally, the observation that a small proportion of providers augmented Doppler assessment with Pinard suggests that a wide-scale training effort may benefit from activities that reinforce adoption of a new device. This is further reinforced by a study in Tanzania, which found that some midwives who used both Doppler and Pinard preferred the device with which they were most comfortable (15). This study found that midwives did not necessarily prefer using the Doppler over Pinard, citing unfamiliarity with the Doppler and the need for charging and maintenance as potential barriers to use. The authors concluded that Doppler devices should be made available during midwifery and in-service training, and that both Pinard stethoscopes and Doppler devices should be available to practitioners (15). Our findings build upon these previous conclusions and provide supportive evidence for revising both pre-service and in-service training curricula to provide skills and knowledge on use of the Doppler to support clinical care in maternity wards (36).

The structural factors (37) assessed in association with the study’s clinical workflow showed that health care providers were generally able to properly store, charge, maintain, and use the Doppler, and that material and other resource inputs were not a substantial barrier to integrating Doppler use for these providers. Structural factors with the lowest scores were the availability of the gel (76%) and wipes to clean the probe (88%). The difficulties noted with maintaining an adequate supply of ultrasound gel are not surprising in LMIC settings but suggest that facilities should use existing budgeting and procurement measures to prevent stock-outs. In addition, the small but notable number of times that the device was locked in a cupboard or not available in the triage work area suggest that an application of standard operating procedures may be useful.

Time spent using the Doppler

The time required to assess fetal status, including Doppler use, averaged 4.1 minutes. Although we did not collect data on the time needed for fetal assessment using a Pinard stethoscope, it is not unreasonable to assume that the devices have similar time requirements. From the authors’ collective field and clinical experiences, the 4.1 minutes does not represent an unreasonable burden on provider time, and may decrease as providers use the Doppler outside the context of formal observation and study. There is no standard recommendation on the amount of time to spend on each clinical workflow step needed to triage and admit women in labor to maternity services: the recommendation calls for the provider to identify actionable risk factors, regardless of the amount of time spent (38). However, the realities of human resource challenges and crowding in Tanzanian maternity wards require that admissions are conducted as rapidly as possible. A Doppler assessment of a healthy fetus can take under a minute. Notably, our measure included a variety of associated steps that started when the provider located the Doppler, applied the gel, and all subsequent tasks, including storing the device after use. However, our finding of 4.1 minutes as the average time does offer opportunity for improved efficiency.

Potential applications of fetal heart rate data for quality improvement

Ensuring high quality of recorded data is recommended for improving processes of care (39). For the parent study associated with the current study, HMIS registers were modified to provide a field to record FHR upon triage and admission to maternity wards. We believe this modification improved not only the practice of recording FHRs but also a provider’s overall practice of assessing FHRs. If Doppler use during women’s admission to labor and delivery wards is scaled up across wider regions of Tanzania, it will be critical to standardize similar revisions to HMIS tools to accommodate proper recording of FHR data. Inclusion of a field for measuring an FHR upon triage and admission will have the additional benefit of allowing facilities to calculate and track a validated indicator of facility-based perinatal mortality, which has the potential to track improvements in quality of labor, delivery, and intrapartum care (22).

Limitations of the study

It is important to consider the findings of our study in the context of several limitations to our approach. Because the study training sessions were conducted during facility operating hours, and only 1 day was allotted per facility for the training, the full active roster of maternity ward providers was not available to participate in the study. Some risk of bias is inherent in our use of the same team to conduct the training as well as to assess knowledge and skills transfer. However, use of structured and validated assessment methodologies (OSCE and multiple choice questions) as well as the experience of the trainers in medical, nursing and midwifery education offsets the potential bias. Further emphasis on knowledge and skill retention could be achieved using on-the-job training supported by continued mentoring and by creating explicit connections to efforts to improve intrapartum quality of care (36, 40). Training and reinforcement of clinical decision-making in response to abnormal fetal assessment was outside the scope of this study but should be attached to future efforts to integrate Doppler to improve the quality of intrapartum care.

Although all of the providers who participated in the observation component at 6 months following the training session were using the Doppler in practice, the study design was such that only a half of the trained providers were observed. This measure does not allow us to describe the extent to which Doppler was used in admission for all of the clients admitted to study facilities. This is a limitation of the study. The study design does not allow us to comment on retention of the study skills and knowledge in clinical practice at the time of follow-up, but this is an important operational consideration for future scale-up efforts. Some were not observed due to illness or leave, and these factors could not be predicted. However, in some cases, the researchers and facility administration could have screened out providers who do not often conduct admissions (in the case of physicians) or who had planned transfers to other departments. Some providers were observed multiple times, and although we chose to include this small number of multiple observations, doing so may have introduced some bias to our findings. Facilitators of future trainings should screen providers to ensure that the persons who need to learn the new skill are able to participate in the training and subsequent assessments, and should further standardize the number of observations conducted per provider.

This study was not comparative in design, and data from the pre-intervention workflow were not collected. Furthermore, we do not have data to compare time spent using the Doppler against the time spent using the Pinard stethoscope. Comparative studies that document clinical management of the woman and fetus, following detection of abnormal FHRs, will elucidate the pathway for improving perinatal survival. Although institutional deliveries are increasing in Tanzania, in some regions, close to half of the deliveries take place at home (3). This study did not address the potential of the Doppler to improve community-based care in home-based deliveries. Additionally, the Kagera Region of Tanzania selected for this study has long experienced poorer than average health outcomes. This may limit generalizability of our findings to settings with ample resources and high performance on health indicators. Finally, we collected provider rosters during study planning, but because of staffing fluctuations and vagueness in “who is a provider at this hospital” (ie, some semiretired, some part-time), we cannot accurately specify the proportion of health care providers working in the maternity ward who were trained in our intervention.

CONCLUSION

Improving the quality of maternal and newborn services to prevent intrapartum stillbirths and newborn deaths is a pressing concern in Tanzania, as in many other LMIC settings. Our study assessed the effectiveness of provider training sessions along with the inputs needed to integrate the Doppler to assess fetal heart status during maternal triage upon admission to labor and delivery wards. We found that the training sessions effectively supported transfer of knowledge and skills that providers need to integrate this device into routine care during facility admission, and that it is likely that minimal resource inputs will be required to maintain use of the device in practice. Further applications of these findings should assess determinants and levels of provider acceptance and comfort using the Doppler and look into institutionalizing Doppler use for tracking perinatal deaths that take place after the woman is admitted to maternity services.

Acknowledgments

The authors would like to acknowledge the health care providers in the study facilities, without whom this study would not have been possible, as well as the support of the regional health authorities of Kagera Region. Many thanks to Benny Ngereza and Ruth Lemwayi for their support of the study. This paper was reviewed by Rachel Haws and Alain Labrique, and edited by Vipra Ghimire.

Ethics approval: This study was submitted to the Johns Hopkins Bloomberg School of Public Health Institutional Review Board (IRB) and was determined to be exempt (JHU IRB #00007059). The study was approved by the Tanzania National Institute of Medical Research (NIMR) IRB (NIMR/HQ/R.8a/vol.IX/2219). Health care providers participating in the study signed a written informed consent statement, allowing themselves to be observed conducting facility admissions. Oral assent was provided by all mothers whose admissions were observed.

Availability of data and materials: The data are available from the corresponding author (Mary Carol Jennings, (marycaroljennings@jhu.edu) upon reasonable request, subject to obtaining clearance from the Medical Research Coordinating Committee (MRCC) of the National Institute of Medical Research of Tanzania. For information on how to obtain clearance to access the data, contact the MRCC at 255-22-2121400 or headquarters@nimr.or.tz.Funding: This study is made possible by the generous support of the American people through the United States Agency for International Development (USAID), under the terms of the Cooperative Agreement AID-OAA-A-14-00028. The contents are the responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Notes

[1] Conflicts of interest Competing interests: The authors completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available upon request from the corresponding author), and declare no conflicts of interest.

References

1L Liu, S Oza, D Hogan, Y Chu, J Perin, and J Zhu. Global, regional, and national causes of under-5 mortality in 2000–15: an updated systematic analysis with implications for the Sustainable Development Goals. Lancet. 2016;388(10063):p.3027–35. DOI: 10.1016/S0140-6736(16)31593-8. [27839855]

2World Health Organization. Every Newborn: an action plan to end preventable deaths. June 2014. Available: http://www.healthynewbornnetwork.org/hnn-content/uploads/Every_Newborn_Action_Plan-ENGLISH_updated_July2014.pdf. Accessed: 1 May 2019.

3Ministry of Health Zanzibar. National Bureau of Statistics Dar es Salaam, Office of Chief Government Statitician Zanzibar. Tanzania Demographic and Health Survey and Malaria Indicator Survey (TDHS-MIS) 2015-16. Dar es Salaam, Tanzania, Rockville, MD. 2016.

4UN Inter-Agency Group for Child Mortality Estimation. Levels & Trends in Child Mortality. 2017. Available: https://childmortality.org/wp-content/uploads/2018/10/UN-IGME-Child-Mortality-Report-2017.pdf. Accessed: 1 May 2019.

5AH Baqui, DK Mitra, N Begum, L Hurt, S Soremekun, and K Edmond. Neonatal mortality within 24 hours of birth in six low- and lower-middle-income countries. Bull World Health Organ. 2016;94:p.752–758B. DOI: 10.2471/BLT.15.160945. [27843165]

6UNICEF. Maternal and Newborn Health Disparities: Tanzania. 2016. Available: https://data.unicef.org/wp-content/uploads/country_profiles/United Republic of Tanzania/country profile_TZA.pdf. Accessed: 1 May 2019.

7ZA Bhutta, MY Yakoob, JE Lawn, A Rizvi, IK Friberg, and E Weissman. Stillbirths: what difference can we make and at what cost? Lancet (London, England). 2011 Apr 30;377(9776):p.1523–38. DOI: 10.1016/S0140-6736(10)62269-6. [21496906]

8KE Dickson, A Simen-Kapeu, MV Kinney, L Huicho, L Vesel, and E Lackritz. Every Newborn: health-systems bottlenecks and strategies to accelerate scale-up in countries. Lancet. 2014;384:p.438–54. DOI: 10.1016/S0140-6736(14)60582-1. [24853600]

9B Madaj, H Smith, M Mathai, N Roos, and N van den Broek. . Developing global indicators for quality of maternal and newborn care: a feasibility assessment. Bull World Health Organ. 2017;95:p.445–452I. DOI: 10.2471/BLT.16.179531. [28603311]

10AJ Milat, L King, AE Bauman, and S Redman. . The concept of scalability: increasing the scale and potential adoption of health promotion interventions into policy and practice. Health Promot Int. 2013;28:p.285–98. DOI: 10.1093/heapro/dar097. [22241853]

11World Health Organization. Intrapartum care for a positive childbirth experience. 2018. Available: http://apps.who.int/iris/bitstream/10665/260178/1/9789241550215-eng.pdf?ua=1%0Ahttp://www.who.int/reproductivehealth/publications/intrapartum-care-guidelines/en/. Accessed: 1 May 2019.

12Ministry of Health and Social Welfare. The National Road Map Strategic Plan To Accelerate Reduction of Maternal, Newborn and Child Deaths in Tanzania. 2005. Available: http://www.who.int/pmnch/countries/tanzaniamapstrategic.pdf. Accessed: 29 August 2018.

13D Devane, JG Lalor, S Daly, W McGuire, and ASV Cuthbert. . Cardiotocography versus intermittent auscultation of fetal heart on admission to labour ward for assessment of fetal wellbeing. Cochrane Database Syst Rev. 2017;1:CD005122.[28125772]

14HL Kidanto, I Mogren, J van Roosmalen, AN Thomas, SN Massawe, and L Nystrom. Introduction of a qualitative perinatal audit at Muhimbili national hospital, Dar es Salaam, Tanzania. BMC Pregnancy Childbirth. 2009;9:p.45 . DOI: 10.1186/1471-2393-9-45. [19765312]

15PF Mdoe, HL Ersdal, E Mduma, R Moshiro, H Kidanto, and C Mbekenga. . Midwives’ perceptions on using a fetoscope and Doppler for fetal heart rate assessments during labor: A qualitative study in rural Tanzania. BMC Pregnancy Childbirth. 2018;18:p.103 . DOI: 10.1186/s12884-018-1736-y. [29661165]

16BA Kamala, HL Kidanto, PJ Wangwe, I Dalen, ER Mduma, and JM Perlman. Intrapartum fetal heart rate monitoring using a handheld Doppler versus Pinard stethoscope: a randomized controlled study in Dar es Salaam. Int J Womens Health. 2018;10:p.341–8. DOI: 10.2147/IJWH.S160675. [30022861]

17BA Kamala, HL Ersdal, I Dalen, MS Abeid, MM Ngarina, and JM Perlman. Implementation of a novel continuous fetal Doppler (Moyo) improves quality of intrapartum fetal heart rate monitoring in a resource-limited tertiary hospital in Tanzania: An observational study. PLoS One. 2018;13:e0205698. DOI: 10.1371/journal.pone.0205698. [30308040]

18B Kamala, H Kidanto, I Dalen, M Ngarina, M Abeid, and J Perlman. Effectiveness of a Novel Continuous Doppler (Moyo) Versus Intermittent Doppler in intrapartum detection of abnormal foetal heart rate: A randomised controlled study in Tanzania. Int J Environ Res Public Health. 2019;16:p.315 . DOI: 10.3390/ijerph16030315. [30678354]

19R Byaruhanga, DG Bassani, A Jagau, P Muwanguzi, AL Montgomery, and JE Lawn. . Use of wind-up fetal Doppler versus Pinard for fetal heart rate intermittent monitoring in labour: A randomised clinical trial. BMJ Open. 2015;5:e006867. DOI: 10.1136/bmjopen-2014-006867. [25636792]

20L Mangesi, G Hofmeyr, and D Woods. . Assessing the preference of women for different methods of monitoring the fetal heart in labour. S Afr J Obstet Gynaecol. 2009;15:p.58–9.

21S Rivenes Lafontan, J Sundby, HL Ersdal, M Abeid, HL Kidanto, and CK Mbekenga. . “I was relieved to know that my baby was safe”: Women’s attitudes and perceptions on using a new electronic fetal heart rate monitor during labor in Tanzania. Int J Environ Res Public Health. 2018;15:E302. DOI: 10.3390/ijerph15020302. [29425167]

22M Plotkin, D Bishanga, H Kidanto, MC Jennings, J Ricca, and A Mwanamsangu. Tracking facility-based perinatal deaths in Tanzania: Results from an indicator validation assessment. PLoS One. 2018;13:e0201238. DOI: 10.1371/journal.pone.0201238. [30052662]

23National Bureau of Health Statistics and Office of Chief Governmental. The United Republic of Tanzania Mortality and Health. Census 2012. Dar es Salam; 2012.

24World Health Organization. The WHO application of ICD-10 to deaths during the perinatal period. 2016. Available: http://apps.who.int/iris/bitstream/handle/10665/249515/9789241549752-eng.pdf;jsessionid=A8CA8A7309D15BC1072A06FA6951371D?sequence=1. Accessed: 1 May 2019.

25World Health Organization. Maternal Death Surveillance and Response Technical Guidance. 2013. Available: http://apps.who.int/iris/bitstream/10665/87340/1/9789241506083_eng.pdf?ua=1. Accessed: 1 May 2019.

26Laerdal Global Health. Moyo fetal heart rate monitor user guide. 2014.

27Lean Six Sigma. 2016 [cited 2018 Aug 29]. Available from: https://www.6sigma.us/

28The United Republic of Tanzania Ministry of Health and Social Welfare. Learning Resource Package for Comprehensive Emergency Obstetric and Newborn Care (CEmONC). 2015.

29E Proctor, H Silmere, R Raghavan, P Hovmand, G Aarons, and A Bunger. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm Policy Ment Health. 2011;38:p.65–76. DOI: 10.1007/s10488-010-0319-7. [20957426]

30M Futamura, M Saito, J Batchelor, M Nakahara, and T Nakahara. . Furue M OY. The Japanese version of Topicop scale among patients with atopic dermatitis: A translation and feasibility study. Arerugi. 2016;65:p.66–72.[26923656]

31StataCorp. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP; 2015.

32CommCareHQ. 2016. Available: https://www.dimagi.com/commcare/. Accessed: 29 August 2018.

33J van Roosmalen. . Perinatal mortality in rural Tanzania. Br J Obstet Gynaecol. 1989;96:p.827–34. DOI: 10.1111/j.1471-0528.1989.tb03323.x. [2765428]

34K Mahomed, R Nyoni, T Mulambo, J Kasule, and E Jacobus. . Randomised controlled trial of intrapartum fetal heart rate monitoring. BMJ. 1994;308:p.497–500. DOI: 10.1136/bmj.308.6927.497. [8136665]

35J Reisman, N Martineau, A Kairuki, V Mponzi, AR Meda, and KG Isangula. Validation of a novel tool for assessing newborn resuscitation skills among birth attendants trained by the Helping Babies Breathe program. Int J Gynaecol Obstet. 2015;131:p.196–200. DOI: 10.1016/j.ijgo.2015.05.019. [26283225]

36J Bluestone, P Johnson, J Fullerton, C Carr, and J Alderman. . BonTempo J. Effective in-service training design and delivery: Evidence from an integrative literature review. Hum Resour Health. 2013;11:p.51 . DOI: 10.1186/1478-4491-11-51. [24083659]

37C Siriwardhana, A Adikari, K Jayaweera, and B Abeyrathna. . Integrating mental health into primary care for post - conflict populations : a pilot study. Int J Ment Health Syst. 2016;10:p.12 . DOI: 10.1186/s13033-016-0046-x. [26925160]

38National Institute for Health and Care Excellence (NICE) UK Department of Health. Intrapartum care for healthy women and babies, Clinical guideline [CG190]. 2017. Available: https://www.nice.org.uk/guidance/cg190/chapter/Recommendations#initial-assessment. Accessed: 17 September 2018.

39M Bing, RL Abel, P Pendergrass, K Sabharwal, and C McCauley. . Data used to improve quality of health care. Tex Med. 2000;96:p.75–9.[11070739]

40P Schwerdtle, J Morphet, and H Hall. . A scoping review of mentorship of health personnel to improve the quality of health care in low and middle-income countries. Global Health. 2017;13:p.77 . DOI: 10.1186/s12992-017-0301-1. [28974233]