Evaluating High-tech Health Approaches in Low-income Countries

Pesinet, a health program operating in Bamako, Mali, is harnessing new technologies to address malnutrition, which accounts for more than 50 percent of child deaths. Every week, Pesinet health workers visit enrolled children and collect health data on their mobile phones. It then sends this data to a web interface at a health facility for review by a physician, who determines which children are at risk and need a check up. Patients using this low-cost approach are more than twice as likely to seek care when they are sick, and their mortality is much lower than the local average.

The rapid growth of information and communications technologies (ICTs) in low- and middle-income countries has created great interest in their application to health care. Our research group, Toronto Health Organization Performance Evaluation (T-HOPE) team, recently published a Stanford Social Innovation Review article, “The Future of Health Care Access,” that explores the use of ICTs in improving access to care, and describes how organizations are using ICTs to educate and empower both patients and providers. Since then, we have been looking at promising approaches in three health areas: maternal and child health, malaria, and tuberculosis (TB).

While the potential for ICTs to improve health care delivery is immense, there is currently little evidence of their impact, particularly in developing countries. A recent World Health Organization survey found that while 83 percent of its member states offer at least one type of mHealth service (health interventions using mobile technologies), only 12 percent report evaluating them. A 2013 systematic review of mHealth interventions found modest benefits, but we need much more robust evaluations of clinically important outcomes to go beyond the hype.

Last year, we worked with the Center for Health Market Innovations (CHMI) to look at more than 350 health innovations in the areas above. In the process, we identified models that had some evidence of benefit, and others that lacked formal evidence but seemed promising.

Innovations that use ICTs in this area target patients, practitioners, and administrators to improve access to and quality of care, or better manage health system resources. Here are a few snapshots of our findings:

ICT Tools for Patients

Several health programs use ICTs to help facilitate payments and deliver reminders. For example, in Kenya, Changamka Microhealth Limited provides a maternity smartcard that patients can top off by mobile phone; the platform includes a dedicated savings mechanism to access antenatal, maternity, and post-natal services at participating facilities. This system allows patients to add money whenever it is convenient and in whatever increments they want. This allows them to save at their own pace and pay for treatments without needing to carry cash. Since hospitals started accepting the Changamka cards, regular check-ups by women have increased 30 percent.

Other programs remind patients to attend appointments and take medications through mobile phone text messages. In South Africa, On Cue Compliance uses specially designed pill bottles to encourage TB drug adherence. When the patient opens a pill bottle, a device in the lid sends an SMS message to a designated healthcare worker; if a patient does not open a pill bottle at the scheduled time, she receives a reminder to take the medication. This approach has yielded a 90 percent compliance with treatment, compared to the previous average of 60 percent or less.

ICT Tools for Health Care Practitioners

Tools for health practitioners include electronic medical records (EMRs), which help providers access and manage patient data. In our review of CHMI programs and relevant literature, this was the only ICT approach that showed evidence of impact in these health areas. Studies show that EMRs decrease lab reporting delays and improve patient tracking, reduce data entry errors and TB lab test reporting errors, reduce health care provider workload, and save patients and providers time.

Programs are also using mobile phones to connect remote health workers with specialized providers for advice and coordination of care. In Indonesia, the Aceh Behar Midwives with Mobile Phone project connected midwives with obstetrician-gynecologists, increasing the capacity of the midwives and reducing response time.

New mobile technologies are also transforming cell phones into diagnostic tools. D-Tree International provides clinical decision support software on mobile phones that guides health workers through the screening, examination, and treatment process, and helps them make appropriate decisions regarding care. Meanwhile, programs are adding applications and accessory attachments to mobile phones that turn them into diagnostic tools, such as a low-cost stethoscope made by attaching an eggcup to a mobile phone. And in the case of Health Care at my Fingertips in Kenya, programs are using wireless e-health tablets to take high-resolution photographs and perform diagnostic evaluations.

ICT Tools for Health Care Administrators

Health administrators can increasingly use ICTs to understand the health needs of populations they serve and optimally allocate scarce resources. Programs such as uNotify in India and Health at Home/Kenya use mobile devices with customized software and GPS to collect data and track patients, This approach provides information on the incidence of malaria, HIV, and TB that is more timely and less labor intensive than paper-based approaches.

Also, programs such as Uganda Health Information Network and SMS for Life in Tanzania are using mobile technologies to collect information on drug usage and stock; they send electronic messages to administrators, as drugs need refilling. This promising approach can reduce stock-outs of vital medications, which helps ensure effective medication management and reliable drug supply—essential for high-quality treatment of TB and malaria.

While all of these approaches have a great deal of potential, many are still in the pilot stage, and their impact remains small-scale and difficult to assess in complex and changing contexts. A program such as M-Vaccine that sends reminders to parents encouraging them to vaccinate their children may be linked to higher vaccination rates, but it can be challenging to separate this out from other factors, such as changing societal norms regarding vaccination, greater access to affordable vaccines, and other incentive programs. And while efforts to assess new health technologies are increasing, most of the evidence is from high-income countries. A systematic review of mHealth interventions in pregnancy found increases in smoking cessation, confidence, and self-efficacy, but not on maternal outcomes. Extrapolating these findings to low-income settings is difficult because baseline health outcomes are worse, and the potential for improvement is greater. Low-income settings have fewer resources for evaluation and less money to waste on ineffective services.

Spending evaluation dollars wisely requires that we focus on what is credible, feasible, and comparable. The Global Impact Investment Network’s Impact Reporting and Investment Standards’ health working group has developed a core set of metrics that are comparable, but they focus on only a small number of process measures pertinent to clinics and hospitals. CHMI and T-HOPE’s Reported Results framework identifies comprehensive, credible, and relatively feasible measures that can describe program impact more broadly, and determine strategic tradeoffs between quality, cost, and accessibility. The IRIS metrics could be used to scan across a range of organizations to see what is promising, while the Reported Results framework can structure in-depth analyses of a specific organization. Evaluation strategies like these will help us understand how organizations use these technologies to address health problems and manage ICT implementation issues. These include incentives for front-line staff and remote medical experts to collaborate on patient care; linking advances in diagnostic technologies with the availability of necessary treatments; and integration of ICTs with existing back-end medical processes. More robust evaluation of new health technologies could help address these issues and develop game-changing uses of ICTs in global health.

Read more stories by Kate Mossman, Onil Bhattacharyya, Anita McGahan & Will Mitchell.