2.9 Literature review
Literature review is defined as the collection, synthesis, and evaluation of published studies. Published studies could be primary, secondary, and tertiary. Primary literature review makes use of materials encompassing empirical studies – ranging from observation, interviews, to experimental and is meant to provide an assessment of what has been published on a topic of interest mainly of actual practices. The secondary literature derived from the exposition of the primary literature sources are generally reviewed studies. These studies could be narrative, systematic, semi-systematic and integrative. Authors of these studies integrate primary literature studies through synthesis, thus, further expanding on the findings of primary literature studies. The outcomes of this type of literature review have the potential to be generalizable. The tertiary source encompasses the distillation and collection of materials derived from the hybrid of primary and secondary literature sources, for example, textbooks, guidebooks, encyclopedia. Thus, the purpose of the tertiary literature is to offer an impression of important findings of existing studies or research. One of the importance of a tertiary literature review is its ability to introduce principles to practices in a discipline. By far this type of literature review is the least used one in academia. Literature can either be narrative or systematic. Narrative review is the conventional and the oldest means of studying extant studies and is generally qualitative with no particular “formal” guide for undertaking it. As a result, it does not generally seek generalization. Below, we provide an overview of systematic literature review.
Literature review in the medical field has largely been systematic and meta-analysis providing a quantitative evidence for reliable conclusion and generalizability (Tranfield et al., 2003). Narrative literature review has been used extensively in the non-medical field to provide basis for further exploration of multi-facet and ongoing changes of situations and a timely response to the experiences of people/subjects and situations. Whilst not generally generalizable, it serves a great purpose for providing immediate overview/understanding of existing and new phenomena using varied methodologies including socio-cultural, natural and library and processes in the context of data collection and conception, data analysis, and data reporting (McAlpine, 2016). It is interesting how researchers can draw conclusion from a narrative literature study given unstructured nature. There must be some assurance that the study is comprehensive and has covered prominent research of the given time. The selection of and critically analysing large set of literature from varying sources is appearing to be relevant to produce generalizable results.
Often, and generally, SLR studies selection criteria have been one that is based on published empirical materials. A concern with this criterion emanates from the fact that “there is a growing recognition that often evidence is difficult to find because of decisions that are made about where, how, and when to publish the results of studies based on the findings of those studies” (Balshem et al., 2013, p.1). This makes room for undetected biases in publish materials. One of such sources of materials is the grey literature. Grey literature is literature from unpublished reports from government and others such as dissertation that is now being touted as becoming an important additional source of literature towards evidence-based studies. The Institute of Medicine and the Agency for Healthcare Research and Quality (AHRQ) and current Cochrane guidance made recommendations to this effect. Generally, useful interventions are published far more than interventions that produced negative poor outcomes. Whilst the inclusion of grey literature is being encouraged, the impact of their exclusion has been found to be relatively small or negligible. Given both technical (structured) and non-technical (abstract) nature of IS, the careful collation and consolidation of materials from grey literature and unpublished dissertation and theses will be useful. Like in management this will help respond directly and timely to changing practice and policy needs. Whilst majority of SLR studies searched for non-English and unpublished papers, less than 5% of these studies are included for full review (Hartling et al., 2017).
As research continue to grow and information explodes in the discipline of IS, literature reviews have increasingly become crucial in the definition and understanding of IS for academics, practitioners, and policymakers alike.
Overview of systematic literature review
In the medical and public health fields, SLR has been hailed as a gold standard methodology for evidence-based research since the establishment of Cochrane Collaboration in 1993. In the discipline of IS, a rigorous systematic literature review study would be critical to collate and summarise evidence on how IS technologies, specifically digital health technologies are improving healthcare delivery and outcomes. Currently, digital health technologies are generally known for their “potential”.
SLR approach is already in use in the IS research. SLR requires thought, probing of concepts, interrogation of assumptions and expectations, and a considered appraisal of where a SLR might fit into the general scheme of knowledge and keep expanding in the domain of research that apply it. Can machines be trained to apply the criteria of inclusion and exclusion? Can machines be trained on how to apply assessment of bias? (Gough, Oliver, & Thomas, 2017). Theory-based SLRs, which summarise evidence on what works, when and why, strive for more than greater policy relevance. In fact, they strive for premise for decision-making by top management in practice. For example, in the assessment of the evaluation process of MIS products, King and Rodriguez (1978) categorised assessments into four categories – attitudes, value perceptions, information usage, and decision performance. Holistically, each of these categories requires a systematic evaluation that is exhaustive of existing literature to gather evidence on these categories to avoid the pitfalls of common scientific basis, fallacy of affirming the consequent, summative validity and called for basic reasoning that accommodates the latest developments in positivist, interpretive, action, and design research (Lee & Hubona, 2009). This ultimately leads to technology adoption mindfulness in an era populated with mundane and bleeding edge technologies. This is the case for digital health technologies. These technologies still lack rigorous evaluations as with methods used in evaluating them to provide sufficient/reliable evidence on their functionalities and the most suitable circumstances in which they will be more beneficial. Reviews that answer these questions adopt a mixed methods approach and draw on a range of study types. Answering the ‘what works’ and ‘what doesn’t’ questions mean drawing on effectiveness studies, conducted to standards of high-quality impact evaluation. But in formulating answers to the ‘when’ and ‘why’ questions require a broader range of evidence from both quantitative and qualitative research (Snilstveit, 2012) for policy relevance. Ideally, this would normally be theory-based SLRs, which are usually mixed method-based reviews (White, 2018). According to White, mixed methods review is one which draws on a variety of evidence, factual and counterfactual, qualitative and quantitative, to address different questions along the causal chain (p.1). Drawing on different types of studies theory-based SLRs are policy-focused (Snilstveit, 2012). Both theory-based SLR and health technology assessment (HTA) are all theory-focused evaluations.
Let us look at the concept of Health Technology Assessment (HTA), which is the application of organized knowledge and skills in the form of devices, medicines, vaccines, procedures, and systems developed to solve a health problem and improve quality of lives” given the rampant medical errors and system inefficiencies during the pre-digitization. The World Bank also defined HTA as a complete policy-focused research that evaluates short and long-term impact from the application operationalization of technology, which include but not limited to benefits, costs, risks (World Bank, 1995), and accessibility/availability. With advanced health information technologies such as electronic health/medical records and many more the domain of health care has witnessed improvement in every facet of its management and administration. From improved communications between healthcare provider and consumer to improved medication safety, tracking, and reporting; and promoting quality of care through optimized access to and adherence to guidelines” (American College of Obstetricians and Gynecologists, 2015, p.1). Health technologies can range from medicine and medical devices, and pharmaceuticals as with computer-aided information systems such as electronic healthcare records (EHR) systems, health information exchange (HIE), personal health records (PHR), national information networks, 3D printing, Artificial intelligence (AI), etc are examples of computer- supported IS. Again, a revisit to the definition of IS provides an understanding of the social-technical aspect of IS vis-à-vis people and their roles to accomplish task using technology. The challenge remains with the availability of limited contribution of high-quality evaluations resulting in ungeneralizable reviews in the context of policy and practice implications, necessitating the need for further rigorous studies. Whilst substantial progress has been made, however, there challenges with the wider adoption of health information technologies given the lack of concrete evidence to answer ‘when’ and ‘why’ questions in the adoption of these technologies, costs and risks associated with such technologies would need a structured-procedure/process-based evaluative methodology with fidelity. These requirements fit well into SLR.
There are not many studies that have provided guidance for conducting SLR in general. Existing ones include (Okoli & Schabram, 2010; Onwuegbuzie & Frels, 2016; Tranfield et al., 2003). The guide/steps recommended by these authors are summarised in the Table 2 below.