INTRODUCTION
There is a growing body of nursing literature about the merits of using integrative reviews (IR) as a research approach, and the methodological steps involved (Dhollande et al., 2021; Whittemore & Knafl, 2005). We note anecdotally the increasing use of integrative reviews in postgraduate Masterate level nursing programmes. The use of these reviews is likely because they are a legitimate research project that is suited for the small research component in clinical Masterate programmes where a full thesis is not required. These projects can be undertaken across one or two semesters and introduce students to each step of the research process, enabling them to become producers of research evidence. Integrative reviews also have the pragmatic benefit for supervisors and course planners as the timeline is much more predictable and contained than primary empirical studies involving an ethics application and human subject data collection.
Although there is a range of review types, IRs are particularly pertinent for Masterate nursing students as they can comprise a broad range of sources, including qualitative empirical studies and theoretical and grey literature (Whittemore & Knafl, 2005). This range of sources is often suited to enculturating Graduate Entry Nurses (GEN) into the holistic focus of nursing research questions and aims. Integrative reviews encompass literature, such as qualitative and non-experimental empirical data exploring lived experience beyond a narrow biomedical lens (da Silva et al., 2020; Hopia et al., 2016; Torraco, 2005). Alternative methods such as systematic reviews and meta-analyses commonly exclude nursing research. The emphasis of exclusively quantitative review types is on positivist conceptualisations of rigour, in particular, the hierarchy of evidence, which pertains only to quantitative studies. These approaches privilege randomised controlled trials; a method that often cannot be ethically justified to answer nursing research questions (da Silva et al., 2020).
While there is a growing body of literature providing guidance around the steps in the IR research process, this article makes a novel contribution to field. The authors are three academics involved in student supervision, examination, and moderation of the IRs, and they also mentor colleagues through the process of supervising these projects. In this article, we provide additional advice about each step of the IR. We highlight the common pitfalls for novice researchers and conceptual and methodological ‘red flags’ for supervisors to heed early to ensure projects are rigorous and publishable. We focus on the exemplar of students in a Graduate Entry Nursing (GEN) programme to illustrate these points. Despite the apparent clarity of the IR steps in the extant literature, we agree with Oermann and Knafl (2021) that expert guidance is required with synthesis research. This discussion paper thus highlights recurring problems that require vigilant supervisory oversight to ensure timely remediation to foster student success.
BACKGROUND
Graduate entry nursing (GEN) programmes offer an accelerated pathway to becoming a registered nurse and have been operating globally for many years (Macdiarmid, Winnington, et al., 2021; Neill, 2011). Often appealing to mature students (Downey & Asselin, 2015), GEN courses build on previous undergraduate success, to enable students to simultaneously gain a professional qualification alongside a higher degree (usually a Masters) (Downey & Asselin, 2015). Yet, while such accelerated programmes dualistically meet the demands of this student cohort alongside providing more nurses for the workforce, these study programmes pose challenges for both students and academics alike.
As part of the higher degree qualification GEN students in our setting are required to undertake a small independent research project, usually an IR. Given, however, that many enrolees have never studied research previously, these students are expected to go from being research naïve to completing projects within a year. While this accelerated process poses pedagogical challenges to teaching and learning in relation to GEN students (Macdiarmid, McClunie-Trust, et al., 2021), supervision is also problematic given the students have not usually undertaken any other postgraduate study and are only in the infancy of developing their knowledge and identity as registered nurses.
One means of managing such situations is by guiding students to undertake a highly structured literature review. Integrative reviews (Whittemore & Knafl, 2005) offer a rigorous scaffold (Dhollande et al., 2021) for the novice researcher to follow and from which the supervisor can support the student to produce a robust dissertation. Yet, as mentioned above, despite the framework being clearly articulated in terms of the purpose of an IR and how to undertake such a review (Whittemore & Knafl, 2005), the road to success has a series of pedagogical hurdles. Specifically, the students appear to grapple with the rigorous, lengthy, and detailed procedures involved in undertaking an IR, while supervisors commonly appear to presume GEN students’ ability will be aligned with standard graduate nurses who are Masters’ students, which we have discovered is not the case. The presumption that an IR is an easily followed ‘recipe’ can mean supervisors leave GEN students for extended periods to work autonomously on their dissertations. However, the reality is that GEN students need regular and vigilant supervisory support to achieve success in their research project.
We have identified key challenges for GEN students in successfully undertaking an IR. They are concurrently making two significant identity shifts, firstly, in developing a nursing identity and additionally, being required to construct an identity as a scholarly researcher, albeit a novice. The latter is a shift beyond the student identity they have been inhabiting. We concur with Badenhorst (2018) that conducting reviews is pedagogically complex and Masters students require mentorship through these identity shifts.
THE INTEGRATIVE REVIEW AS A THRESHOLD CONCEPT
We draw from the pedagogical notion of threshold concepts to illuminate why the seemingly straightforward, well-defined process of an IR commonly poses challenges for GEN students. We concur with Walter and Stouck (2020) and Wisker (2015) that for novice researchers, undertaking a rigorous literature review is a type of threshold crossing with the identity shift from merely being a consumer to a producer of research. Threshold concepts are “akin to a portal, opening up a new and previously inaccessible way of thinking about something” (Meyer & Land, 2003, p. 1). Threshold concepts differ from core concepts in curricula in that they are associated with a shift in worldview which entails the unsettling of previously held ideas. This disruption means that students experience something of a liminal space where, to be successful they need to tolerate a period of uncertainty and even identity disruption (Land et al., 2014). For example, our students often navigate a phase of resistance to the IR projects with claims of their irrelevance as they never intend to be a researcher and they want to be a ‘real nurse’ – a ‘hands-on’ nurse. Chen et al. (2016) consider that supporting students through this identity work is part of appropriate academic guidance through the literature review process. This reticence is part of the threshold crossing, where students come to appreciate that the ability to analyse research is fundamental to answering clinical questions and optimising patient care (Carter-Templeton et al., 2022). In the case of an integrative literature review, students’ undergraduate experience and initial post-graduate study typically do not demand such rigorous engagement with the literature, where students may rely on superficial descriptive writing when populating academic work with citations (Walter & Stouck, 2020). Wisker (2015) considers the threshold crossing of the literature review involves students “…moving between research activity, reading, interpretation of theoretical perspectives, the importance of conceptual and interpretive findings, and the actual processes of writing” (p. 65).
There are five facets of a threshold concept (Meyer & Land, 2003), all of which we have noted GEN students are engaged with as they navigate the phases of the IR. These facets are explored in detail as we unpack each step of the IR process. As a brief definitional introduction, drawing on Meyers and Land (2003) these facets are as follows: Firstly, a threshold concept is transformative in that mastery leads the learner to identify more as a professional in the field than a student. Secondly, a threshold concept is troublesome for learners and as such typically requires prolonged engagement and critical reflection to navigate. For example, in the instance of the IR, the practices students have previously considered would suffice to undertake a literature review are now far from adequate. Thirdly, threshold concepts result in what are often irreversible changes in viewpoint. In the case of undertaking an IR, for example, students may come to think about what counts as evidence in a substantively different way. Fourthly, threshold concepts are integrative in that for the learner, they bring together ideas they previously considered disparate. In the case of our students, they come to have a much deeper appreciation of the autonomy and scope of nursing practice and the profession’s contribution to health equity. Fifthly, threshold concepts are bounded and therefore the IR becomes a delineated type of research, substantively distinct from what students have previously considered to be a literature review in the assignment context.
We concur with Kistler and Tyndall (2022), that in nursing programmes, academic cognisance of threshold concepts can assist students with troublesome knowledge acquisition. Titchen’s (2003) clinical nursing concept of critical companionship is also relevant to the GEN supervisory relationship, where students experiencing the high challenge of undertaking an IR are consistently met with high support. What follows is our analysis of the common pitfalls for GEN students and the supervisory ‘critical companionship’ they benefit from as they navigate the threshold crossing of the integrative review process.
OVERVIEW OF THE ISSUES
While the challenges faced by GEN students are now well-defined (Macdiarmid, Winnington, et al., 2021), and subsequent supports required have been identified to ensure the successful completion of their pre-registration programmes (Jarden et al., 2021), their learning needs in relation to completing a Masters’ level research component have, until now, been missing. Specifically, as these students are most often research naïve together with engaging in a fast-paced accelerated programme of learning, our experiences highlight that there is an increased demand on academic supervisors to closely monitor and guide such students to ensure success. As previously noted, engaging with high quality literature reviews is fast becoming a staple for post-graduate nursing research students (Dhollande et al., 2021). Our experiences of supporting such students to complete an IR have provided an opportunity to share key moments along the research journey that require increased vigilance to aid success. Specifically, our experiences suggest that while the IR is a structured approach to undertaking a literature review, and that Whittemore & Knafl (2005) offer clear steps to complete the process, there remain numerous key points that have not yet been articulated in the literature that would prove helpful for novice researchers, supervisors, and GEN students alike.
The notion of the literature review as a threshold concept is useful because the ‘terrain’ does not pass smoothly in a linear trajectory from difficult to easy (Chatterjee-Padmanabhan et al., 2019; Meyer & Land, 2003). At each step there is a process of coming to understand conceptually difficult material, but as soon as there is relative mastery of that stage there is the destabilisation with new learning in the next phase of the IR. This ongoing instability contributes to students’ sense of liminality. Students are perpetually shifting between brief experiences of gaining proficiency at one step and then having to accomplish new skills as they progress, throughout the entirety of the IR project (Wisker, 2015). Over the course of four years, we have used a model of online small supervision groups, rather than one-to-one supervision. We and the colleagues we have mentored into supervision roles have found that peer companionship can play a large part in keeping students’ morale boosted through the IR. We and our colleagues typically provide supervision with four to six students working with an experienced supervisor and a collegial mentee. What follows is an exploration of what is troublesome and transformative for GEN students undertaking IRs, and the vital component of learner-centred supervision to keep students engaged with the reflection and questioning required to navigate this threshold concept (Zepke, 2013). Also refer to Table 1 for a summary of issues.
Phase one: Preparing the guiding question
For GEN students, as both consumers and producers of research, undertaking an IR is not without its dangers. These students often have no healthcare background, thus leaving them exposed to not understanding the legislative landscape, the socio-political underpinnings of nursing practice, nor the boundaries governing nurses’ scope of practice. Therefore, these knowledge gaps position GEN students unfavourably when preparing their guiding research question and argument for an IR. A preliminary investigation of their area of interest may be overwhelming as they begin to realise that their nub of an idea is part of a sprawling ‘conversation’ they will be attempting to join (Wisker, 2015). There is no easy fix to this situation, given there is no substitute for actual clinical experience and immersion in the healthcare context. Supervisors can, however, offer critical companionship at this stage of the IR process, which will set up the remainder of the supervision process for success (Chatterjee-Padmanabhan et al., 2019).
Such critical companionship can also include a broad educative supervisory role. For example, undertaking these IRs also requires students’ engagement with health and legislative contexts that are new and unknown for the GEN student. This learning contributes to the development of their nursing identity as well as ensuring the research question contributes to nursing knowledge (da Silva et al., 2020). To support students to craft a relevant question, supervisors can facilitate the exploration of the topic including whether and how an argument can be crafted. Students may latch on to one aspect of what is a ‘wicked’ problem in health (Glasgow & Colbert, 2022) and much more complex than their initial binarised assessment of the problem. To develop an argument the students first require a grasp of the wider scholarship and debates around the topic and then need to establish their positionality – what focus and approach they will take (Wisker, 2015). In terms of a threshold concept what is also challenging is that students may become overly attached to a question not suitable for an IR. This issue arises where there is either an abundance of literature, including recent reviews on the topic, or a dearth of current literature, making the project untenable.
Phase two: From literature search to data extraction and reduction
Prior to the IR, students typically have no experience of using literature as a data set and are likely to have only searched for literature to support claims for other assessment purposes such as essays (Badenhorst, 2018). This burgeoning comprehension that a literature review has multiple forms is destabilising for students (Chatterjee-Padmanabhan et al., 2019; Walter & Stouck, 2020). It is perhaps unsurprising then that, when considering the stages of an IR, we noted that GEN students often underestimate the time and workload involved in searching the literature for their data sources. A quick dive into a database to pluck out a citation to support a claim will no longer suffice. Therefore, while in principle the Whittemore and Knafl (2005) framework identifies the need to undertake rigorous and structured database searching, the reality of the level of diligence is typically shocking for students. In part, this bewilderment may also be due to their näivety of research processes as novice researchers, or indeed the IR may appear ‘easier’ than gathering empirical data. Yet the reality is, structured literature searching is time-consuming and requires critical thought (Chatterjee-Padmanabhan et al., 2019). As possibly the most demanding part of the IR process, we have found that GEN students are often overwhelmed by the level of precision required in a formal literature search and that the literature is their data source.
Moreover, given their recent introduction to research as being part of evidence-based nursing practice, together with the fact they are not yet registered practitioners, it can be challenging for the GEN students to think sufficiently broadly in terms of their inclusion/exclusion criteria, keywords and phrases and timeframes when data gathering. Our experiences to date indicate that it is at this point in the IR process that supervisors play a vital role. Their oversight is required to ensure that the inclusion/exclusion criteria are adhered to strictly. Most importantly, supervisors need to review the final selection of articles to ensure they match the criteria impeccably. If this support is omitted from the IR process at this time-critical point, it will result in a fundamentally flawed IR due to the impact on the findings of the study, leaving the student in a problematic position from which there is little comeback.
While literature searching and data extraction are both time consuming and challenging for GEN students and supervisors alike, the requirement that the IR process is pure, consistent, and reliable also relates to ensuring students do not get ‘attached’ to certain manuscripts if they do not fit the inclusion/exclusion criteria. This deliberation can, at times, cause tensions. However, clarity around the IR process according to Whittemore & Knafl (2005) remains a priority. We have noted additional points that pose challenges for GEN students. One is that students need guidance to justify the inclusion criteria pertaining to the years searched.
As newcomers to the health sector GEN students are often unaware of the changes in the scope of nursing roles, and key shifts in legislation, politics, policies, and practice that have implications for an appropriate time span for the data set (McKenna et al., 2017). Students need supervisory guidance to be intentional about the justification for the years searched, rather than plucking a time frame from generic recommendations. Additionally, students often uncritically include primary research from countries with widely varying health systems, socio-economic, political systems, nursing programmes, unaware that the studies may have limited relevance to the research question and argument.
Phase three: Data evaluation
When new to the research process, even as a consumer of research, understanding methodological approaches to studies is challenging, and no more so than for these GEN students (Macdiarmid, Winnington, et al., 2021). As novice researchers, honing skills of being able to identify what type of scholarly text is under consideration and what methodology a study has engaged with is imperative given a key component of an IR requires critical appraisal. Students’ variable ability to identify types of scholarly texts and methodologies impacts the choice of critical appraisal tools to be deployed. Thus, close supervisory involvement at this stage is essential if further errors are to be avoided. Yet a further complexity is added to this experience when we again consider that GEN students are unlike other nursing Masters’ students in that they are not yet working in clinical practice and so do not possess knowledge regarding what constitutes appropriate and high-quality nursing journal publications. This is another point where close critical companionship supervision is helpful in guiding novice researchers and introducing them to appropriate nursing literature.
Another distinguishing feature of the IR that sets it apart from students’ previous use of literature is the requirement for them to consider whether the selected document meets ethical standards. This step ideally involves more than checking that articles have been sourced from peer-reviewed journals and that an ethics review board has signed off on the study. Ethical challenges when undertaking reviews include ensuring an accurate representation of each study’s findings and avoiding plagiarism (Wager & Wiffen, 2011). These issues may be hurdles for students who may have made up their minds about what they want their results to say. Accurate representation of a source is a demanding step for some students as in their scholarly experience there may be only a cursory assessment of a document before using it as a source to support an essay argument. Unintentional plagiarism occurs when writers struggle to paraphrase and have not attributed long phrases and sentences as direct quotes. There are also issues of representation to be considered. For example, in the authors’ context in Aotearoa New Zealand, there is a strong and growing ethical critique of how health research has typically problematised Indigenous Māori with a focus on personal deficits rather than interrogating the ongoing impact of colonisation and a structural analysis of health determinants (Cram, 2019).
Phase four: Data analysis
While critical appraisal is a key component of the IR process (Whittemore & Knafl, 2005), students also need to critically analyse and discuss the findings from the literature selected as a mechanism to answer the research question. For many, including GEN students, Braun and Clarke’s (2012) thematic analysis is an appropriate approach to take, as it is theoretically flexible and can be adapted to both inductive and deductive analysis. However, as with previous stages of the IR process, the development of themes is not without its pitfalls for GEN students. The idea of immersive reading, coding and theme formulation may be daunting for many novice researchers and a key issue for many is finding their authorial voice as opposed to seeing literature as the master and gatekeeper of knowledge (Badenhorst, 2018; Chen et al., 2016). This switch from consumer to producer of knowledge is unfamiliar.
Specifically, our experiences highlight that the shift from a descriptive to a critical writer appears to be a particularly difficult hurdle for many to navigate. Part of developing a critical authorial tone is the confidence to synthesise the findings (Chatterjee-Padmanabhan et al., 2019; Wisker, 2015). This shift in students’ identity of taking up the authority as a researcher to make arguments and claims is often outside of their sphere of comfort. Nonetheless, this authorial voice is an important development point in the GEN students’ learning and one that can be fully supported by the supervisory team. Supervision requires close involvement at every stage of theme development (Wisker, 2015). Key supervisory points include ensuring students are not distracted by interesting data components that do not answer their research question. Additionally, students may require guidance around the shift from codes to themes to ensure the ‘story’ of the dataset is clearly addressed coherently and with enough depth. Braun and Clarke (2012) recommend two to six themes for a 10,000 piece of research. Typically, students’ initial attempts to shift from codes to themes result in themes that do not adequately address the research question because they are either are too broad or too reductive. Supervisory conversations that incorporate mind mapping can help to ‘corral’ codes.
Phase five: Writing up discussion and conclusion
Graduate Entry to Nursing students, not dissimilar to other postgraduate students, have difficulty in finding their critical authorial voice in their writing and critiquing data. These combined challenges are further notable when faced with writing the IR discussion chapter. Supervisors have become familiar with the parameters of a research project discussion in terms of a scholarly ‘conversation’ between the data and the wider literature. However, our experience as supervisors, mentors and examiners of the GEN integrative review projects convinces us that this integration is a struggle for GEN students. After the narrow focus of the data analysis, they are challenged to consider more broadly the issues arising to answer their research question.
While writing a discussion has the potential to be a transformative process for GEN students, many have already formulated underlying beliefs and values that are difficult for them to reconcile in view of their findings. For example, a long-held neoliberal view of health may mean it is novel for the student to view the issue through a social justice lens (Shahzad et al., 2022). Also, students may again experience ‘growing pains’ to find their authorial voice as they attempt to synthesise a discussion that includes wider literature not aligned with their argument and findings (Chatterjee-Padmanabhan et al., 2019). Students navigating similarities, differences and gaps between the data set and the wider literature and additionally addressing limitations and recommendations mean that the ‘home stretch’ of the IR continues to be a discomforting marathon for many of our students. These challenges are noted by other authors (Walter & Stouck, 2020; Wisker, 2015).
We recommend that supervisors do not underestimate the challenge for students in this phase, which entails participating in knowledge creation by being able to identify the significance of their study. Students’ past skills in merely recording knowledge become redundant, and hence the troublesome nature of IRs as students iteratively move between the integration of understanding one step in the process only to feel they are back at the beginning and feeling confused by the next step (Chatterjee-Padmanabhan et al., 2019). The IR process outlined by Whittemore and Knafl (2005) indicates that drawing a conclusion and verification are the final stages in writing up the study. However, we draw attention to the important role of supervisors in continuing to support and nurture GEN students to construct sophisticated nursing-specific recommendations from their findings. Commonly we note this guidance around recommendations includes encouraging shifts in thinking beyond a simplistic attribution of blame towards individuals, institutions, and systems to students being able to identify how their project might contribute to nursing knowledge and practice. In terms of the IR as a threshold concept, we have found that what ‘locks in’ the irreversible change and transformation (Land et al., 2014) is supervisors supporting students to publish. This shift from being an outsider to an insider of the academe is a source of pride for students and deepens their appreciation of the relevance of evidence to practice.
CONCLUSION
Graduate entry nursing students are typically mature students who bring a wealth of diverse experiences to GEN programmes. Given the demands of an accelerated programme, it behoves supervisors to be as strategic as possible in guiding research projects. The neatly packaged descriptions of the steps in the IR process belie the complexity of the iterative nature of these projects for GEN nursing students. No sooner have students gained mastery in one step than they have the sense of stumbling backwards with new and troublesome skills to be gained. The notion of the IR as a threshold concept usefully assists supervisors and students in making visible why these small projects are destabilising for student confidence and demanding in terms of supervisor time. Students’ earlier encounters with undertaking literature reviews for essays typically lull students into a premature sense of confidence with the process and they rapidly experience the shock that an IR is beyond what they have imagined the project to be. Each step of the IR has students experiencing multiple troublesome facets of the process simultaneously, including grappling with scholarly writing, finding an authorial voice, growing skills in critical reflection, and committing to the prolonged engagement required to produce a coherent document. Mastery of one step brings only fleeting relief before the challenges of the next step. We consider that the supervisor’s role as a critical companion through this iterative process can support students through the transformative process of becoming a producer of research.
Funding
None
Conflict of interest
None