Deliberate (Mis)Representations: A Case Study of Teacher Influence on Student Authenticity and Voice in Study Abroad Assessment

International Journal of Student Voice

A peer-reviewed, independent, open-access journal

Pennsylvania State University

Volume 3                            IJSV                           April, 2018

Deliberate (Mis)Representations: A Case Study of Teacher Influence on Student Authenticity and Voice in Study Abroad Assessment

Kayla M. Johnson,  Pennsylvania State University

 

Citation: Johnson, K.M. (2018). Deliberate (Mis)Representations: A case study of teacher influence on student authenticity and voice in study abroad assessment. International Journal of Student Voice, 3(4).


Abstract: This article examines teacher influence on the authenticity of students’ representations of their learning in a study abroad context. Through presenting a case study of a research project aimed at uncovering what and how students learn in study abroad programs, this article suggests that the representations of learning that students shared with their professors in assignments misrepresented their experiences abroad and the learning that resulted from those experiences. Using an innovative ethnographic technique-Photo-Cued Interviewing (PCI)-students shared with me that they did not feel able to authentically express what they learned during their experience in assignments monitored by their teachers and others, such as public blogs and class presentations. The paper concludes with practical recommendations for educators and educational institutions regarding ways to more democratically facilitate open dialogue through which understanding student learning may be more possible.
Keywords:student voice, study abroad, teacher influence, photo-based research, assessment, student learning


“How is someone going to know how my experience truly was unless you get my honest opinion and point of view on it?” – Danielle

Introduction

A Case of Deliberate Misrepresentation

One month after returning home from her study abroad program in Sweden, Danielle and I discussed her experience over a few of her photos. As she talked about a photo of Gothenburg’s southern archipelagoes (Danielle, Photo 1, shown below), she shared that this brief excursion, which was an impromptu outing separate from the course itinerary, was more impactful for her learning than, for example, their visits to companies like IKEA and Volvo. I remembered that Danielle had talked at length about these company visits during her in-country presentation, which was delivered in front of classmates, her professors, and Swedish faculty members. The presentation was meant to highlight what she had learned throughout the program, but this learning moment was absent. When I mentioned this discrepancy to Danielle, she explained that her presentation reflected what she was supposed to say, not what really happened. There were some things, Danielle explained, that she decided to leave out.


Figure 1s

Figure 1—Danielle, Photo #1: View from Gothenburg Archipelagoes

Tensions between Teacher Influence and Student Authenticity

Danielle’s comment exposes the central tension explored in this paper—how teachers influence student voice and authenticity in assessment. As I interviewed Danielle’s 14 classmates in this case study, each student shared with me that, on some level, the reflective assignments they completed throughout their course did not authentically represent their experiences or learning. Teacher influence on authenticity and voice was identified as the main catalyst for the students’ deliberate misrepresentations of their experiences. While this paper is part of a larger study on student learning abroad, the students’ discussions about their teachers’ influence on the content of their assignments begged a new question: In what ways can teacher influence impact student authenticity and voice in assessment efforts?

Issues of teacher power and influence on student authenticity and voice are not new in educational research (e.g., Katz, 1992; Scott et al, 2006), and are not unique to study abroad contexts. For example, teacher power can influence the authenticity of the work students produce (Smith, 2000; Bower, 2003; Reinsvold & Cochran, 2012), as research shows that students often fabricate assignments to please their teachers (Casey & Hemenway, 2001). This power imbalance can become problematic when teachers attempt to understand student experiences through reflection tasks, particularly when students are trying to interpret and act upon what they think teachers might want to hear.

Using reflections to understand student learning is a current trend in research on learning in study abroad contexts (e.g., Lindsey, 2005; Dietz et al, 2017; Goetz & Holliday, 2017), but the field is relatively under-researched. Teacher-researchers, often the study abroad course instructors, solicit student reflections in order to understand what students learn and how learning occurs. For example, the student-participants in this case study were assigned three reflective assessment tasks: daily blog posts, in-country presentations, and post-program video presentations (see Appendix A). However, the students said that what they shared with their instructors in these assignments inaccurately represented their experiences abroad.

In ways explored in this paper, the authenticity and voice of these students were negatively impacted by teacher influence, which resulted in incomplete and inaccurate depictions of their experiences, and thus, their learning. Using this study abroad program as a case study (see Johnson, 2017 for more details), this paper examines the influence that teachers can have on student authenticity and voice in educational contexts, highlighting how teachers influence how students talk about their experiences, and problematizing the impact this has on student learning assessment. While this paper is not meant to be generalized to all study abroad programs, educational contexts, teachers, or assessment efforts, it serves as a cautionary tale of student assessment efforts gone awry. It provides insights into how educators can be more mindful of their influence on students’ authenticity and voice, and why this is important for understanding student learning.

Context

Understanding Student Learning in Study Abroad

Student learning assessment in study abroad is a burgeoning field that has gained considerable attention in recent years (Vande Berg, Paige, & Lou, 2012), but is still under researched. While accountability standards have become a more central focus (The Forum on Education Abroad, 2015), some critics argue that study abroad programs facilitate very little student learning gains (e.g., Charbonneau, 2013; Eyler & Giles, 1999; Marklein, 2004). Field leaders have called for scholar-practitioners to respond to these pressures by challenging untested claims and avoiding assumptions about the benefits of education abroad (Ogden, 2017).

Recent efforts have demonstrated that studying abroad can lead to: gains in intercultural competence and cross-cultural development (Salisbury et al, 2013; Carrillo, 2014; Heinzmann et al, 2015); social-emotional learning (Johnson, 2017; DeGraaf et al, 2013; Bathke & Kim, 2016); and foreign language acquisition (Kinginger, 2017). Studying abroad is also positively correlated with higher GPAs and critical thinking, and greater postsecondary persistence and student engagement (NSSE, 2007; Kuh, 2008; Kuh et al, 2010). Such outcomes, however, are contingent upon several factors including program duration, program location, and the level of cultural immersion (Anderson & Lawton, 2011; Donnelly-Smith, 2009; Kinginger, 2013). This study emphasizes another influential factor: the conditions under which student assessment is conducted. Current research on student learning in study abroad has not problematized how learning outcomes are assessed, and this paper, in part, seeks to address this gap by highlighting how teachers may influence student authenticity and voice in certain kinds of assessment efforts.

Situating Student Voice in Learning Assessment

Although existing research on student learning in study abroad has not examined the influence of the method of data collection, there seems to be a growing awareness of how different methods can elicit different types of understanding. While study abroad assessment efforts have traditionally been conducted quantitatively, qualitative studies have become popular in recent years (e.g., Lee & Green, 2016; Caldwell & Purtzer, 2015; Vatalaro et al, 2015). This shift is important, as qualitative research has great potential for mitigating power imbalances between the researcher and the researched (Denzin & Lincoln, 2018). When conducted democratically with younger populations, like students, qualitative assessment research has the potential for opening up spaces for students to exercise their voice and agency (Gonzalez et al, 2017), allowing researchers to better understand students’ experiences from students’ perspectives.

While investigating the student perspective is just budding in study abroad research, the broader student voice field has deeper roots. For example, Mitra’s (2006) student voice framework refers to the ways in which students are given opportunities to participate in decision-making processes in school settings. This conception of student voice work occurs in three phases: students being heard, students collaborating with adults, and building students’ capacity for leadership. My conceptualization of student voice in this paper relates most closely to Mitra’s (2006) first phase—being heard. While I recognize the breadth of student voice scholarship, I have selected Mitra’s framework because of its foundations in listening to students; research shows that students appreciate when their voices are heard and that their perspectives are valued (Mitra, 2003; Oldfather, 1995). Adults are then tasked with providing spaces for students to share their voices safely and comfortably, and with awareness of their interpretative biases, to ensure students are heard accurately (Levitan, 2018). This case study highlights what can happen when these spaces are not seen as safe and comfortable and explores how this impacts what students say and, thus how they can be heard.

Conceptual Framework: Problems with Power and Positionality in Education

While this paper employs a grounded theory analytical approach (Corbin & Strauss, 2015), it is loosely informed by literature on power and positionality in education. This paper also assumes a constructivist perspective, which acknowledges the representational aspects of reality—that realities are constructed, contextual, and subjective (Crapanzano, 1999). From a constructivist perspective, what students say and do does not authentically reflect “reality” in an objective sense; rather, their representations are just one version of a reality. Without acknowledging the representational aspect of students’ realities and what influences those representations, educators cannot come to valid conclusions about student learning or make responsible curricular recommendations based on students’ assignments.

One influence over students’ representations of realities is power. Issues of power and representation are not new in educational ethnography (Katz, 1992), or in classrooms (Bianchi, 1997; Scott et al, 2006; Shepardson & Britsch, 2006). Critical theorists (Levinson, 2011; Apple, 1995) argue that power relations between the researcher and the researched inhibit informants from sharing their most authentic selves, instead prompting informants to act in ways—or offer representations of reality—that will please the researcher. In educational settings, perceived teacher power and students’ desire to please the teacher can prohibit teachers from accessing certain representations of realities (Bower, 2003). For example, when students know a teacher is going to read what they write, it may influence the content of their writing (Smith, 2000). In addition, teachers may choose to reshape, edit, or ignore student responses if they represent realities that go against “acceptable” or “desired” points of view (Lemke, 1990; Reinsvold & Cochran, 2012). For students, this power imbalance leads to less intrinsic motivation to complete tasks in ways that accurately and authentically represent the self; completing assignments then becomes a “balancing act” (Cleary, 1996) in which students produce something between what they want to say and what they think the teacher wants to hear. What results then is often student self-censorship, which can lessen the impact of the cognitive contribution they are able to make regarding their own learning (Roberts & Nason, 2011).

This is particularly problematic when considering reflective tasks, such as in this study. Reflection involves four phases: the experience itself, the description of the experience, the analysis of the experience, and taking intelligent action (Dewey, 1933; Rodgers, 2006). However, when the description phase becomes distorted—in this case, heavily influenced by the teacher—the analysis and action phases become compromised. In other words, if students do not authentically represent what and how they learned, then the analysis and conclusions the teacher and students can draw become inaccurate (Spaulding, 1995). Students also lose motivation to reflect when the designated format for a reflective task, which is dependent upon the audience, often their teacher, is uncomfortable for them (Cleary, 1996). Students’ sense of pride, ownership, and engagement with the topic diminishes as the reflection becomes less representative of themselves and their thoughts (Casey & Hemenway, 2001).

Therefore, examining teacher influence on student authenticity and voice is critical, particularly as understanding the experiences of students is important for understanding their learning. Teachers must be cognizant of how they may be influencing their students and the work they produce and should reflexively examine this influence as they assess and learn from student work. It is imperative to understand how students perceive and define their situations before teachers and researchers can make sense of their reflections on them (Delamont, 1976).

Research Methods

Part of a larger study on student learning in short-term study abroad programs (Johnson, 2017), this paper examines teacher influence on student authenticity and voice in learning assessment efforts. The case study I present utilizes an original ethnographically (Spradley, 1980) and phenomenologically (Van Manen, 1990) inspired assessment strategy—Photo-Cued Interviewing (Johnson, 2017)—to examine the experiences students have abroad and what they learn from those experiences. I compare these student accounts with the reflective tasks they completed for their course professors to develop a deeper understanding of how their representations of reality change under the influence of their teachers. I come to this study with a constructivist lens and use the following interpretive methods.

Sample

This case study details the experiences of fifteen university students who were participating in a 10-day faculty-led study abroad program in Sweden. They all attended the same large public university in the U.S. Eleven students were female, four were male, they were all between the ages of 19 and 30 (most were 20-22), and they were predominantly white. My connection with this program was the result of my work in the university international office, where I worked on initiatives relating to faculty-led study abroad courses. I contacted professors of three different programs as part of my study site selection. I ultimately selected this program because of the professors’ positive reputation among my colleagues.

Participant-Observation

As a participant-observer (Spradley, 1980), I followed the fifteen students during their program. My role as an ethnographic researcher was, as Adler and Adler (1994) argue it should be, between detached and non-participatory (Gold, 1958) and “going native” (Malinowski, 1922). I functioned as an insider-outsider to gain emic (within the student culture) and etic (outside the student culture) perspectives (Spradley, 1980) of the students’ experiences. I participated in program activities with the students, held informal conversations with them, and observed their engagement with the program activities, the host country, and each other. I recorded extensive fieldnotes (Emerson, Fretz, & Shaw, 2011), creating a regular and systematic record of my observations.

Photo-Cued Interviewing

Photo-Cued Interviewing (PCI; Johnson, 2017) capitalizes on this generation’s ubiquitous use of photography as a means of expression, meaning making, and communication (Levine & Dean, 2012)—to understand their experiences in a non-invasive way that is rich with context, rigorous in methodology, but also speaks the language of students (Gibson et al, 2013).  PCI as a method facilitates dialogue around students’ photos, which serve as visual representations of their learning or meaning making. Like Tobin et al.’s (1989) video-cued ethnography approach, the photos are not viewed as data, but as methods for eliciting data.

One month after the program, I conducted individual PCI interviews (N=9). Each student selected three photos from their trip that represented something meaningful or significant about their experience. Using a semi-structured interview protocol (Spradley, 1979), I asked students to reflect upon their experiences—using their own photos as prompts—to understand what they found meaningful, what they learned, and how/when/where that learning occurred. I also conducted two PCI focus groups with students (N=3, N=4). Each student selected one of his/her pictures to discuss with others, allowing them to reflect together, offer opposing viewpoints, and co-create knowledge.

Most of the PCI conversations with these students were free-flowing and animated. However, during one focus group outside of a local restaurant, one of the professors walked up and joined us unexpectedly. The students’ animated discussion was abruptly derailed by their professor’s presence. Even after walking away, the professor’s presence lingered as the students struggled to regain traction on the topics they had been so animated about just moments before (interview notes, 6/26/16). When I asked the students about what had just happened, they said that they were nervous that the professor had overheard our conversation.

This anecdote, in part, justifies my PCI approach, which allowed the students to freely express the meaning of their experiences in a non-judgmental (i.e., non-assessed, instructor-free) environment. This approach produced insights into informants’ subjective experiences beyond what and how they learned, particularly how these experiences were misrepresented, falsified, or absent from their other forms of reflection, like class assignments. Interviews and focus groups lasted 85-145 minutes.

Ethical Considerations

Because this research involves human subjects and photographs, several ethical considerations were made. At the professors’ request, I obtained consent from all students enrolled in the course. Students were informed verbally and in writing that they would be asked to participate in interviews, provide pictures, and that I would travel with them throughout their program. Students were informed that their photos would be shared in presentations and publications, and that they should refrain from submitting photos showing their faces if they did not wish to be identified. While nearly all participants submitted at least one photo showing their face, I have elected to blur their faces in this article to provide enhanced confidentiality, in case their decision to reveal their identity changes in the future. I have also blurred the faces of bystanders who appear in the students’ photos. Students were given one week to consider their decision to consent, and every student submitted a signed consent form before any data was collected.

Analysis

Interviews and focus groups were audio recorded and transcribed verbatim, and my field notes were typed. Data were uploaded into NVivo qualitative data software. I employed a grounded theory analytical approach, loosely informed by my literature review and conceptual framework (Corbin & Strauss, 2015). When necessary, I conducted member-checks with informants to ensure validity and to minimize the influence of researcher bias (Lincoln & Guba, 1985).

Findings

The following findings highlight how students’ authenticity and voices were influenced by their teachers. Both indirectly and directly, teacher influence had profound (and largely adverse) impacts on students’ authenticity and voice in their reflective assignments: “[The assignments] were very fake. It wasn’t an accurate story of what happened.” Students generally described their reflections as “censored” and “guarded.” I explore three emergent themes that contributed to the students’ deliberate misrepresentations: wanting to earn a good grade, adhering to restrictive assignment instructions and narrow conceptions of learning, and responding to institutional pressures.

Wanting to Earn a Good Grade

The most prominent explanation of teacher influence on their reflections centered on the students’ desire to earn a good grade on their assignments. While not a new finding (Smith, 2000; Casey & Hemenway, 2001), our conversations revealed the lengths to which students went to produce “A-worthy” work, and that their desire to earn a good grade from their professors caused students to omit information about their learning experiences.

Several students feared that the content of their reflections would negatively impact their grade for the course. Danielle’s three photos were all from Gothenburg, the group’s last stop in Sweden (Danielle, Photo #1, #2, & #3, shown below). They each depicted experiences that were not facilitated by the professors or included in the course itinerary—they were incidentallearning experiences (Marsick & Watkins, 2001). However, these experiences, which she deemed her most significant, were not mentioned in any of her three reflective assignments. When asked to explain, she said:

I had to really sugar coat everything and put some fluff into it and make it an A presentation. It was a struggle sometimes to put it together because there were other things that I wanted to say but I knew it probably wouldn’t go over well with my grade.

Figure 1s

Figure 1—Danielle, Photo #1: View from Gothenburg Archipelagoes

Figure 2

Figure 2—Danielle, Photo #2: Bikes on Gothenburg Archipelago Islands

Figure 3s

Figure 3—Danielle, Photo #3: Anchor on Display in Gothenburg Harbor

Because she was being graded, Danielle felt indirectly pressured to only discuss activities that were directly related to the course. So, Danielle significantly altered how she presented her learning experiences in ways that would better please the professors, saying that she learned from course lectures, instead of, for example, on a boat tour.

Dana explained that as she wrote her blog posts each evening, the experiences depicted in her writing reflected more of her desire to earn a high grade than the meaningful learning that she was experiencing. Talking about her photo of a lake (Dana, Photo #1, shown below), she said:

I was like, “Sweden is like this,” and even when I was saying it I was like this is wrong. But, and this is going to sound bad, but it’s what they wanted to hear. I just said what they wanted to hear basically because I’m trying to get a good grade.

Figure 4

Figure 4—Dana, Photo #1: View of Lake in Jönköping

Dana explained that, for example, while she wrote about how Swedish employment practices should be implemented in the U.S. (which, to her, was the “lesson to be learned” from the course), she actually believed that differences between the U.S. and Swedish contexts, like size, would make such implementation difficult. However, Dana felt that making such a statement, even a critical and insightful one like this, might earn her a lower grade.

The pressure felt by Danielle, Dana, and other students to please their professors in order to earn a good grade on their assignments limited their authenticity and voice. It also significantly impacted the reflection, and thus learning (Van Manen, 1990), that these types of assignments were likely meant to facilitate.

Navigating Value Differences around “Learning”

A number of students noted that the professors’ implicit focus on learning related to the course objectives limited the kinds of experiences they shared in their reflective assignments. The perceived pressure to stick to what students described as “restrictive” instructions caused some students to omit certain meaningful experiences from their writing. For example, as Emily discussed her photo from an impromptu hike in Jönköping (Emily, Photo #1, shown below), she said:

They’re giving you an assignment and then give you a word limit and THEN are like, “Oh, don’t mention this.” Like, you want to me to write about what I did today—how am I not supposed to mention that? And then you don’t want to lie about it, you don’t want to twist it, so you just leave it out.

Figure 5s

Figure 5—Emily, Photo #1: View from Hill During Hike in Jönköping

The “this” that Emily refers to were all of the incidental experiences that contributed to her learning, such as her mishaps traveling to Sweden for the program (Emily, Photo #2, shown below). Because the professors were limiting the word count and content of the blogs, Emily and others had to leave other things—important, meaningful things—out of their reflections.

Figure 6

Figure 6—Emily, Photo #2: Train Station in Copenhagen, Denmark

Danielle also shared that the professors’ seemingly narrow conceptualization of learning limited what she felt able to write about. The blog posts were “to be taken seriously,” but Danielle explained that her most meaningful learning experiences were silly (like her canal boat tour; Kate, Photo #1, shown below—as discussed in focus groups) and fun (like the archipelagoes). These did not seem “serious” enough to be included, no matter how meaningful they were for her learning:

It sort of didn’t capture—it seemed like we had to suppress a lot of stuff and [the blogs were] kind of fake. I really couldn’t be totally genuine and honest about my experiences. I had to shelter all the viewers from what was really going on.

Figure 7

Figure 7—Kate, Photo #1: Canal in Gothenburg

Danielle provided other examples of activities that seemed to prompt learning for her and her classmates, but that might not be considered “serious” (i.e., academic): visiting an amusement park (Kari, Photo #1, shown below—as discussed in focus groups), dancing with Swedish students at a nightclub, the hike mentioned by Emily, etc. However, because Danielle had to “suppress” these experiences, she felt that her reflections were not authentic representations of her learning.

Figure 8

Figure 8—Kari, Photo #1: Students at Gothenburg Amusement Park

Responding to Institutional Pressures

A final way in which teachers influenced the students’ authenticity and voices occurred in response to institutional pressures—specifically from the university administration and academia more broadly. External links to the students’ blog posts and presentations were shared with university administrators and students’ parents, which signaled to some students that these assignments were to “make the program look good.” As Brandon shared a photo of a public tram (Brandon, Photo #1, shown below), he, like many of his classmates, talked about how travel itself was the catalyst for his most meaningful learning: “It’s about getting lost and finding yourself in a foreign country.” However, he believed that such lessons would not “make the program look [as] good” as if he were to write about planned course activities.

Figure 9

Figure 9—Brandon, Photo #1: Tram in Gothenburg

More directly, students suggested that the professors sometimes edited students’ blogs, which students submitted to them each night, before posting them online. For example, Marcy’s account of traveling to Sweden on her own was deleted from her blog before was posted to the class website: “They edited them. If there was something bad, they would take it out.”  Marcy explained that she believed the professors did not want parents or administrators to read about how students had to travel alone and talk to “strangers” in order to find their way. Even though Marcy’s blog post talked about how she, someone who suffers from severe social anxiety, was empowered by the journey, she felt that the professors feared that her story could spark negative thoughts about their ability to keep the students safe.

While Rachel’s blogs were not edited by the professors, she commented on the “redact[ing]” of her classmates’ reflections:

That wasn’t fair. The blogs were supposed to be about what we did—what we’ve taken from this.  So when they’re redacting things that they don’t want other people to know, it’s like, well isn’t this part of the trip? What I’m supposed to use to reflect on?

It seemed clear that the students believed that the professors were under intense pressure to “perform,” and that these pressures negatively influenced the ways in which the students were able to reflect. While some students, like Marcy, expressed frustration toward the professors themselves, Emily spoke about the professors’ influence in very understanding, and even empathetic terms. Emily purposefully changed the content of her own blogs to show respect for the professors and the pressures they were under:

In the presentations, I’m in front of a bunch of adults who feel like they’ve given me an experience that I can really take something back to my country and apply to my life and my future career. [The professor] wanted them recorded, so for [them], that was [their] take home from the trip and everything [they] had worked for. They want to see how much they helped you and influenced you. It’s a respect thing for me. Because you don’t want to embarrass the people who brought you. You don’t want to embarrass [the university].

To Emily, the professors deserved presentations that highlighted the influence that they had on the students’ learning. Emily chose not to include learning outcomes or learning moments that had not been directly facilitated by the professors. She felt that not highlighting what she had learned academically could “embarrass” the university, noting how teachers are often pressured to show what their students learned that directly connects with the course. As Emily astutely noted, if her professors could not demonstrate that students met the academic objectives of the program, then they likely would not be permitted to run the program again in the future. Emily did not want to be the reason why.

Discussion

It is worth reiterating that when I approached the professors about conducting a study on learning outcomes with their students, they expressed concern that I might negatively influence the students’ behavior throughout the program. They stated that the program was “[the students’] experience,” and that the students should be able to “get what they want out of it.” Their worries were valid and well-received, which is why I took extra steps to ensure that students understood the purpose of the study and their right to refuse participation. The professors’ request acknowledges their recognition of power relations in the classroom, which is why these findings were so surprising to me.

Although from a single case study, the findings presented here provide a narrative that speaks to how teacher influence can adversely affect student authenticity and voice. While literature points out that power imbalances often influence people in ways that they are not even aware (e.g., Markus & Bjorn-Andersen, 1987), this study presents a case in which students recognized and were responding to teacher influences by deliberately misrepresenting their experiences.

“How is someone going to know how my experience truly was unless you get my honest opinion and point of view on it?” Danielle’s question from the beginning of this paper highlights the importance of understanding how student authenticity and voice can be influenced by teachers; if students do not feel free to discuss their learning experiences, how can we as educators and researchers come to know what or how they learned? The reflective assignments devised, facilitated, and monitored by the professors, while undoubtedly pedagogically sound and well-intentioned, were unable to capture authentic representations (Crapanzano, 1999) of their students’ experiences abroad. As Danielle states above, her resulting “edited” and “censored” reflections led to inaccurate and incomplete understandings about her experiences, which leads us as educators to incorrect conclusions about her learning outcomes.

First, many students felt incapable of sharing accurate and complete accounts of their learning experiences lest they not earn a good grade on the assignments. Danielle and Dana both expressed that they carefully curated the content of their assignments to make their work “A-worthy.” This finding supports previous research on “pleasing the professor” (Bower, 2003; Smith, 2000; Cleary, 1996), and underscores how this type of teacher influence in the form of assigning grades can limit what students share and thus, how they can be heard (Mitra, 2006).

Second, students expressed that the professors’ narrow conceptions of what constitutes “learning” differed from the kinds of learning they felt was most important and limited what they felt able to write about. This is an important tension to be explored. On the one hand, the assignments were likely crafted with the goal of understanding what students learned in relation to the course objectives, which is expected in academia. Assessing students’ achievement in relation to course objectives is necessary for evaluating the efficacy of a course. On the other hand, students expressed that the non-academic growth—personal, social-emotional—they experienced abroad was much more impactful than their academic learning. Thus, while the assignments may have accurately captured student academic learning to some extent, they seemingly failed to capture: a) important student social-emotional learning, and b) authentic descriptions of how students learned these lessons. What resulted were mechanical, formulaic (Casey & Hemenway, 2001), and sometimes fabricated representations of how X course activity contributed to Y learning outcome, a form of self-censorship that limits what both educators and students can come to know about student learning (Roberts & Nason, 2011).

This finding highlights some important and underexplored tensions in assessment and academia more broadly: What constitutes “learning”?; Should academic and non-academic forms of learning have equal value in education?; and, How can learning be assessed more holistically? These are areas for future research. It is not my intention to argue that professors should not place restrictions on what students should write about—professors need to assess academic learning, and do not have time to investigate everything students learn. However, it is important to recognize that the value commonly placed on academic learning may not be matched by the students’ values, particularly when their “other” learning experiences seem more salient and impactful.

Finally, perceived institutional and external pressures on the professors influenced the authenticity and voice of the students. Pressures to demonstrate efficacy and impact are increasingly prevalent in academia (Cowan, 2013), and these pressures can have adverse effects as teachers are tasked with demonstrating student achievement in their courses. These findings, like Marcy’s example, support previous research on how and why teachers may reshape the content of student work in response to such pressures (Lemke, 1990; Reinsvold & Cochran, 2012). Institutional pressures also influence students to show their learning in ways that the academy values. Students like Emily felt the need to alter the content of their reflective assignments so they would not “embarrass” anyone.

This final theme—responding to institutional pressures—likely relates to the first two. Because educators are pressured to demonstrate that their courses lead to student learning, they likely craft assignments to elicit student responses around course concepts, and because the guidelines of these assignments are restrictive, students may feel pressured to only write about things that will earn them good grades. These institutional pressures may be contributing to the stifling of student authenticity and voice, which limits what educators can come to know about student learning. This study suggests that students may have much more to share about their learning, especially incidental (Marsick & Watkins, 2001) and social-emotional learning, but that some assessment methods are not well-suited to capture those experiences, and that some considerations of what “counts” as important learning can limit student expression.

There is an important caveat to address in this paper. Some students were angry; they expressed deep frustration that they were silenced, suppressed, and censored. However, much of this frustration likely stems from the fact that academia is inherently hierarchical. Administrators have power over professors. Professors have power over students. For example, professors must place some limits on student writing, lest they have to read lengthy responses that may not demonstrate the kinds of learning they are attempting to facilitate and assess. This can seem limiting to students who recognize that their learning stretched beyond the course objectives, but it is important to recognize that, in these assessment contexts, not all “silences” are bad. In fact, they can be necessary for understanding specific kinds of learning. Future research should explore ways to balance this tension between anticipated and incidental learning and how to give students spaces to share other types of learning.

Implications for Study Abroad

Though relevant for other contexts, these findings have strong implications for study abroad programming. First, if students feel unable to share the breadth of their experiences and the learning that results from them, then professors and researchers can only capture limited understandings about learning outcomes. This is particularly troubling for the study abroad field, as programs are often criticized for facilitating limited learning gains (see e.g., Charbonneau, 2013; Eyler & Giles, 1999; Marklein, 2004). Second, the context of study abroad programming, in which students are inherently experiencing new cultural contexts, provides unique opportunities for facilitating student learning beyond academic content. Future assessment efforts should give due attention to such social-emotional and incidental learning in these contexts.  Finally, authentically understanding what and how students learn is important from a curriculum and program design perspective. If students feel the need to lie to educators about what and how they learn, then the resulting misunderstandings will inhibit the kinds of data-driven decision making that educators can make. Mitigating such disconnects is important.

Positionality in Student Voice Work

While not necessarily a limitation of this research, positionality plays an important and potentially limiting role in student voice research (Mitra & Gross, 2009). Much of what I was able to come to know about the students’ experiences here related to my positionality. I was the (relatively) “powerless” ethnographer. Students explained that their openness with me, as opposed to their professors, partly resulted from my lack of power over them. I was not “assessing” them (even though I was informally assessing them in my larger study; Johnson, 2017). So, I was invited to join them during their free time. They added me to their “secret” group chat where they discussed things they did not want the professors to know (things that, as I discovered during the interviews and focus groups, often resulted in significant learning). Of course, educators must assess their students in one form or another, so obtaining this particular positionality can be elusive for educators. However, acknowledging positionality and cultivating more relational (Greene, 2001) environments may be a way for educators to counteract students’ tendencies to close them off from more authentic accounts of their experiences and learning.

 “Authenticity” of Student Voice

The PCI method, while allowing for more subjective conversations surrounding students’ experiences, still may not capture authentic representations of student learning. As a constructivist researcher, I do not believe that the representations of reality that students shared with me were the full “Truth”—although the students and I do see them as more authentic representations than those shared with the professors. True representations of reality cannot be accessed, and may not even exist (Crapanzano, 1999).

To illustrate this limitation, I return to Marcy. When Marcy’s interview began, she stated, “I think I sent you the wrong photo.” It was a panorama of Gothenburg (Marcy, Photo #1, shown below). The photo Marcy intended to send was the same photo, but with a wider view that included her classmate Rachel “scarfing down a candy bar” in the corner (Marcy, Photo #2, shown below).

Figure 10

Figure 10—Marcy, Photo #1: Edited Panorama of Gothenburg

Figure 11

Figure 11—Marcy, Photo #2: Unedited Panorama of Gothenburg

Marcy explained that when people go abroad they “curate what [they] think is beautiful;” the experiences they talk about and reflect upon are the good things, often the edited things, and these reflections often do not depict an authentic “reality.” Rachel eating the candy bar was a perfect metaphor for Marcy’s trip—ridiculous, messy, “derpy”—but that metaphor was not one that she, or the professors (as she perceived), wanted as a representation of her experience. Instead, the “curated” panorama was chosen because it was more beautiful, even if less authentic.

The students’ photos and stories are themselves representations. While I did not specifically request original, unedited photos, I was struck when several students submitted photos that were filtered and edited. Angela sent me her images, but then later emailed “fixed” ones, noting how the improved lighting made it look better (Angela, Photo #1, shown below). Brandon altered his photo of a quaint street lined with shops and old cars in Stockholm’s Old City using Instagram’s vintage filter, adding that it made it seem more like he was stepping back in time: “That’s the one I posted [online]” (Brandon, Photo #2, shown below).

Figure 12

Figure 12—Angela, Photo #1: Atop a Mountain

Figure 13

Figure 13—Brandon, Photo #2: Street in Stockholm’s Old Town

The students’ comments and edited photos highlight the breadth of external influences, beyond teacher influence, on their authenticity and voice in representing their experiences. This limitation of accessing “truly authentic” representations certainly impacts how researchers can understand student experiences and what they are able to report about them. However, acknowledging that there are representations of reality that I may not have accessed is an important step in attempting to understand student learning. Researchers should constantly cast a critical eye over the data they gather and acknowledge that the representations of reality they were presented with may not be the most authentic representations available (Crapanzano, 1999).

Are Authentic Representations Best?

Much of the learning described by students resulted from activities and experiences that fell outside of the program’s planned itinerary and curriculum. Students gleaned important life lessons from these incidental (Marsick & Watkins, 2001) experiences, and they strongly lamented their inability to share and reflect on them in their assignments. However, it should also be noted that some of these incidental learning moments involved somewhat illicit activities and behaviors, such as drinking and clubbing. It makes sense that students did not feel comfortable talking about these experiences in their assignments or with professors. I mention these types of experiences, not to highlight the debauchery for which study abroad is sometimes criticized, but because these students were eager to talk with me about how these experiences meant something. From learning self-control to empowerment to responsibility to cross-cultural skills, many students walked away from these experiences having learned something meaningful. While educators should not encourage students to participate in such activities, they should acknowledge that students sometimes do engage in these activities, and that important learning can occur when they do. However, in order for that learning to occur, students need to be given spaces to talk about these experiences and to reflect upon them (Dewey, 1933; Van Manen, 1990). Finding a balance between validating and valorizing such experiences is a skill that educators would do well to practice.

Conclusion

It is easy to see that educators impact the lives of their students. But, these impacts may occur in ways that educators may not realize or intend, and they may not be positive. Teachers must, therefore, practice greater reflexivity that examines how they influence their students’ authenticity and voice, particularly as they attempt to assess what and how students learn. Educators must work to understand how students perceive and define their situation—i.e., the power dynamics between the teacher and the student—before they can make sense of their reflections on it (Delamont, 1976). Teachers should ask themselves: What processes have helped produce these representations of student experiences? How have I influenced this representation? Are there other possible representations that are not being shared? And, how can these other representations be better accessed?

Minimizing negative teacher influence on student authenticity and voice is not an easy task. However, teachers would do well to understand the influences they may have on their students and to develop strategies for enhancing and empowering, not stifling or suppressing, student authenticity and voice. Establishing more open dialogue between teachers and students can help. For example, teachers can acknowledge and value other kinds of learning to encourage students to more freely share their experiences. This can be facilitated through informal reflective discussions, which mitigates the institutional pressure to demonstrate learning by leaving course assignments to focus more on course content while still providing students an outlet to share other experiences.

While this paper focuses on the influence of teachers on students, educational institutions should consider the influence that their policies and practices can have on teachers and students as well. Assessment cultures and increasing accountability measures, while intended to have positive impacts on student learning, can also put negative pressure on teachers, which can in turn negatively affect students. The professors in this case study were, in many ways, reacting to this pressure, wanting to impress the administrators and parents who had access to the students’ assignments. They were in no way “bad” professors; they were professors who seemed to be under immense pressure to perform.

This study also implies that educators and researchers may need a greater focus on authenticity when examining learning. The students in this study explicitly stated that what they shared with their professors about their experiences was largely inauthentic:fake, sugar-coated, incomplete, redacted, edited,bullshit,etc. Many of the stories they told in their assignments—read by professors, administrators, and parents—purposefully did not provide authentic accounts of their experiences. This inauthenticity obscured a large portion of what students learned during this program from these audiences. Understanding how to mitigate such an effect in educational settings is important for being able to capture more authentic representations of student learning.

It is not my attempt in this paper to demonize teachers for stifling student voices. Rather, my broader goal is to highlight how student learning is likely occurring in ways that current research often does not see, and that rethinking teacher influence to empower student authenticity and voice can help illuminate, and even facilitate, student learning. Since all experiences are pre-reflective (Van Manen, 1990), students need to be given spaces to reflect on these experiences so that meaningful learning can transpire. Students seem to be learning quite a bit abroad, but they need to be able to talk about it.

The issues highlighted in this paper are not unique to study abroad contexts. Ultimately, educators writ large should think carefully about how to productively work on power imbalances in and outside of the classroom. Working to turn power dynamics on their head and equitably wielding the influence educators inherently hold can elicit more authentic representations of student experiences, and demonstrate to students that they are committed to providing spaces that best facilitate their learning. Reflection can be “practiced, assessed, and perfected” (Rodgers, 2002), but it can also be suppressed, censored, and fabricated, and until educators acknowledge this and demonstrate their desire to hear and understand the whole of student experiences, assessment efforts based on reflective tasks may be misrepresentative of what students learn. Students have much to share, and they want to “be heard” (Mitra, 2006), but educators must give them spaces to speak freely, and they must be ready to really listen.

Discussion Questions

  1. In what ways might teachers minimize their “negative influence” in reflective assignments to increase student authenticity and voice?
  2. How can educators work to balance student-teacher power dynamics?
  3. How might students share their thoughts on authenticity and voice, and how can educators provide spaces for this kind of feedback?
  4. (How) can “authentic” representations of experiences be facilitated?
  5. The PCI Method is novel, and still being developed. What do you see as the strengths and weaknesses to using PCI as an interpretive assessment method in education?

 

Author Biography

Kayla Johnson is an Affiliate Scholar with the Center for the Study of Higher Education and an Instructor of Higher Education at Penn State University. Her research focuses on student learning and development, culturally responsive curriculum design, assessment and evaluation, and photo-based research methods.

References

Adler, P. A., & Adler, P. (1994). Observational techniques. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 377-392). Thousand Oaks, CA: Sage.

Anderson, P. H. & Lawton, L. (2011). Intercultural development: Study abroad vs. on-campus study. Frontiers: The Interdisciplinary Journal of Study Abroad, 21,86-108.

Apple, M. W. (1995). Education and power(2nded.). New York, NY: Routledge.

Bathke, A., & Kim, R. (2016). Keep calm and go abroad: The effect of learning abroad on student mental health. Frontiers: The Interdisciplinary Journal of Study Abroad, 27(Spring), 1-16.

Bianchi, J. A. (1997). Where knowledge construction, equity, and context intersect: Student learning of science in small groups. Journal of Research in Science Teaching, 34(10), 1039-1065.

Bower, L. L. (2003). Student reflection and critical thinking: A rhetorical analysis of 88 portfolio cover letters. Journal of Basic Writing, 22(2), 47-66.

Caldwell, P., & Purtzer, M. A. (2015). Long-term learning in a short-term study abroad program: “Are we really truly helping the community?” Public Health Nursing, 35(5), 577-583.

Carrillo, R. (2014). The effectiveness of short-term study abroad programs in the development of intercultural competence among participating CSU students. Retrieved from https://dev.csustan.edu/sites/default/files/honors/documents/journals/Stirrings/Carrillo.pdf

Casey, M., & Hemenway, S. I. (2001). Structure and freedom: Achieving a balanced writing curriculum. The English Journal, 90(6), 68-75.

Charbonneau, L. (2013, March 18). The (ambiguous) benefits of short-term study abroad. University Affairs. Retrieved from http://www.universityaffairs.ca/opinion/margin-notes/the-ambiguous-benefits-of-short-term-study-abroad/

Cleary, L. M. (1996). “I think I know what my teachers want now”: Gender and writing motivation. The English Journal, 85(1), 50-57.

Corbin, J., & Strauss, A. (2015). Basics of qualitative research: Techniques and procedures for developing grounded theory(4thed.). Thousand Oaks, CA: SAGE Publications, Inc.

Cowan, K. (2013, December 15). Higher education’s higher accountability. ACE: American Council on Education. Retrieved from http://www.acenet.edu/the-presidency/columns-and-features/Pages/Higher-Education’s-Higher-Accountability.aspx

Crapanzano, V. (1999). Reflections. Ethos, 27(1), 74-88.

DeGraaf, D., Slagter, C., Larsen, K., & Ditta, E. (2013). The long-term personal and professional impacts of participation in a study abroad program. Frontiers: The Interdisciplinary Journal of Study Abroad, 23(Fall), 42-59.

Delamont, S. (1976). Interaction in the classroom. London: Methuen.

Denzin, N. K., & Lincoln, Y. S. (2018). The SAGE handbook of qualitative research. Thousand Oaks, CA: SAGE Publications.

Dewey, J. (1933). How we think. Buffalo, NY: Prometheus Books.

Dietz, S. S., Dotson-Blake, K. P., Enselman, D., Jones, L., Sexton Jr., E. H., Waller, M. S., & Richardson, M. E. (2017). An international learning experience: Looking at multicultural competence development through the lens of relational-cultural theory. Journal of Counselor Practice, 8(1), 22-44.

Donnelly-Smith, L. (2009). Global learning through short-term study abroad. Peer Review, 11(4), 12-15.

Emerson, R. M., Fretz, R. I., & Shaw, L. L. (2011). Writing ethnographic fieldnotes(2nded.). Chicago, IL: University of Chicago Press.

Eyler, J., & Giles, D. E. (1999). Where’s the learning in service-learning? San Francisco, CA: Jossey-Bass, Inc.

Gibson, B. E., Mistry, B., Smith, B., Yoshida, K. K., Abbott, D., Lindsay, S., & Hamdani, Y. (2013). The integrated use of audio diaries, photography, and interviews in research with disabled young men. International Journal of Qualitative Methods, 12, 382-402.

Goetz, L. A., & Holliday, W. (2017). Designing programs to foster intercultural competence through interdisciplinary study abroad. NECTFL Review, 80, 11-45.

Gold, R. (1958). Roles in sociological field observations. Social Forces, 36, 271-223.

Gonzalez, T. E., Hernandez-Saca, D. I., & Artiles, A. J. (2017). In search of voice: Theory and methods in K-12 student voice research in the US, 1990-2010. Educational Review, 69(4), 451-473.

Greene, J. C. (2001). Dialogue in evaluation: A relational perspective. Evaluation, 7(2), 181-187.

Heinzmann, S., Künzle, R., Schallhart, N., & Müller, M. (2015). The effect of study abroad on intercultural competence: Results from a longitudinal quasi-experimental study.Frontiers: The Interdisciplinary Journal of Study Abroad, 26(Fall), 187-208.

Johnson, K. M. (2017). “Finding yourself in a foreign country:” The learning experiences of students in an embedded short-term study abroad program. (Unpublished doctoral dissertation). Pennsylvania State University, State College, PA.

Katz, C. (1992). All the world is staged: Intellectuals and the projects of ethnography. Environment and Planning: Society and Space, 10(5), 495-510.

Kinginger, C. (2013). Social and cultural aspects of language learning in study abroad. Amsterdam: John Benjamins Publishing.

Kinginger, C. (2017). Second language learning in a study abroad context. Second and Foreign Language Education, 125-136.

Kuh, G. D. (2008). High‐impact educational practices. Washington, DC: American Association for Colleges & Universities.

Kuh, G. D., Kinzie, J., Schuh, J. H., & Whitt, E. J. (2010). Student success in college: Creating conditions that matter. San Francisco, CA: Jossey-Boss.

Lee, J., & Green, Q. (2016). Unique opportunities: Influence of study abroad on black students. Frontiers: The Interdisciplinary Journal of Study Abroad, 28(November 2016), 61-77.

Lemke, J. L. (1990). Talking science: Language, learning, and values. Westport, CN: Ablex Publishing.

Levine, A., & Dean, D. R. (2012). Generation on a tightrope: A portrait of today’s college student. San Francisco, CA: John Wiley & Sons, Inc.

Levinson, B. A. (2011). Beyond critique: Exploring critical social theories and education. Boulder, CO: Paradigm Publishers.

Levitan, J. (2018). The danger of a single theory: Understanding students’ voices and social justice in the Peruvian Andes. Teachers College Record, 120(3).

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.

Lindsey, E. W. (2005). Study abroad and values development in social work students. Journal of Social Work Education, 41(2), 229-249.

Malinowski, B. (1922). Argonauts of the Western Pacific: An account of native enterprise and adventure in the archipelagoes of Melanesian New Guinea. New York: E. P. Dutton & Co.

Marklein, M. B. (2004, February). Study abroad: The short-course: Students like quick studies, but critics question their direction. USA Today. Retrieved from http://usatoday30.usatoday.com/educate/college/education/articles/20040208.htm

Markus, M. L., & Bjorn-Andersen, N. (1987). Power over users: Its exercise by system professionals. Communications of the ACM, 30(6), 498-504.

Marsick, V. J., & Watkins, K. (2001). Informal and incidental learning. New Directions for Adult and Continuing Education, 89, 25-34.

Mitra, D. (2003). Student voice in school reform: Reframing student-teacher relationships. McGill Journal of Education, 38(2), 289-304.

Mitra, D. (2006). Increasing student voice and moving toward youth leadership. The Prevention Researcher, 13(1), 7-10.

Mitra, D., & Gross, S. J. (2009). Increasing student voice in high school reform: Building partnerships, improving outcomes. Educational Management Administration and Leadership, 37(4), 522-543.

National Survey of Student Engagement (NSSE). (2007). Experiences that matter: Enhancing student learning and success. Bloomington, IN: Indiana University Center for Postsecondary Research. Retrieved from: http://nsse.iub.edu/nsse_2007_annual_report/

Ogden, A. C. (2017). Foundational reading for education abroad scholar-practitioners. Retrieved from http://educationabroad.isp.msu.edu/files/8614/9925/6468/Foundational_Reading_EAProfessionals.pdf

Oldfather, P. (1995). Songs “come back to most of them”: Students’ experiences as researchers. Theory into Practice, 34(2), 131.

Reinsvold, L. A., & Cochran, K. F. (2012). Power dynamics and questioning in elementary science classrooms. Journal of Science Teacher Education, 23(7), 745-768.

Roberts, A., & Nason, R. (2011). Nobody says no: Student self-censorship in a collaborative knowledge building activity. Journal of Learning Design, 4(4), 56-68.

Rodgers, C. R. (2006). Attending to student voice:The impact of descriptive feedback on learning and teaching. Curriculum Inquiry, 36(2), 209-237.

Salisbury, M. H., An, B. P., & Pascarella, E. T. (2013). The effect of study abroad on intercultural competence among undergraduate college students. Journal of Student Affairs Research and Practice, 50(1).

Scott, P. H., Mortimer, E. F., & Aguiar, O. G. (2006). The tension between authoritative and dialogic discourse: A fundamental exercise of meaning making interactions in high school science lessons. Science Education, 90(4), 605-631.

Shepardson, D. P., & Britsch, S. J. (2006). Zones of interaction: Differential access to elementary science discourse.Journal of Research in Science Teaching, 43(5), 443-466.

Smith, J. B. (2000). Know your knowledge: Journals and self-assessment. In J. B. Smith & K. B. Yancey (Eds.), Self-assessment and development in writing: A collaborative inquiry. Cresskill, NJ: Hampton Press.

Spaulding, C. L. (1995). Teachers’ psychological presence on students’ writing task engagement. The Journal of Educational Research, 88(4), 210-219.

Spradley, J. P. (1979). The ethnographic interview. Long Grove, IL: Waveland Press.

Spradley, J. P. (1980). Participant observation. Long Grove, IL: Waveland Press.

The Forum on Education Abroad. (2011). Code of ethics for education abroad(2nded.). Carlisle, PA: The Forum on Education Abroad. Retrieved from http://forumea.org/wp-content/uploads/2014/10/ForumEA-CodeofEthics2011-2ndEdition.pdf.

Tobin, J. J., Wu, D. Y. H., & Davidson, D. H. (1989). Preschool in three cultures: Japan, China and the United States. New Haven, CT: Yale University Press.

van Manen, M. (1990). Researching lived experience: Human science for an action sensitive pedagogy. Albany, NY: SUNY Press.

Vande Berg, M., Paige, R. M., & Lou, K. H. (2012). Student learning abroad: Paradigms and assumptions. In M. Vande Berg, R. M. Paige, & K. H. Lou (Eds.), Student learning abroad: What students are learning, what they’re not, and what we can do about it (pp. 3-28). Sterling, VA: Stylus Publishing, LLC.

Vatalaro, A., Szente, J., & Levin, J. (2015). Transformative learning of pre-service teachers during study abroad in Reggio Emilia, Italy: A case study. Journal of the Scholarship of Teaching and Learning, 15(2), 42-55.

 

 

 

APPENDIX A
(Taken from course syllabus)

Daily In-Country Blog Posts: “Blogs are meant to enhance learning and allow students to voice their opinion on a topic. These are typically one-page responses to a question posed to the class or a reflection on learnings or experiences. They are more experiential, but should be taken seriously. Blogs should be completed online daily while in country.”

In-Country Presentations: Later during the visit, you will be asked to give a second 6-8 minute PowerPoint presentation to the host university faculty and students. You will work on this presentation while in-country and will work in a group with one or two other students. The presentation should be a reflection of your time in Sweden, what you learned about [Human Resource Management], comparison to the USA, and your personal experiences. The grade allocated will be for the pair/team as a whole.

Post-Program Video Wrap-Up Presentations: Students will submit a final video summary of their experiences via ANGEL by two weeks after the trip to Sweden ends. The summary should be 7-10 minutes in length and should reflect on topics such as comparison of initial expectations to actual experiences, main concepts learned and evidence of that learning, thoughts and feelings on own cultural identity, thoughts and feelings on Sweden cultural identity, comparison of the two HRM systems, thoughts on HRM policies, and plans to incorporate this experience into future endeavors. Summaries should include audio and video using VoiceThread or YouTube. In addition, for this embedded study abroadprogram, students should also submit in the drop box slides based on the format of their oral video.

 

 

Johnsonv3i4.pdf