2 Art in Engineering as Open Educational Resources

Chapter Overview

“Flow Visualization: The Art and Physics of Fluid Flow” is a 20-year-old technical elective in the Mechanical Engineering program at the University of Colorado Boulder. I developed the course to give both engineering and art students the opportunity to explore the interface between art and science, an interface that is open to everyone but which is often not valued in engineering education. In this course (abbreviated here as the Flow Vis course), students make aesthetic visualizations (photographic images or videos) that illustrate the physics of gasses and liquids. Students then write reports that describe the fluid physics and the methods of making the visualization in sufficient detail so that other students can duplicate the visualization. Students are instructed in optics, photography, specific flow visualization techniques, technical communication, and critique. What is not included is also important: students are not told to use specific techniques or study specific fluid physics. Instead, they are asked to be creative, play with fluids and photography, and let the art drive the science. They find their own aesthetic visions in the process.

This course exemplifies open pedagogy in that students make use of open educational resources (OER) and produce them. All student work is published on a high visibility and archival website, https://Flowvis.org, under a Creative Commons license. Together, the visualization and report make the knowledge students acquired during each formative assessment available on a global basis. Furthermore, all course materials, including syllabus, schedule, lecture notes and lecture video recordings, assignment details, and an OER textbook by the instructor (Hertzberg, 2024a) using student work as examples, are published on the same website. In this way, the instructor and students have co-created an OER.

Rationale

Open pedagogy encompasses various practices that share common features, as noted by Clinton-Lisell (2021), including the emphasis on students producing innovative and valuable artifacts that extend beyond mere learning, often involving renewable assignments intended for public sharing and open licensing. The definitions of renewable assignments (Wiley & Hilton, 2018) and non-disposable assignments (NDAs) (Seraphin et al., 2019) overlap to a great extent. Wiley and Hilton (2018) outline four criteria for open pedagogy: (a) The student creates an artifact, (b) The artifact holds value beyond supporting the student’s learning, (c) The artifact is made public, and (d) The artifact is openly licensed. These criteria are all met in the Flow Vis course, as discussed below. Additionally, Seraphin et al. (2019) provide further criteria, including revision, creativity, modification of objectives, cooperative critique, and innovation potential.

In contrast to typical ‘hands-on’ engineering courses where students follow specific lab procedures or work on proprietary client-driven problems, students in Flow Vis choose their study topics and methods. First, students are given an open-ended prompt to create a photograph or video of physical phenomena featuring gasses or liquids. In truth, students often begin with only a vague idea of what to do, perhaps inspired by a previous student’s visualization or a strange YouTube video. For example, recently, students have been fascinated by “Elephant’s Toothpaste,” a startling eruption of foam from a mixture of hydrogen peroxide, potassium iodine, and soap (Rober, 2019).

They then assemble their apparatus out of their daily environment: kitchens, bathrooms, and the sky outside. Students also have access to university resources such as large flow facilities like flumes and wind tunnels, some research laboratories, various small fluids demonstrations, and photographic equipment. The nonlinearity of fluid flows practically guarantees they won’t be able to replicate their inspiration exactly. Instead, students create a novel result that illustrates a phenomenon that may have escaped scientific scrutiny. Much of the science of fluid mechanics has been driven by military and industrial applications, limiting its scope. In contrast, student work motivated by aesthetics has often generated new knowledge.

Simply setting students free to explore satisfies two criteria for a non-disposable assignment: modification of objectives and chance innovation (Seraphin et al., 2019). Using such non-disposable assignments is rare in engineering education; instead, faculty jealously guard solutions to exams and homework sets in the hope of preventing plagiarism. Exceptions may be found in freshman and senior design courses, but public dissemination of project outcomes is still rare. In the Flow Vis course, the opposite approach is used. Students are expected to be inspired by and build upon previous students’ work and contribute their work for the use of future students and the public.

The Flow Vis course heavily emphasizes renewable assignments, requiring students to complete six projects that feature an image or video created with artistic and intellectual control, along with a detailed report explaining the visualization process and the physics illustrated. This report is essential for ensuring the scientific usefulness of the work and is a reasonable expectation for all students. Additionally, this activity aligns with the concept of “building OERs with your students” (DeRosa & Jhangiani, 2017, para. 17), as students contribute bite-sized, stand-alone resources.

Collaborative learning, a key aspect of open pedagogy, is implemented in two ways in each course assessment. The first is an unusual modification of a traditional approach. Traditionally, engineering students are placed in teams to accomplish a project and collaborate to produce a single deliverable package. Each student’s performance on the team may or may not be graded individually, and both approaches are problematic. For example, it is often difficult to identify a student’s contribution to the project. In Flow Vis, students are placed in teams for mutual support but are expected to produce unique work. Since this differs significantly from their prior team experiences, the following team expectations and reasons for collaboration are explicitly outlined for students:

1. To allow students to attempt imaging more complex flow phenomena by distributing the work of developing a setup across the team, enabling the exploration of more challenging experiments.
2. To allow students to try more advanced imaging techniques by pooling photographic and fluid expertise and equipment within the team.
3. To encourage collaboration, allowing students to bounce ideas off one another, fostering creativity and idea generation.
4. To offer informal feedback from team members on each student’s work.
5. To enable students to interact with peers from diverse backgrounds (Hertzberg, 2024b).

The second area where collaborative learning is implemented in the Flow Vis course is via formal peer critique using the Critical Response Process (Lerman, 2002). Students are trained in a highly supportive and constructive critique method. In this oral critique process, students present their work to small groups during class and get feedback.

In addition to the oral feedback, students review two peers’ written reports using the “Rubric for Self and Peer Assessment,” a check-plus style rubric (see Appendix A), and are then encouraged to revise and repost both their visualizations and reports. This satisfies two more criteria for open pedagogy: collaborative learning and opportunities for revision (Seraphin et al., 2019, p. 86). Importantly, participation in the critique is mandatory but does not impact the course grade, ensuring that critique remains a formative, learning-centered activity. Further details on the critique process, including roles and facilitation, are described in the ‘Peer and Expert Critiques’ section below.

Assessment Description

Students complete six assignments spread over the semester. These assignments have similar learning objectives and share a common formative assessment structure, including critique format and report rubric. The first assignment, ‘Get Wet,’ will be described in detail.

The learning objectives for the course comprise these common elements of each assignment’s objectives:

1. Train your perception of fluid physics in the real world.
2. Employ aesthetics (art) as a valid method for exploring fluid physics.
3. Demonstrate the ability to communicate aesthetics and fluid physics to a wide audience.
4. Critique your peers using supportive but substantive techniques.
5. Demonstrate basic skills in scientific and artistic imaging: focus, exposure, and composition.
6. Design and analyze flow visualization experiments.

Of the six assignments, three are completed as individuals and three as team members. Early in the semester, students are surveyed using the CATME platform for their schedules, demographics, photography experience, and equipment. CATME algorithmically creates teams based on the instructor’s parameters, such as similarity of schedules and dissimilarity of photography experience. ‘Get Wet’ is completed individually, as are two ‘cloud’ assignments that involve photographing atmospheric clouds throughout the semester. The other three assignments are done in teams, where students assist each other in creating visualizations, although each student is still responsible for a unique visualization and report.

Part A: The First Assignment ‘Get Wet’

On the first day of class, students receive a list of initial assignments, including logistical tasks like reading the syllabus and joining the Slack workspace, along with two substantive assignments: ‘Get Wet’ and ‘Clouds First.’ The ‘Get Wet’ assignment focuses on recognizing the challenges of creating an accurate and aesthetically pleasing flow visualization. Students are encouraged to “get their feet wet” by capturing images or videos of fluids (air, water, gas, or liquid) that effectively demonstrate observed phenomena while also being visually appealing. They can use any familiar imaging technique—analog or digital, still or video, in black and white or color. Additional instructions guide students on publishing their work on Flowvis.org and submitting an archival version to Canvas, the university’s learning management system. This assignment is due in the third week of the semester, and lectures during that period are designed to support completion. They cover flow visualization techniques, camera technologies, and image editing tutorials.

Part C: The Written Report for ‘Get Wet’ Assignment

The six visualization assignments are critiqued two days after the image/video is submitted online to both Flowvis.org and Canvas. Students are assigned to small ‘critique pods’ of 8 to 10. The pods meet simultaneously in Zoom breakout rooms during the regular class period. The students perform most of the critique and are very successful at addressing the aesthetic and photographic aspects of the work. Still, even the most advanced Mechanical Engineering student does not have the background to assess the wide range of fluid physics that students reveal in their work. Thus, an expert in fluid mechanics participates as well. The expert’s role is to help the artist (the presenting student) describe the fluid physics in the visualization and provide keywords for the artist to use in researching the physics for the written report, due a week later. The experts are a mix of paid and volunteer research and science communication professionals drawn from university faculties and research laboratories. Every student’s work is critiqued using the Critical Response Process (Lerman, 2002). A facilitator moderates a discussion between the artist and the responders (students in the audience). Students each take a turn as facilitators at least once over the course of the semester and thus gain experience with all three roles. The discussion is structured in five stages:

1. Artist presents the work. Students briefly describe their flow and photography setup.
2. Statements of meaning (usually positive). Responders are given the following prompts for this stage when the method is introduced:
   1. What does this image/video say about fluids? What is being shown?
   2. What does this image/video say about aesthetics? Does it strike you with beauty, power, destruction, oddness, or other aesthetic?
   3. What does this image/video say about the imaging technique? Does it impress you or inspire questions?
   4. Are there other meanings in the image/video?
   5. If making a positive comment, be honest and specific. What did you like and why? Do not just say, ‘good job.’
3. Questions from the artist to the responders. The student presenting is expected to ask for specific feedback to guide further development of the work, such as:
   1. What do you think of the way the image is cropped?
   2. What about how color is presented in the image?
   3. Did you notice where the light pole is edited out?

Students are told “Don’t ask just ‘what do you think’; that’s too vague. You will get more useful answers if your question is focused.”

4. Neutral questions (questions without embedded opinions) from the responders to the artist. This stage is the most difficult for responders, sometimes requiring the facilitator to help by rephrasing the question. Students are told, “It is tough to ask a question without embedding an opinion. It will take practice. For example, instead of ‘it’s kind of dark’ or ‘why is it so dark at the bottom?’ ask, ‘what do you think about the balance of light and dark areas?’ Be sure to ask about the fluid physics: ‘why does it look like that?’”
5. Permissioned opinions. Responders name the topic of their opinion and then ask the artist for permission to state it. For example, ‘I have an opinion about the depth of field and the focus. Do you want to hear it?’ The artist can answer yes or no. If they already know that the focus was bad and what to do, they can say ‘no thanks.’ This stage allows responders to voice opinions that they couldn’t fit into other stages of the critique while still giving the artist a measure of control.

The Critical Response Process defuses the natural defensive reaction to criticism of one’s work and makes it easy for audience members to provide truly constructive comments. Engineering students are now more willing to offer remarks in class. In exit surveys, some students objected to the method as being overly structured, while others appreciated the method, noting its utility in other contexts.

Get Wet Part C: The Written Report

Students document their work for each major assignment in a written report, which is added to their post on Flowvis.org and submitted on Canvas two weeks after the oral critiques. They describe their artistic approach and as much of the fluid physics as they can. All students are also expected to explain their setups in sufficient detail so that someone else could recreate the experiment. This level of documentation is normal for engineering students while novel for art students. The importance of such documentation for artists is emphasized, noting that it will improve their control over their tools, techniques, and work.

The course grade uses a labor-based contract grading (“Contract Grading,” 2023) approach, where students are expected to complete their assignments at a level appropriate to their backgrounds, and their course grade is based on completing the work. In support of this approach, I developed the “Rubric for Self and Peer Assessment” for assessing student work based on four categories: photographic technique, whether the artistic intent was achieved, whether a fluid phenomenon was made visible, and whether there was appropriate content in the written report. Students use the “Rubric for Self and Peer Assessment” primarily for self-assessment of the written report and secondarily for peer critique of the reports. Each student is randomly assigned to give peer critiques of two written reports for each of the six major assignments using this rubric; this is easily implemented in Canvas.

Debrief

One of the key benefits of renewable assignments is their positive impact on student motivation (Abri & Dabbagh, 2019; Baran & AlZoubi, 2020; Stancil, 2020). Engineering students are often motivated by the points awarded for completing assignments and exams. However, this type of grading is deliberately avoided in the Flow Vis course. Instead, contract grading (“Contract Grading,” 2023) and ungrading statements (Blackwelder et al., 2020) are used to assess course performance, aiming to promote intrinsic motivation. The agency and empowerment fostered by publication are expected to contribute to this goal (Seraphin et al., 2019). Early in the semester, when surveyed about their motivations, students primarily cited creative freedom and aesthetics rather than publication. The impact of the renewable assignment on motivation later in the semester, after students have engaged with publication, is still unknown. A possible synergy between creating an artifact with scientific and artistic value and its subsequent publication may further enhance motivation. To explore this, a survey is being designed (Scribner et al., 2021).

Student work is reused and remixed locally in the course by students in subsequent semesters. In both conceiving the visualization and writing the report, students use a variety of freely available educational resources, including student work from previous semesters as published on Flowvis.org, as well as other OER such as Wikipedia and YouTube. For example, a student exploring soap films may read a previous student’s description of their apparatus and procedure before attempting their own, then turn to a professional soap film artist’s blog about best practices for lighting. Students are expected to cite all published resources, including copyrighted material from relevant technical literature such as textbooks and professional journals.

In the Flow Visualization course, open licensing has allowed assignments to meet the criteria for renewable assignments, as proposed by Wiley and Hilton (2018). Since 2003, student work has been published on the course website, initially under traditional copyright, and later, in 2016, a shift to Creative Commons licensing was considered. A survey at that time revealed most students favored a more open approach, though 27% expressed concerns about commercial uses and preferred to prevent corporations from profiting from their work. This led to the adoption of a restrictive license (Attribution-NonCommercial-NoDerivatives) to balance student concerns with copyright protections. Interestingly, many students were unaware of the complexities of licensing. Despite the requirement for student work to be published, no alumni (out of 676 surveyed) raised concerns about licensing. However, one student with a background in online security requested their name be removed from their work.

All content on the Flow Vis website is licensed as CC-BY (Attribution Only), aligning with open remixing practices. However, students retain copyright over the highest resolution versions of their images, which are not published on the website. Through a formal Copyright Use Agreement developed with university legal counsel (see Appendix B), students consent to allow academic use of these original images. This agreement has enabled the inclusion of students’ images in textbooks, journal articles, and promotional material for the University of Colorado.

In the Fall of 2024, a survey assessed student familiarity with Creative Commons licenses. While 35% had never heard of Creative Commons and only one student understood license differences, after discussion, 75% supported CC-BY, believing “knowledge should never be restricted,” and 25% favored CC-BY-SA, emphasizing sharing of derivative works.

Appraisal

Student work submitted in response to the first assignment often exceeds expectations, even though some students have no prior fluid mechanics or imaging background and have yet to receive extensive instruction in these areas. This demonstrates the effectiveness of the course’s approach in fostering learning and engagement. For example, Figure 1 illustrates the graceful movement of low-speed fluid flows, specifically showing laminar flow as a falling droplet of cream penetrates the water and then rises due to the Rayleigh-Taylor instability. This phenomenon occurs because unwhipped cream is less dense than water due to its fat content (Rayleigh–Taylor Instability, 2023). The ability of students to create such detailed and accurate depictions despite limited prior knowledge underscores the success of the course’s focus on accessibility and experiential learning.

The artist was Janelle Montoya, an undergraduate mechanical engineering student who had completed a course in fluid mechanics but had no background in imaging. In her report, provided in Appendix C, she describes how the cream illustrates the beauty of the swirling flow and the difficulty of avoiding reflections. She concludes that she satisfied her intent with the accuracy of the image.

Figure 2 presents a striking aesthetic, demonstrating fluid phenomena’s surprising and often paradoxical nature. Although humans intuitively understand fluid flows, the oddness of certain fluid behaviors can still be surprising. For example, Figure 2 contrasts the upraised thumb, symbolizing a positive, joyful reaction, with our instinctual withdrawal from flames, creating an aesthetic of curiosity and contradiction. This represents the exploration of fluid phenomena from a more creative perspective.

However, despite this creative visual approach, Kevin Oh, a Mechanical Engineering undergraduate, must fully integrate this analysis into his report. While the visual work is compelling, the lack of deeper analysis in the written component suggests an area for growth in balancing aesthetic exploration with scientific reasoning. This gap highlights an opportunity for future course improvements in guiding students to connect their creative work with more rigorous analysis.

Open pedagogy is most successful when student work reaches a larger community, and the Flow Vis course exemplifies this through its global audience. The course website, https://FlowVis.org, attracts visitors who are either drawn by the aesthetics of the visualizations or are searching for examples of specific fluid phenomena. Beginning in 2007, the website was the top result on Google for “flow visualization,” maintaining this status for many years before being overtaken by SEO-driven sites like Wikipedia and NASA. The website is still in the top few results and continues to draw a steady stream of traffic, with approximately 10,000 annual visitors since 2018, as shown in Figure 3. In the most recent year (October 2023–September 2024), 30% of the 27,543 views came from non-English-speaking countries, spanning 130 nations. This broad international engagement highlights the reach and success of the course’s open pedagogy approach in promoting global knowledge sharing. Details are provided in Appendix D.

Seraphin et al. (2019) propose a framework for evaluating NDAs based on time, space, and gravity. Flow Visualization assignments excel in these areas. The website has existed for 20 years, with plans for future growth, demonstrating its strong performance on the time scale. On the space scale, the site ranks highly and is used globally. In terms of gravity—defined as the “extent of value/impact” (Seraphin et al., 2019)—the course has had a transformative effect on students, expanding their perception and motivating them to analyze fluid physics and appreciate its aesthetics (Goodman, 2015; Goodman et al., 2018; Pugh, 2011). The impact extends beyond students, as fluid mechanics educators have adapted the course for use at 12 universities, further enhancing the gravity of the Flow Vis NDAs.

The Flow Vis course and website have successfully reached a global audience, with a significant percentage of visitors from non-English-speaking countries. However, there is potential to further amplify the social justice aspects of the course by enhancing its appeal to diverse audiences. One possible improvement is integrating an automatic language translation service to make the content more accessible to non-English speakers. Evaluating the effectiveness of this initiative through website usage statistics could provide valuable insights into its impact on engagement and inclusivity, further strengthening the course’s commitment to accessibility and global participation.

Beyond simple language accessibility, DeRosa and Jhangiani (2017) argue that open pedagogy reduces costs and democratizes knowledge creation, making it accessible to all rather than an elite domain. In the Flow Visualization course, this principle is reflected in the accessibility of course content and the expectation that engineering students engage in artistic practices while art students explore engineering concepts. Inclusivity is further enhanced through assignments focusing on everyday physics, which can be easily observed and understood with minimal equipment and basic mathematics. This approach is particularly crucial for scientific topics, where barriers to participation can be significant. Fluid mechanics, often perceived as a challenging subject by engineering students and the general public, is made more approachable; the complexity lies primarily in mathematics, not the underlying physics.

Conducting critique sessions remotely offers several benefits that enhance accessibility and efficiency. First, it ensures equitable participation for fully remote students when the course is taught in a hybrid mode, leveling the playing field by having all students engage remotely. Second, it alleviates the need to schedule conference rooms for in-person participants, addressing a significant limitation on class size. Since each student’s work is critiqued in 10–15-minute intervals, the need for multiple simultaneous critique sessions across several class periods can be met more easily through remote sessions. This also eliminates the challenge of securing conference rooms with the appropriate videoconferencing technology. Finally, the remote format facilitates participation by distant fluid mechanics experts, allowing for greater expertise and variety in feedback. Overall, the shift to remote critique sessions has improved the logistics and inclusivity of the course, benefiting both students and external contributors.

The structure of the Flow Vis course can be readily applied to other science, technology, engineering, art, and math (STEAM) courses. Students can be invited to create aesthetic representations illustrating the fundamental concepts, write about them, and publish the results with an open license. An example of application in a different STEAM topic is a course in engineering design: the Aesthetics of Design course at the University of Colorado.

Extending this course structure to disciplines outside of STEAM is straightforward as long as aesthetics relevant to the discipline can be imagined. Having an aesthetic component ensures endless variations in student work, with each example providing a unique perspective on the topic, thus allowing the published resource to grow in value with each successive semester’s offering. Even without an aesthetic component, some of the tools described here will be useful for other pedagogies, such as the Critical Response Process for peer critique or an ongoing blog site for the publication of student work.

Summary

In closing, the Flow Visualization course exemplifies open pedagogy in numerous ways, allowing students to blend aesthetics with scientific inquiry as they navigate the intersection of art and science. The high visibility of their work on a well-established, globally-recognized archival website enhances its reach over time and space, as outlined in Seraphin et al.’s (2019) framework. Furthermore, the course has significantly impacted students and faculty, contributing to its overall gravity within this framework. Ultimately, the Flow Vis course highlights the utility and value of open pedagogy in engineering education, underscoring the importance of empowering students to unleash their creativity.

References

Abri, M. H. A., & Dabbagh, N. (2019). Testing the intervention of OER renewable assignments in a college course. Open Praxis, 11(2), 195-209. https://doi.org/10.5944/openpraxis.11.2.916

Baran, E., & AlZoubi, D. (2020). Affordances, challenges, and impact of open pedagogy: Examining students’ voices. Distance Education, 41(2), 230–244. https://doi.org/10.1080/01587919.2020.1757409

Blackwelder, A., Blum, S. D., Chiaravalli, A., Chu G., Davidson, C., Gibbs, L., Katopodis, C., Kirr, J., Kohn, A., Riesbeck, C., Sackstein, S., Schultz-Bergin, M., Sorensen-Unruh, C., Stommel, J. & Warner, J. (2020). Ungrading: Why rating students undermines learning. In S. D. Blum, (Ed.), How to ungrade (pp. 35-36 ). West Virginia University Press.

Clinton-Lisell, V. (2021). Open pedagogy: A systematic review of empirical findings. Journal of Learning for Development, 8(2), 255–268. https://doi.org/10.56059/jl4d.v8i2.511

Contract grading. (2023, January 5). In Wikipedia. https://en.wikipedia.org/w/index.php?title=Contract_grading&oldid=1131731023

DeRosa, R., & Jhangiani, R. S. (2017). Open pedagogy: A guide to making open textbooks with students. https://press.rebus.community/makingopentextbookswithstudents/chapter/open-pedagogy/

Goodman, K. (2015). The transformative experience in engineering education [Doctoral dissertation, University of Colorado]. The ATLAS Institute repository. https://www.colorado.edu/atlas/sites/default/files/attached-files/the_transformative_experience_in_engineering_eduaction.pdf

Goodman, K., Hertzberg, J., & Finkelstein, N. (2018). Surely you must be joking, Mr. Twain! Re-engaging science students through visual aesthetics. Leonardo, 53(3), 311–315. https://doi.org/10.1162/LEON_a_01604

Hertzberg, J. (2024a). The flow visualization guidebook. https://www.flowvis.org/Flow%20Vis%20Guide/introduction-to-the-guidebook/

Hertzberg, J. (2024b). Team expectations. https://flowvis.org/media/course/TeamExpectations.pdf

Lerman, L. (2002). Critical response process: A method for getting useful feedback on anything you make, from dance to dessert. Liz Lerman Dance Exchange.

Pugh, K. J. (2011). Transformative experience: An integrative construct in the spirit of Deweyan pragmatism. Educational Psychologist, 46(2), 107–121. https://doi.org/10.1080/00461520.2011.558817

Rayleigh–Taylor instability. (2023). Wikipedia. https://en.wikipedia.org/w/index.php?title=Rayleigh%E2%80%93Taylor_instability&oldid=1176070139

Rober, M. (Director). (2019, August 29). World’s largest elephant toothpaste experiment [Video recording]. https://www.youtube.com/watch?v=Kou7ur5xt_4

Scribner, H., Goodman, K., & Hertzberg, J. (2021, June 9). The influence of aesthetics on engineering learning [Presentation/Workshop]. ASEE Rocky Mountain Section Unconference 2021, Online. http://www.uwyo.edu/asee/rms/details.html

Seraphin, S. B., Grizzell, J. A., Kerr-German, A., Perkins, M. A., Grzanka, P. R., & Hardin, E. E. (2019). A conceptual framework for non-disposable assignments: Inspiring implementation, innovation, and research. Psychology Learning & Teaching, 18(1), 84-97. https://doi.org/10.1177/1475725718811711

Stancil, S. K. (2020). Exploring working graduate students’ experiences with reusable assignments [Doctoral dissertation, North Carolina State University]. Proquest. https://www.proquest.com/docview/2400696925/abstract/89347948CE084B4CPQ/1

Wiley, D., & Hilton III, J. L. (2018). Defining OER-enabled pedagogy. The International Review of Research in Open and Distributed Learning, 19(4), 133-147. https://doi.org/10.19173/irrodl.v19i4.3601