Standard V – Educational Effectiveness Assessment

A robust assessment culture informs all operations at Geneseo, but it is most fully developed and pervasive in the area of educational effectiveness. Individual academic programs and general education areas routinely assess program learning outcomes, which align upstream with baccalaureate learning outcomes for the institution as a whole and downstream with learning outcomes in individual courses. Assessment results drive improvements and innovations in curriculum and pedagogy, which are themselves assessed to evaluate their effectiveness and perpetuate an iterative process.

Through the Geneseo website, outcomes for baccalaureate learning, programs, and general education are communicated both to the campus community and external stakeholders, making visible the college’s intentional and systematic approach to undergraduate education. Programs and general education areas record assessment results, together with reflection and conversation about those results, in the Geneseo wiki, where they may be viewed by all Geneseo account holders, thus promoting an open culture that encourages programs to learn from one another. The resulting transparency meets the reviewers’ recommendation in response to Geneseo’s 2017 Periodic Review Report (PRR) that “the results of the assessments of student learning outcomes be made easily accessible and available college-wide so that they may be used to improve programs and services and inform resource allocations.”

A few programs conduct assessment through the use of standardized examinations; most do so by applying rubrics to course-embedded assignments and test questions.

For roughly two decades, this process has been overseen by the Academic Program Assessment Committee (APAC), whose members (faculty and administrators) review program assessment results and assist programs in understanding assessment best practices. The chair of APAC serves on the College Assessment Committee (CAC), a college-wide body that oversees institutional effectiveness for all aspects of Geneseo’s operations and that serves, among other things, as the means for assessing the educational effectiveness processes described in this chapter. CAC also provides the link between learning outcomes assessment and strategic planning for the institution by sharing the audit report results annually with the Strategic Planning Group. In using the wiki to collect annual assessment and to share the results of the assessment audit widely, CAC has helped Geneseo meet the PRR reviewers’ recommendation that “the institution complete the implementation of the IE [Institutional Effectiveness] plan by creating and approving a common platform for reporting assessment data that can be used to link planning to budgeting processes and by fostering a strong assessment culture at Geneseo.”

1: Educational goals

Clearly stated educational goals at the institution and degree/program levels, which are interrelated with one another, with relevant educational experiences, and with the institution’s mission. (SV.C1; RoA9, 10)

Since 2016, Geneseo’s curriculum has been organized by a statement of specific educational outcomes titled GLOBE—an acronym for Geneseo Learning Outcomes for Baccalaureate Education. GLOBE divides learning outcomes at Geneseo into “Broad and Specialized Knowledge,” “Intellectual and Practical Skills,” and “Integrative and Applied Learning.”^[1]

Each academic program at Geneseo has articulated program learning outcomes and mapped them to GLOBE. In addition, each program has mapped its individual course outcomes to its program outcomes. The program outcomes and maps may be found in the Academic Assessment space of the Geneseo wiki, where each year programs record their assessment results and actions designed to close the assessment loop.^[2]

Students, faculty, and other stakeholders will find a comprehensive history and explanation of GLOBE in the college’s online bulletin, where GLOBE is explicitly linked to the college’s mission. In addition, each program’s landing page on the Undergraduate Bulletin website lists that program’s learning outcomes. Thus, learning goals at Geneseo are articulated transparently and prominently from the highest institutional level (GLOBE), through the level of academic programs, to the level of individual courses, supporting Requirement of Affiliation 10. As described in this chapter, the assessment activity recorded in the Academic Assessment wiki space also supports Requirement of Affiliation 9.

2: Organized and systematic assessments

Organized and systematic assessments, conducted by faculty and/or appropriate professionals, evaluating the extent of student achievement of institutional and degree/ program goals. Institutions should define meaningful curricular goals with defensible standards for evaluating whether students are achieving those goals;articulate how they prepare students in a manner consistent with their mission for successful careers, meaningful lives, and, where appropriate, further education; collect and provide data on the extent to which they are meeting these goals; and support and sustain assessment of student achievement and communicate the results of this assessment to stakeholders.

Meaningful goals, defensible standards

(SV.C2a, RoA 10)

As described above, the Academic Assessment space of the Geneseo wiki houses pages where the college’s 18 departments and two schools publish their learning outcomes, record yearly assessment results, and describe actions planned or taken to close the assessment loop. The information housed in the wiki demonstrates how Geneseo is meeting the first part of Requirements of Affiliation 8-10.

Each program has a landing page in the wiki. Links from a program’s landing page lead to assessment results by academic year. (The landing pages are publicly visible, but only authenticated Geneseo users can view assessment results.) Results pages follow a standard template: learning outcome assessed; connection of outcome to the college’s mission and values, the college’s strategic plan focus area, and the program’s learning goal or outcome; means of assessment for that outcome (including defensible standards for evaluating whether students are meeting the outcome); results for the current round of assessment for that outcome; and reflection on those results, including actions planned in light of the results.

Many programs assess student learning using embedded assignments or exam questions to which faculty apply a scoring rubric, with standards for success tied to the percentage of students meeting or exceeding a target score. Some programs assess using a standardized exam on which students must earn or exceed a minimum score.

For example, to assess their students’ research ability, the Department of Communication analyzed research essays submitted by students in both a 100-level course and two 300-level courses in 2019-20.^[3] Using their own rubric, which built on a SUNY rubric for assessing research skills for general education, the department examined the essays, recorded the rubric results in the wiki, recorded conclusions from the results, and described how the results related to their efforts at the time to revise the communication curriculum. They found that students in the upper-level courses were more proficient at locating and incorporating scholarly resources than those in the 100-level course, but that students at all levels rely too heavily on generic search-engine results as opposed to results from the more focused research techniques that faculty teach them. Curriculum revisions currently underway in communication include an Introduction to Communication course for all majors that educates students about scholarly journals in the field and the best methods for finding high-quality scholarly sources. Two other examples of how Geneseo connects program assessment results to program improvement, from the Department of History and the Department of Languages and Literatures, are described below in Section 3.b.

The above case not only exemplifies the assess-record-reflect-improve loop common across all major programs at Geneseo; it also illustrates an important assessment efficiency that Geneseo has achieved by assessing general education outcomes within the major when they are also outcomes for the major itself. Whether undertaken within major programs or through a free-standing assessment, assessment of student learning outcomes in all general education areas (such as fine arts, western humanities, natural sciences) follows the same assess-record-reflect-improve process, typically employing a rubric designed by instructors in that area. Each general education area has a landing page in the wiki and dependent pages that record assessment results. General education assessment is not conducted on each area each year but instead follows a three-year cycle. By contrast, each major program assesses a subset of its learning outcomes every year, so that all learning outcomes within each major program are assessed at least once every five years. The yearly cycle of program learning outcome assessment informs curricular revision and program development, thus serving as a crucial component of the five-year program review process described under Standard III (Criterion 8). It is discussed further in the current chapter under criterion 3c. Assessment timelines are publicly shared in the wiki.

Preparing students for life

(SV.C2b)

Geneseo’s stated mission to “inspire[s] students to develop their knowledge and skills to build a better world” drives the institution’s efforts to prepare students for successful careers, meaningful lives, and further education. As part of Geneseo’s annual assessment audit, academic programs and administrative divisions (academic affairs, college advancement, enrollment management, finance and administration, office of the president, and student and campus life) document outcomes assessed, goals identified based on assessment data, and the alignment between these goals and strategic plan objectives, college values, and the institutional mission.

Orientation at Geneseo helps transition first-year and transfer students to the next steps of their academic and life journeys. The purpose of orientation is to facilitate opportunities for students to acclimate to campus, create positive and inclusive connections, foster a sense of pride in Geneseo and establish a foundation for academic success. The aim of orientation, including Weeks of Welcome (WOW), is to lay the groundwork for the rest of our students’ college experiences. (See StdIV.C4.) The Fall 2020 report has assessment data from orientation and WOW.^[4] Based on the assessment results, the following improvements were suggested:

• Condense the amount of material offered on Canvas.
• Allow time in small groups simply dedicated to “get-to-know-you” style activities.
• Provide more opportunities to get together in small groups.
• Continue to incorporate a virtual component, even after limitations are lifted.
• Continue using this model for Student Organization Expo, even after an in-person one is reinstated.
• Work with Residence Life to ensure Resident Assistants (RAs) are aware of the schedule to educate and market events to students. Students reported RAs were the group from whom they were least likely to hear about orientation programming. Discuss the RA’s role moving forward since their roles have changed.
• Ramp up efforts for marketing events. Move away from relying solely on the website schedule of events.

The Geneseo Opportunities for Leadership Development (GOLD) program provides an opportunity for any student to learn, develop, and practice leadership and life skills. The open architecture of the program permits students to participate whether they are new students or seniors. The open, flexible, and inclusive model keeps students engaged with their own leadership development. This alignment is illustrated in the Department of Career Development’s 2019-20 assessment report.^[5] Over the 2019-2020 academic year, there were 3,578 participants in 282 workshops. Of the students, 1,728 students were unique participants in one or more GOLD workshops. The GOLD program collected 2,015 student self-report surveys, and students submitted over 1,100 journal reflections. Examining their perceptions of the program, students were asked to evaluate various statements:

• “I expect to utilize the information I learned in my current or future endeavors.” 90.3 percent of students (n = 1,820) either agreed or strongly agreed with this statement.
• “I would recommend this program to other students.” 89.3 percent of students (n = 1,800) either agreed or strongly agreed with this statement.
• “I would like to attend other programs and activities on this topic.” 81.1 percent of students (n = 1,634) either agreed or strongly agreed with this statement.

In Fall 2018, in collaboration with the School of Business and the career development department, Geneseo launched a new certificate called the School of Business Professional Development Certificate. This program has been very successful in providing professional development workshops required by the Association to Advance Collegiate Schools of Business (AACSB) for accreditation. Although this certificate is open to all Geneseo students, it has been mostly pursued by School of Business students. As an ongoing extension effort of this program evaluation, data from 286 students taking these fully synchronous workshops were analyzed in Fall 2020. The following analysis was based on workshop data from this semester. When students were asked, “The topic was helpful to my career development,” 93.0 percent (n = 285) either agreed or strongly agreed. In Spring 2021, in a new collaboration with the career development department, Geneseo will launch the Career Management Leadership Certificate. This certificate will focus on preparing all Geneseo students with the skills to network; connect with a range of diverse colleagues, managers, and clients; and articulate both their strengths and goals for the future.

In 2020, Geneseo was reclassified on the Carnegie Community Engagement list for our commitments to community engagement. A large number of our students are engaged in many hours of volunteer work each year. Below are the data for volunteerism at Geneseo.^[6] (The pandemic reduced volunteering in Spring 2020 and AY 2020-21.)

More broadly, results of the 2020 National Survey of Student Engagement revealed that 91 percent of Geneseo seniors participated in at least one high-impact practice, including service learning, participation in a learning community, research with faculty, internship or field experience, study abroad, or culminating senior experience. Sixty-six percent of seniors reported participating in two or more of these activities.^[7]

For further examples of evidence from alumni surveys, see evidence from the School of Education and the career development department.^[8] As of 2021, the Career Design Center has been moved to the academic affairs division, to further integrate career preparation into academic programs.

Transparency for stakeholders

(SV.C2c; RoA 8)

APAC has primary responsibility for supporting and sustaining assessment of student achievement and for ensuring that both the assessment process and assessment results are transparent to stakeholders. APAC membership consists of the associate provost for assessment and curriculum, the respective assessment coordinators for the School of Education and the School of Business, and selected assessment coordinators for humanities and fine arts, natural sciences, and social sciences.

Program learning outcomes are assessed by academic programs on a five-year cycle, using rubrics created by each program.^[9] Departments collect data to assess at least one program learning outcome each academic year. (One instrument per outcome must collect quantitative data.) In addition, departments are encouraged to assess unofficial outcomes that may be of interest in a particular year, such as during the pandemic. In each program, an assessment coordinator is responsible for collecting the data, sharing it with department colleagues and recording results and reflection in the Geneseo wiki, where they are visible to the entire college community.

APAC organizes and oversees this activity and provides support to coordinators and departments in collecting data, interpreting it, and “closing the loop” as described below under criterion 3c. Department assessment coordinators are required to report findings to their departments no later than the second department meeting of the fall semester, and departments are expected to discuss these findings to determine potential action steps. These loop-closing discussions, too, are to be added to the wiki where all stakeholders can read them. By supporting this process, APAC helps Geneseo meet the second part of Requirement of Affiliation 8.

3: Use of assessment results for institutional effectiveness

Consideration and use of assessment results for the improvement of educational effectiveness, including some combination of the following: assisting students in improving their learning; improving pedagogy and curriculum; reviewing and revising academic programs and support services; planning, conducting, and supporting a range of professional development activities; planning and budgeting for the provision of academic programs and services; informing appropriate constituents about the institution and its programs; improving key indicators of student success, such as retention, graduation, transfer, and placement rates; implementing other processes and procedures designed to improve educational programs and services. (SV.C3)

The steps that Geneseo has taken to collect, reflect on, and act upon assessment data help it to meet its third institutional priority: “The links between assessment and action plans at various levels within the institution will be strengthened.” Systematic assessment at Geneseo drives continuous improvement in every area of the educational enterprise, from the classroom to support services to planning and budgeting, ensuring that effort and resources are allocated where they will have the greatest positive impact on student success, and enabling the entire campus community to recognize institutional strengths and identify areas for improvement.

Assisting students, improving pedagogy and curriculum, closing the loop, and reviewing support services

(SV.C3a, SV.C3b, SV.C3c)

The following are examples of departments assisting students in improving their learning through the creation of peer tutoring and academic support programs and services.

Tutoring at Geneseo is largely decentralized, with the exception of the Writing Learning Center (WLC), the Math Learning Center (MLC), and Supplemental Instruction (SI), which are supervised by the associate provost for academic success. In Spring 2019, the College supported the coordinators of these programs, the associate provost, and the coordinator of AOP tutoring to design common training for undergraduates employed as tutors and a common approach to usage data. The coordinators designed and executed a training program that is now standard for these programs; they assessed these programs as integrative and applied learning experiences and presented their results at the 2020 Assesstivus event.^[10] In Spring 2020, the WLC, MLC, and AOP tutoring also moved to scheduling and appointment tracking through EAB Navigate, and began reviewing usage data on a semester basis. (Note that because this occurred during the pandemic, full assessment has not been conducted but data has been collected.)^[11] Based on the evidence provided, the total number of appointments, both scheduled and held, has increased for all three centers from Spring 2020 to Spring 2021. Hours have also increased over time; however, there was a significant decrease in hours observed in Spring 2021 for the AOP tutoring center.

Many departments at Geneseo provide first-year courses that introduce students to college-level work and strategies for success, together with useful knowledge about the programs themselves. The list has expanded each year. The 2020-2021 roster of first-year experience courses includes 14 different courses such as Chemistry 100, First-Year Experience; Education 102, Welcome to the Major; and Psychology 188, Orientation to PSYC at Geneseo. Assessment of student outcomes revealed that undeclared students had lower retention rates; the provost’s office initiated a pilot to offer an exploratory first-year seminar with learning strategy support specific to undeclared students; assessment of this pilot resulted in additional funding and scaling up. (See evidence and discussion under Standard IV, Criterion 1.)

The examples below illustrate how programs have used assessment data to make major curricular improvements:

• Languages and Literatures: Based on assessment results, the Department of Languages and Literatures has moved to a four-credit course model. General education students will be provided with more time on task per course in the target language. The change will also increase cultural knowledge by enabling the integration of media and technology to increase the quality of time on task—an essential step on the path to the GLOBE outcomes of “skill in another spoken language” and “global awareness and engagement.^[12]
• INTD 105: The coordinators of the required first-year writing seminar received a strategic planning grant to engage an external consultant (Michael Murphy, professor at SUNY Oswego and chair of the SUNY Council on Writing), who reviewed the courses during the summer and fall of 2019. Among other recommendations, Murphy made the following three that the coordinators have implemented: (1) Assess the effectiveness of INTD 105 and 106 through portfolio-based assessment. Faculty began this in the spring of 2021 (after a one-year delay due to the pandemic) and, after analyzing the portfolio assessment, plan to use it in faculty training. (2) Formalize a system of professional development for faculty teaching INTD 105. One of the INTD 105 coordinators received a Teaching and Learning Center grant that supported a reading and planning group studying anti-racist writing pedagogy in the summer of 2021. This group will incorporate antiracist pedagogy training into the biannual workshops supported by the provost’s office. (3) Remove the INTD 106 requirement, which is unsustainable. INTD 106 is a one-credit online complement to 105 focused on grammar, usage, and other writing conventions. The elimination of the requirement was approved by the College Senate in Spring 2021. In its place, the coordinators will make the instructional materials from INTD 106 available to all writing courses. The one-credit course will continue to be available to targeted groups.^[13]
• Mathematics: For the mathematics department’s 2019-2020 assessment,^[14] the department studied whether students were amply prepared for upper-level math courses (MATH 221, 213) and whether taking lower-level math courses (MATH 112) increased preparedness. Students who had taken MATH 112 performed significantly better on precalculus examinations, but no effects were observed on final upper-level course grades. (This lack of effect may be partially attributed to the confounding circumstances of remote learning.) Based upon these results, the department has suggested changes to MATH 112 that might better prepare students for upper-level calculus courses, most notably a greater focus on and review of fundamental concepts that will continue to be used in these upper-level courses.
• School of Business: The school responded to 2018-19 assessment results^[15] by aligning ECON 205 and ECON 305. Based on analysis of the data, faculty thought it was important for students to take these courses in successive semesters. Messaging went out to faculty and students and there was follow-up communication with School of Business advisors.
• Biology: Faculty received a SUNY Performance Improvement Fund (PIF) grant to close equity gaps and reduce the DEW rate among first-year biology students. They steered students at risk into BIOL 128 and 188. Students enrolled in the BIOL 128, Freshman Experience in Biology, completed an end-of-semester survey. A similar survey was administered to students in the spring BIOL 188 laboratory course. Nearly 58 percent of students agreed or strongly agreed that “I am glad that I enrolled in BIOL 128 Freshman Experience in Biology.” Based on student comments, common strengths of the course included the Microsoft Excel training and the academic advising provided as well as helping students feel more comfortable engaging with professors outside class. Students in BIOL 188 felt that they were part of the STEM community. They interacted well with their instructors, undergraduate teaching assistants, and each other, allowing them to feel very comfortable in the laboratory. The end-of-the-semester survey showed that they valued these interactions and realized it was important for their success. These results demonstrate progress in developing a sense of belonging—one of Geneseo’s values. Some students in BIOL 128 indicated that there was overlap with another concurrent seminar offered to students in the AOP program. In response, the biology department altered the syllabus to complement rather than duplicate AOP seminar offerings.^[16]

The academic assessment processes described above represent the primary means whereby Geneseo makes continuous improvements to pedagogy and curriculum.^[17] In addition, the past few years have witnessed a higher-level, more comprehensive push for curricular improvement through the efforts of the Curricular Design Working Group (CDWG), established in the spring semester of 2017 to develop a complete re-envisioning of Geneseo’s curriculum based on GLOBE, as described under Standard III, Criterion 5. The new curriculum is expected to go into effect in Fall 2022.

Assessment is a crucial element in the comprehensive review that programs undertake every five years. The review process includes the writing of a self-study and a visit from two external reviewers. Program review is another means whereby Geneseo meets Requirement of Affiliation 8. Program review documents are archived in a Geneseo wiki space linked to the space for academic program assessment.^[18]

Professional development activities

(SV.C3d)

Many units on campus use assessment in planning, conducting, and supporting a wide range of professional development activities for faculty and staff. (See Standard III, Criterion 2: Competency of Faculty.)

For example, the Center for Digital Learning assesses the effectiveness of its professional development activities by surveying participants to learn how well the activities met their needs. Questions for participants concern such matters as whether the activity was relevant to their work at the College, whether they found the format of the activity was effective, and whether there are topics they would like to see taken up in the future. Feedback from these surveys is used to plan new activities and improve the next iteration of already developed ones. Similarly, the Center for Integrative Learning practices ongoing assessment of faculty/staff experiences and college-wide programs and ventures through a number of direct feedback forms. For faculty/staff professional development, CIT participates in the Measuring Information Service Outcomes survey (MISO) every four years. In that survey, faculty and staff can report what training they would like. After workshops, CIT staff survey attendees about effectiveness and solicit ideas for future events. (For more on CIT, see Standard 4 Criterion 4, Integrative and Applied Learning.)

Some assessments related to professional development are specific to particular programs. For example, the biology department has begun surveying students about diversity, equity, and inclusion, and is using the results to plan training opportunities for faculty and staff each semester; some of these opportunities are open to students as well. The School of Business has formulated goals for a formal professional development program with specific goals matched to required events. The content of each event is then updated to align it with learning objectives. Evaluators composed of School of Business faculty and staff visit a sample of events and provide feedback using a standardized evaluation form. A student survey is administered at the end of every event and an analysis report of the survey results is provided periodically. The Student Services Committee takes charge of reviewing and monitoring the effectiveness of the program. Information from the graduating student survey, the employer survey, and feedback from the Business Advisory Council member is also used for continuous improvement.^[19]

Planning and budgeting

(SV,C3e)

As noted in Geneseo’s 2017 Periodic Review Report (PRR), “In general, requests for resources reach the president and cabinet by multiple paths—both formalized and ad hoc. Formal institutional pathways include requests coming from the college’s [Strategic Planning Group (SPG)] and from campus-level commissions” (46). The PRR also noted that Geneseo’s adoption of a new institutional effectiveness plan was expected to “better link institutional planning with budget decision-making going forward,” in part through a provision that every department/division annually complete a standardized assessment report whose results would be visible to the College Assessment Council (CAC) (45). The PRR described some of the difficulties faced by CAC in efficiently harvesting assessment results from units across the College and making them available to SPG and cabinet in a way that would make them useful for budgeting and planning. Ultimately, a simple form was adopted to streamline the assessment reporting process and enable SPG to link key assessment findings to one or more strategic planning objectives or mission objectives.

At the conclusion of 2016-17, responding to the emergent, pressing need to take action to ensure the college’s financial sustainability, President Battles charged SPG and the Budget Priorities Committee (BPC) jointly with soliciting funding requests for sustainability-enhancing projects aligned with the 2021 strategic plan.^[20] Based on the formal proposals presented to SPG and BPC in September 2017, eight projects were funded. All were assessed, and the assessment reports have been archived in the BPC wiki space, where they are visible to the entire campus.^[21]

In February 2019, President Battles expanded BPC’s membership and charged the committee with canvassing the campus community for revenue-enhancing and expense-reducing ideas that might help the College address a structural budget gap of approximately $3.2 million. BPC issued a comprehensive report at the end of 2018-2019 analyzing the suggestions that came in and offering suggestions for next steps.

Unfortunately, these urgent, active, forward-looking, transparent, and inclusive responses to a budgetary outlook driven largely by demographic changes affecting much of higher education left little time for SPG and BPC to take up recommendations based on assessment data solicited by CAC. From Spring 2020 to the end of 2020-21, the coronavirus pandemic proved additionally disruptive. Nevertheless, strengthening the link between assessment and resource allocation remains a top priority for Geneseo. In Spring 2021, CAC did bring two resource requests to BPC based on the assessment data it has collected. Decisions regarding these requests are still pending. Meanwhile, BPC and CAC have both recognized the need to make relevant assessment data more readily accessible for the former’s consideration. Finally, the cabinet and BPC will better distinguish the role of each in elevating assessment-based recommendations; changes here should alo help ensure that, even under the most difficult circumstances, assessment results inform budgetary decisions in a regular and timely fashion.

Informing constituents

(SV.C3f; RoA8)

Geneseo informs both internal and external constituents about the institution and its programs through various platforms. Internal constituents can find information in Argos reporting, the Geneseo wiki, and EAB Navigate analytics.

• Argos: Evisions Argos is an enterprise reporting solution that gives access to data needed to run the institution. Argos makes reporting simple, delivering insights that enable Geneseo to make timely, better-informed decisions. Data available in Argos comprises admissions, financial aid, department reports, divisions, employee, faculty, foundation, and student reports. The School of Education has requested custom-built reports to support their accreditation through the Council of Accreditation for Educator Preparation (CAEP).
• Geneseo wiki: Mentioned frequently in this self-study, the Geneseo wiki is an interactive information storage platform that can be edited directly on the web by Geneseo users granted varying levels of permission. The wiki is maintained by CIT. (See wiki policies.^[22]) The Middle States evidence documents and self-study reports are archived here, as well as materials for CAC and APAC.
• EAB Navigate: In 2020, Geneseo adopted EAB Navigate, an early-alert and institutional data analytics system that provides more opportunities to intervene with students who exhibit poor attendance, struggle with course materials, and have challenges accessing tools that will help them succeed.

The college’s external stakeholders can find information and data about the institution on the Geneseo website, where, in the language of Requirement of Affiliation 8, the College “makes public how well and in what ways it is accomplishing its purposes.”

The Office of Institutional Research produces the SUNY Geneseo Fact Book, which contains college-wide data, and makes it available for download from its website.^[23] The Fact Book contains historical and statistical information about Geneseo for students, faculty, staff, and others with an interest in the College. It is updated as data becomes finalized. Also available on the IR webpage is the Common Data Set (CDS).^[24] The CDS initiative is a collaborative effort among data providers in the higher education community and publishers as represented by the College Board, Peterson’s, and U.S. News & World Report.

The School of Education and the School of Business make available information as required per their accrediting bodies. The SOE has a web page displaying three-year trends of impact and outcome measures including graduation rates, cost of attendance, performance on teacher certification exams, alumni impact on P-12 student learning, indicators of teaching effectiveness of program completers, satisfaction of completers and employers, and employment milestones.^[25] The business school has newsletters with an update from the dean, updates on career development, and news highlighting current students and alumni. (For example, see the Fall 2020 newsletter.^[26])

Improving key indicators of student success

(SV.C3g)

Assessing key indicators of student success—specifically retention, graduation, transfer, and placement rates—is directly aligned with our first institutional priority.

Recent trends

Based on the last five IPEDS graduation rates reports, (entering cohorts from fall 2010 to fall 2014), Geneseo observes the following trends with regard to retention, graduation (150 percent, within 6 years), and transfer rates.^[27]

Strengths

The six-year graduation rate for Black/African-American students has increased from 64 percent in the 2010 cohort to 70 percent in the 2014 cohort; the same is true of the graduation rate for students of two or more races (from 66 percent to 74 percent), for Asian students (from 66 percent to 74 percent) and Latinx students (from 68 percent to 76 percent).

Areas for improvement

Full-time retention has remained at 85 percent over the past three cohorts; however the retention rate has historically been 90 percent, as recently as the fall 2013 cohort. This decrease in fall to fall retention rate has begun to impact our six year graduation rate; the overall graduation rate dropped from 81 percent for the 2010 cohort to 78 percent for the 2014 cohort.^[28]

WIG

In 2018-19, Provost Stacey Robertson put together a team of leaders from academic affairs and other units to achieve the “wildly important goal” of increasing the retention rate of first-time college students at Geneseo between first and second years from 86 percent to 88 percent. Drawing its inspiration and approach (as well as the term “wildly important goal”) from the book The Four Disciplines of Execution, by Jim Huling, Chris McChesney, and Sean Covey, the team identified lead measures connected to the goal, divided into subgroups to work independently on these measures, set up a scorecard for gauging progress on each lead measure and the goal itself, established a “cadence of accountability” through weekly whole-group meetings and subgroup reports, and engaged the entire campus through a page on the College website and regular email communications. (For additional details about the WIG process, see Provost Robertson’s 2019 slide presentation to the College Council.^[29])

As discussed under Standard III, Criterion 4 (Outcome: Retention and Graduation Rates), despite falling short of the 88 percent retention rate goal, Geneseo implemented several initiatives that had a measurable impact on student success.

Steps taken to improve outcomes

The College has acted to improve retention and graduation rates for low-income and underrepresented students by securing additional resources through grants—for example, the TRIO Student Support Services Success Program and STEM inclusion initiatives. Under Standard III, Criterion 4, we provide details of these programs, explain the assessment of achievement gaps for these students, and describe the identification of institutional supports/external resources to close those gaps. (See SSS and NSF S-STEM proposals.^[30])

Below are two examples of how academic programs are assessing key indicators of student success:

• 2019-2020 assessment results from the Department of Physics and Astronomy revealed that the majority of physics majors apply for graduate study.^[31] The program will continue to poll graduating seniors each year to see whether they were accepted for graduate study.
• In 2018, the School of Education conducted an alumni survey of undergraduate and graduate students who graduated 2015-2017.^[32] Overall, 83 percent of these alumni are currently employed in an area related to their field of study and 54 percent are continuing their education in the field of education. (The latter figure includes undergraduates and graduate students.)

Other processes and procedures

(SV.C3h)

The following evidence shows the academic standing of our students from Fall 2017 through Spring 2021.^[33] As described under Standard IV, C1b, the Office of the Dean of Academic Planning and Advising (DAPA) helps current students, faculty advisors, and prospective students create plans for successfully navigating Geneseo’s curriculum; celebrates student success; and publishes the Dean’s List and President’s List at the end of each semester.^[34]

DAPA helps students explore potential majors and assists students who have struggled academically through an academic coaching program. An analysis of the academic standing data gave DAPA direction in improving its academic coaching program. Assessment shows students’ overall GPA increased for students during coaching 63 percent of the time.^[35] After coaching, this increase drops overall to 22 percent. In Spring 2020, the coaching program had its greatest impact, with GPAs increasing overall 76 percent of the time, illustrating the importance of the coaching program during the pandemic.

As described under Standard IV, Geneseo provides students with an ongoing degree audit in the form of a report from DegreeWorks. Students meet with advisors once they have reached 75 credits and 75 percent degree completion to ensure they are on track to graduate. Students and advisors share responsibility for tracking their progress toward graduation and fulfilling all requirements.

DAPA hires and trains between five and six academic peer mentors each year to assist any student with study strategies, registration, and DegreeWorks. The Geneseo Points of Support (GPS) program was an innovation of the WIG retention committee. Volunteer staff and faculty help students navigate unfamiliar institutional processes. Geneseo’s academic coaches are faculty and staff trained to assist students in academic difficulty through one-on-one biweekly meetings. They take part in a learning community organized by DAPA and the Teaching and Learning Center. All students who successfully appeal academic dismissals are assigned an academic coach. When staff are available, students on probation are also offered academic coaching. Assessment of academic coaching is discussed under Standard IV.

In Spring 2021, DAPA advisors offered students on probation a two-weekend bootcamp on academic success strategies. Bootcamp assessment demonstrates an increase of 44 percent in participants’ term GPAs and an increase of 47 percent in credits earned by participants.^[36]

In 2020 and 2021, the College as a whole and various individual programs have used assessments surrounding COVID-19 to improve educational programs and services. See, for example, assessments conducted by geological sciences, chemistry, biology and psychology.^[37]

4: Third-party services

Adequate and appropriate institutional review and approval of assessment services designed, delivered, or assessed by third-party providers. (SV.C4)

Several academic programs at Geneseo require students to complete assessments that are designed, administered, and scored by a third-party provider. In reviewing and approving these assessments, Geneseo relies on either New York State guidelines or (in the case of chemistry, for example) disciplinary recognition. The assessments below are used directly for program evaluation and continuous improvement.

Education

To be eligible for New York State teacher certification, students in the School of Education must achieve a passing score on certain assessments, depending on their area of certification. As described on the The New York State Teacher Certification Examinations™ (NYSTCE®) website, these certification examinations address New York education law and commissioner’s regulations, which require prospective New York State educators to pass designated tests as a requirement for receiving state certification.^[38]

The NYSTCE are criterion-referenced, objective-based tests designed to measure a candidate’s knowledge and skills in relation to an established standard rather than in relation to the performance of other candidates. The explicit purpose of these tests is to help identify for certification those candidates who have demonstrated the appropriate level of knowledge and skills important for performing the responsibilities of an educator in New York State public schools.

Test questions were developed using textbooks, New York State learning standards and curriculum guides, teacher education curricula, and certification standards. The tests were developed in consultation with committees of New York State teachers, teacher educators, and other content and assessment specialists. Specific third-party assessments required for each of the three teacher preparation certification programs offered at Geneseo are the Educating all Students (EAS), Content Specialty Tests (CSTs) and the edTPA.^[39]

Business

Certified Public Accountant (CPA Exam). Although it is not a degree requirement, most completers of the MS program in accounting go on to take the Uniform CPA Exam, The CPA exam is developed, maintained, and scored by the American Institute of Certified Public Accountants (AICPA) and administered at test centers in partnership with the National Association of State Boards of Accountancy (NASBA).

Chemistry

Diagnostic Undergraduate Chemistry Knowledge (DUCK). The chemistry department administers a standardized exam, the Diagnostic Undergraduate Chemistry Knowledge (DUCK) exam, to its seniors prior to program completion. DUCK is published by the American Chemical Society (ACS) and normed against national averages. It assesses knowledge in the five sub-disciplines listed in the chemistry program learning outcomes. This exam is administered as a part of Chemistry Senior Seminar (CHEM 352), the department senior capstone course, to seniors nearing completion of their degree program.

ACS Biochemistry Exam, ACS Organic Chemistry Exam. The Examinations Institute of ACS develops, and administers nationally, norm-referenced exams for most chemistry courses within an undergraduate chemistry curriculum in the United States. All ACS exams are created by committees of educators who teach the course for which an exam is intended.The ACS exams in biochemistry and organic chemistry are routinely administered to students in the biochemistry program at SUNY Geneseo.

Languages and Literatures

Oral Proficiency Interview (OPI). Prior to completing their degree, students who are adolescence education majors in Spanish or French take the Oral Proficiency Interview through the American Council on the Teaching of Foreign Languages (ACTFL). The OPI is a live conversation between a certified ACTFL tester and the candidate. It is a valid and reliable test that measures how well a person speaks a language. The procedure is standardized in order to assess global speaking ability, measuring language production holistically by determining patterns of strengths and weaknesses. Results are used to gauge where a group of students stands in terms of oral proficiency. Data from the OPI is used in the development of new curriculum.

5: Assessment

Periodic assessment of the effectiveness of assessment processes utilized by the institution for the improvement of educational effectiveness. (SV, C5)

As mentioned earlier, CAC supports college-wide assessment and institutional effectiveness.^[40] The council ensures the presence of assessment plans across the College and their alignment with the college’s mission and strategic plan. APAC assesses this process at the academic program level.

In order to ensure continued adherence to assessment best practices, CAC reviews the assessment plans of and audit results from the college’s units and responds as necessary to the unit leader who submitted them. The council ensures that the plans contain well-formulated outcomes, employ valid and reliable assessment measures, and close the loop on assessment findings linked to the strategic plan.

CAC organizes a yearly event, ASSESStivus, designed to educate the campus about institutional effectiveness and foster a culture of assessment. ASSESStivus activities include an outside keynote speaker, workshops, roundtable discussions, panel discussions, best assessment practices, and more. Through ASSESStivus, Geneseo provides assessment training and professional development across the campus; reinforces its commitment to a continuous improvement model of assessment; shares innovative and best practices in assessment across programs, departments and divisions; fosters a culture of support, teaching, and learning about assessment; provides opportunities to learn methods to make assessment more manageable and effective; and gathers feedback to improve on these activities in future years. In 2021, ASSESStivus dedicated one day each to assessment basics and assessing our responses to the COVID-19 crisis.

APAC sets the assessment calendar for the academic year, reminds departments of their assessment duties, assists departments with their assessment projects, and reviews program assessment reports. The committee uses a rubric to provide guided feedback to academic programs regarding their assessment processes.^[41] It provides both positive and constructive feedback regarding assessment processes, learning outcomes, assessment artifacts and instruments, and use of results, ensuring that academic programs have sound assessment practices that lead to improvement of educational effectiveness.

Closing the loop is of utmost importance to APAC. For example, APAC provided the following feedback to the mathematics program: “The analysis, conclusions, and recommendations are very nicely done in your report. We recommend making a stronger use of your results by developing an action plan based on which findings the department thinks are significant or can be addressed.” Similarly, APAC informed the business school: “SOB’s use of results is thoughtful and extensive; we are impressed that it is a normal process for SOB.” Discussion of assessment results focuses on both the individual level and entire programs. Attached are reports^[42] from 2019-20 where APAC reviewed the assessment practices of English, communication, mathematics, School of Business, School of Education, music, and geography.

Conclusion

Assessment of student learning demonstrates that our students have accomplished educational goals consistent with their program of study, degree level, Geneseo’s mission, and appropriate expectations for institutions of higher education.

Points of pride

• Thanks to the adoption of a new curriculum based on baccalaureate learning outcomes (GLOBE), students and faculty can see how courses, programs, and general education align with each other and the institution’s mission and values.
• The college uses the Geneseo wiki to record, discuss, and share assessment results. This enables each unit or department to tell its own assessment story, enables departments and units to learn from one another and promotes transparency for stakeholders.
• Geneseo continues to close the loop on assessment in ways that produce significant curricular improvements.
• The WIG process has increased coordination among units within and beyond academic affairs, leading to sustained, intentional efforts that have measurably improved both the student learning experience and institutional performance.
• As a source of information and professional development, the work of APAC is instrumental in maintaining and strengthening Geneseo’s assessment culture.
• ASSESStivus—Geneseo’s yearly celebration of assessment—remains a distinctive feature of the institution’s assessment culture, affirming the value of assessment as educational research, and fostering a vigorous and thoughtful exchange of ideas and methodologies.

Recommendations

None.