Sandra J. Snyder, A Writing Portfolio

 

You might call them The Academic Exercises.

Almost 500 years ago, St. Ignatius Loyola, founder of the Society of Jesus, wrote a meditative guidebook known as The Spiritual Exercises.

At a basic level, the exercises serve as a compass of sorts for those looking to reflect on what works and doesn’t work on their spiritual journey. In the early stages, one key step is the recognition of shortcomings as part of the human experience and the realization that such recognition can pave the path for transformation.

Several University of Scranton professors are intimately familiar with that concept, as it can apply not only to religious journeys but academic as well. In the latter case, the steps taken on the journey are part of a broad concept known as academic assessments, and they can be as altering as spiritual assessments.

To hear Rebecca Haggerty, assistant dean for assessment and programs, who with her husband, Daniel Haggerty, Ph.D., oversees the University’s Special Jesuit Liberal Arts Program, tell it, assessment not only can be transformational for faculty but reveal transformations in individual students.

Haggerty said she and her husband have long known SJLA, an esteemed honors program for the University’s top-tier students, was stellar.

“Generally speaking, everyone talks about how amazing SJLA is,” she said. “We saw that firsthand, but we wanted to put some meat to the bones. We wanted to know why.”

Thus began the process of assessing the University’s signature honors program not only from a hard-data standpoint, collecting statistical information, such as grade point averages and classes taken, but through a softer lens as well, the lens of personal reflection.

Every two years SJLA students and faculty embark on a service trip to Spain and Italy, a journey that puts them up close and personal with St. Ignatius, Haggerty said.

“We were at the cove where St. Ignatius wrote The Exercises,” she noted, explaining how the trip gave students a profound new appreciation for their Jesuit education.

Upon return, Haggerty said, the students were asked to write an essay about what the experience had meant for them, and that’s when major revelations struck.

“We really started to see how amazing these students were after they reflected. Everything they said was amazing, and they realized how important reflection actually is,” Haggerty said.

The mutual realizations led the Haggertys to do a different type of essay-based assessment after each trip, and “each time we have learned something different,” she said. The lessons were so revelatory it was decided students needed even more time to devote to reflection

A companion course was then developed for SJLA students to take each fall after their summer service trip.

The course, “An Ignatian Pilgrimage: Walking In The Footsteps of St. Ignatius,” focuses heavily on reflections, Haggerty said, and those reflections “increased and became very personal and related to the fact that St. Ignatius was on a journey himself.”

Haggerty said students became increasingly aware of their personal journeys and how intricately connected they are to their Jesuit education.

“If you ask them now what a Jesuit education is, they won’t stop talking,” she said.

Taking the show on the road

The Haggertys took their reflection-based approach to a recent Drexel University Assessment Conference, making a presentation that surprised many in attendance as it took the position that transformational experiences are indeed quantifiable.

Assessing the effectiveness of an honors-program travel course is inherently challenging, Rebecca Haggerty said, asking rhetorically, “How do you capture a transformational learning experience?”

One of SJLA’s most successful answers to that question came thanks to a survey distributed via Survey Monkey, with both quantitative and qualitative questions, she said. The surveys were extremely successful, and the high response rates prompted the decision to also put out surveys to SJLA alumni.

“Forty percent of alumni responded. That’s pretty huge,” Haggerty said, pointing out that the respondents represented every class year since 1980.

One key and gratifying survey finding was that students repeatedly said the SJLA program had enhanced their reading and writing skills, she said.

Those skills have been receiving much attention in many University departments lately and have become a key part of the entire assessments process.

Professors in the Kania School of Management, for example, found similar success enhancing communication skills, particularly writing skills, by using revelatory assessments to launch changes.

The tough business of writing

Assessments done at the Kania School between 2005 and 2010 led faculty there to take several actions to improve student performance in those areas. Now, in essence, the business track has become increasingly communication-heavy. Oral-presentation components were added to two existing courses, a new written and oral communication elective was added to the accounting track, and 12 new writing-intensive courses were designated within the general Kania curriculum. Faculty members in the new courses were encouraged to focus heavily on communication outcomes as well.

Faculty also were encouraged to adopt a zero-tolerance writing-error policy for the Freshman Seminar.

As a result, student outcomes in oral and written communication are continually improving, and individual faculty members continue to take steps to increase success, whether by providing more writing feedback, imposing explicit penalties for spelling and grammatical errors, or providing opportunities for writing revision post-feedback.

Sometimes, though, communication improvements also come in the form of better peer interaction, which leads to better outcomes.

The tough-math hurdle

In the Physics-Engineering Department, Professor Nicholas Truncale and colleagues used assessments to identify several ways to increase retention in an admittedly difficult major in which early math requisites had become a roadblock.

“This all started four years ago when we did a program review for all majors in our department – physics, computers and electrical engineering,” Truncale said.

“They really liked physics and engineering, yet we were losing about 50 percent of our students after the first year.”

The faculty engaged in a group effort, he said, to look at four semesters’ worth of data related to incoming students’ math SAT scores, high-school GPAs and scores on the math-placement exam.

Existing math requirements illustrated a case of how what was once thought a service to students can become a disservice.

Precalculus had been required of department majors taking Physics I in the freshman year, but relaxed requirements eventually allowed them to take both courses at the same time, Truncale said.

“We were setting students up to fail,” he said, explaining that enforcing the precalculus prerequisites achieved better results.

The solution was to create a “trailing sequence” through which students, depending on their math placement scores, could take precalculus in fall, then calculus along with Physics I in the spring.

“This helped a lot,” he said, but it also contributed to a slightly different problem: disconnectedness among what were now two tracks of students.

“We didn’t want to have this big split,” Truncale said.

Enter a second solution to a second problem. When the general education program added the Eloquentia Perfecta initiative, designed to prepare students to speak, write and communicate effectively in varied modes and media, departments were mandated to choose one of three approaches.

Truncale and his colleagues chose to create a new course highlighting EP requirements in addition to certain departmental objectives. Created was a Foundations of Physics and Engineering course that focuses on oral communication and digital technology outcomes.

But the course had other, hidden outcomes, Truncale said, the most obvious of which was closing the connection gap among students in different math sequences.

“I noticed a difference immediately,” he said. “The students were now hanging out together, studying together.”

The new sense of bonding then, in turn, helped increase retention, as students looked forward to taking on fun projects together, such as creating a self-navigating robot. They were now able to see what life in the department would be like “after they got past the hard math,” Truncale said.

Numbers quickly began to change.

“This year we had 32 freshmen, and we’ve only lost two or three,” Truncale said.  “Next year there will be 15 students in the new capstone course.”

They’ll assemble in a space not designed to hold nearly as many.

“I’m not sure how they’ll fit in that classroom, but that’s a good problem to have,” Truncale said.

Another tough litmus test

Meanwhile, in the nursing department, assessments also have been proving pivotal to success in an already highly successful nursing program, one in which a certification exam is the key to a future career.

Faculty have discovered that early intervention is key to helping students achieve success not only at the university but on “a very complicated, very rigorous national exam,” said Dr. Margarete Zalon, Ph.D., professor of nursing.

“The University is always charged with doing assessments,” Zalon said, figuring out “what works, what might work better, what can we fix?”

But in a clinical program such as nursing, she said, the most crucial question to ask would be: Are our graduates doing well in the licensure exam?

“The way we define it is can they pass on their first attempt?”

Though graduates are allowed multiple attempts, failure on the first try creates added stress, as loans come due, for example, and students cannot get a job until they pass.

“One of the things we do know is that while we accept students to the program who we believe are going to succeed, some students will have more difficulty than others,” Zalon said. “We wanted to ask: Can we identify students in need of support services early on?”

She and fellow faculty member Mary Jane DiMattio found in their assessments that grades in certain early courses, such as freshman science courses, are predictors of exam success.

Students who struggle in these courses can be identified for early support and steered toward it, Zalon said. That part can be challenging, because some students don’t take well to learning they are considered at risk, but the assessments data lend credibility.

“When we go to a student and say, ‘You need to work on this,’ we’re not going out of gut feeling, but we’re going out of data,” Zalon said.

Faculty often perform a delicate task when students who may have done very well in high school “find the University a very different place,” she said, explaining how students are often advised to prioritize activities or revise study strategies.

In nursing, the latter can be an eye-opener, Zalon said; handling test questions based on ability to analyze a situation rather than recall information is a new skill for some.

That’s why timing is key.

“We really want to identify them (at-risk students) early, so there is time to change,” Zalon said. “Sometimes students just have to learn how to take a test.”

The nursing department began its most recent round of assessments three or four years ago, with preliminary work done eight years ago, and is now preparing for “a second wave.”

That wave will include tapping into several more University resources for help. A statistician might be able to provide new ways to look at data, for example.

“We have expertise within the faculty to do more sophisticated analyses to give us results that are more reliable,” Zalon said. “We can design really good program assessments so that we’re doing more than just patting ourselves on the back and saying, ‘Look we did a good job.’ ”