Erika Orman

Assessment, Evaluation, & Research

 

Category: Exemplary

 

Description: "The Assessment, Evaluation, and Research competency area (AER) focuses on the ability to use, design, conduct, and critique qualitative and quantitative AER analyses; to manage organizations using AER processes and the results obtained from them; and to shape the political and ethical climate surrounding AER processes and uses on campuses” (p. 10).

           

        Ways in which I have improved in this area include:

• “Effectively articulate, interpret, and use results of assessment, evaluation, and research reports and studies, including professional literature” (p. 10)

• “Facilitate appropriate data collection for system/department-wide assessment and evaluation efforts using up-to-date technology and methods” (p. 10)

• “Assess quantitative designs and analysis techniques, including factors that may lead to measurement problems, such as those relating to sampling, validity, and reliability”

• “Construct basic surveys and other instruments with consultation” (p. 10)

• “Use culturally relevant and culturally appropriate terminology and methods to conduct and report AER findings” (p. 10)

 

 

 

 

 

 

If I were to rank one of my competency areas as “Most Improved,” this may be it.  Prior to graduate school, I was very intimidated by assessment.  I had never completed a research methods course and found the entire process of assessment, particularly data analysis, very intimidating.  Not only did I find it intimidating, I struggled to see the value in conducting assessment.  I viewed assessment more as a box to be checked than a useful tool.  The statistics course I took in the first semester of Master’s coursework did not help assuage these concerns.  I observed in this class how raw data could be manipulated, intentionally or otherwise, so that the results could be interpreted in different ways.   Because of this, I had a hard time trusting what the data showed.

 

Despite my reservations, I took a course in Assessment and Evaluation in Higher Education because I wanted to learn more.  I recognized the growing importance of assessment in student affairs to justify the effectiveness of the work we do and thought having this experience would make me a stronger candidate in the job search.  In the class, we learned about survey design and other assessment methods, how to reduce error, and how to effectively use assessment results.  I became proficient using Qualtrics, an online software program for building surveys, for creating assessment tools and collecting data.  I learned about the importance of language and the order of survey questions in reducing response bias.  I demonstrated these principles in the final paper I wrote for the course, which I have included here as an artifact of my learning.  In this paper, I included my justifications for creating the survey the way I did.  An important principle of our course was that there are no perfect assessment tools, so it is necessary to accept trade-offs to create the best survey possible.  By defending the decisions I made in developing the survey, I deepened my understanding of the course principles and felt confident that I had produced something of great value. 

 

While I expected to gain those skills, I did not expect to change my mind on assessment to the degree I did; I went from being skeptical of assessment to thoroughly invested in it.  I have incorporated the knowledge and skills from my assessment, statistics, and research methods coursework in my work in several ways.  For example, I modified the survey I created for the assessment course and used it to collect data about international students at BGSU and their needs regarding programming.  I used this data to propose suggestions for a programming model for the office.  I have also become more oriented toward outcomes-based assessment and the creation of learning outcomes I wish to assess.  Now, when creating a new initiative or planning an event, I frame my thinking around what I want students to achieve and build a program to meet those outcomes. 

 

Overall, as a result of my graduate coursework, I am more curious about assessment and plan to use it in my career.  Through my experiences at BGSU, I have come to understand the importance of assessment in demonstrating the value of student affairs.  I now feel confident that I can use my knowledge and skills in assessment to do just that.  

 

 

Reference

ACPA & NASPA (2010).  Professional competency areas for student affairs practitioners.  Washington, DC: Authors.