Texas Tech University

Collegiate Assessment of Academic Proficiency

2016-2017 Writing Skills Test

Downloadable PDF

EXECUTIVE SUMMARY

This report contains results from the Collegiate Assessment of Academic Proficiency (CAAP) Writing Skills Test form 13-A. Scores were obtained from a sample of 211 students (freshman = 111; senior = 100). Analysis of the results indicates that on average, students scored at the level of their respective normative group. Therefore, the established benchmark of performing at or above the normative group was attained for all classifications.

The Collegiate Assessment of Academic Proficiency is the standardized, nationally normed assessment program from American College Testing (ACT) that enables postsecondary institutions to assess, evaluate, and enhance student learning outcomes and general education program outcomes.

CAAP can be used to:

  • Satisfy accreditation and accountability reporting requirements
  • Measure students' achievement levels on a group and individual basis
  • Compare students' achievement levels with national user norms
  • Evaluate the strengths and weaknesses of general education programs
  • Document the performance gain of students' achievement levels over time

The CAAP Writing Skills module assesses students' knowledge and skills in written English. This module contains 72 items that measure the students' understanding of content in punctuation, grammar, sentence structure, and rhetorical skills. The test is composed of six prose passages that are accompanied by a set of 12 multiple-choice questions. The CAAP Writing Skills Test is administered to a representative sample of students and measures students' core curriculum competency in written English.

The CAAP is administered to a national sample of students and scored to establish a benchmark measure. The benchmark serves as a point of reference to which institutional scores can be compared. Benchmark measures for the CAAP Writing Skills module have been established as being at or above the national average for the classification being tested. Table 1 shows whether benchmarks were met for each classification tested.

BENCHMARK
Freshman At or above CAAP national average for Freshmen Yes
Senior At or above CAAP national average for Seniors Yes

The test was administered to a random stratified sample of freshmen and seniors at Texas Tech University (TTU). A pre- and post-test strategy was used which tested freshman students in the Fall 2016 semester and senior students in the Spring 2017 semester. A breakdown of students by college is provided in Figure 1. Courses were chosen based on enrollment by student classification and size. Freshmen were tested from sections of IS1100: RaiderReady, TTU's freshman seminar course, whereas senior courses were chosen based on capstone status. These are culminating courses in which senior students are required to enroll for their degree program.

A new testing strategy was implemented for this administration to improve both participation rates and effort given by students on the assessment. This involved scheduling hour-long testing slots outside of class for students to voluntarily participate in CAAP in order to receive an incentive. Scheduling of test slots was done in partnership with TTU's Academic Testing Center during the freshman administration, but a move was made to schedule senior testing times in OPA's conference room in the spring. A total of 211 students participated in the CAAP Writing Skills Test, of which all 211 tests were valid for scoring by ACT.

Results of the number of students from each college who participated in the study

RESULTS

Table 2 provides a summary of CAAP scores by student classification. Scores for both samples were averaged to arrive at a mean score by classification. Both classifications tested did not score significantly different from the national mean, resulting in the conclusion that TTU students met the benchmark of at or above the national mean for the Writing Skills module.

Summary of CAAP Scores by Student Classification

  n Sample Mean SD National Mean SD
Freshman 111 61.7 4.2 62.3 5.2
Senior 100 61.9 5.0 62.3 5.2

PERFORMANCE BY QUARTILES

Student performance on the CAAP Writing Skills test was also classified by quartiles for freshman and senior students. The first and lowest quartile encompassed national percentile scores of 1-25, the second quartile scored 26-50, the third quartile scored 51-75, and the fourth quartile scored 76-100. Of particular relevance are the students whose scores fall in the lowest quartile (Q1) relative to the national percentile. Of the total number of students tested, 25.3% fell within the lowest quartile for the assessment. Only 16.2% of the overall sample scored within the highest quartile (Q4). In spite of these results, on a supplemental self-reported performance question, 50.4% of students rated themselves as “Tried My Best” and 34.7% of students rated themselves as “Gave Moderate Effort.” Below, Table 3 depicts the percentage of students in each quartile by classification level:

Quartiles Freshman Seniors Institutional
Q1 22.5% 28% 25.3%
Q2 32.4% 27% 29.7%
Q3 29.7% 28% 28.9%
Q4 15.3% 17% 16.2%

CONCLUSION

The overall findings from the analysis indicate that Texas Tech University students are performing at the national average in the core curricular subject of writing. However, nearly a quarter of the students assessed performed in the lowest quartile relative to the national percentile whereas a significantly smaller percentage performed in the highest quartile. It is recommended that the core curriculum committee, in conjunction with faculty and pertinent administrators, consider these results in order to enhance the educational experience and continue improving student learning at Texas Tech University.


GLOBAL COMMUNICATION ASSESSMENT

This fall and spring, the students who participated in the Collegiate Assessment of Academic Proficiency (CAAP) administration were also given a set of supplemental questions to assess their preparedness for global communication. These questions are used as an additional assessment for Texas Tech University's Quality Enhancement Plan (QEP) Bear Our Banners Far & Wide: Communicating in a Global Society. This report details the results of the assessment.

Q1: How confident do you feel communicating with people from different cultures?

answer results from question one of the assessment

Q2: How confident are you communicating when language barriers exist?

answer results from question two of the assessment

Q3: How confident do you feel discussing your own culture with others?

answer results from question three of the assessment

Q4: How confident are you that Texas Tech University is preparing you to be an ethical leader?

answer results from question four of the assessment

Q5: To what degree does your own culture play a role in your life?

answer results from question five of the assessment

Q6: How often do you interact with others from different cultures?

answer results from question six of the assessment

Q7: How confident do you feel with new perspectives other than your own?

answer results from question seven of the assessment

Q8: To what degree do you dislike learning about new and different cultures?

answer results from question eight of the assessment

Q9: How often do you stay informed of events happening in other cultures?

answer results from question nine of the assessment

Q10: Using the space provided, briefly answer the following question: How do you expect your degree to prepare you to interact with others?

Upon analyzing the overall responses from freshmen and seniors, there was a significant shift in the quality of responses received from senior students. Freshman students tended to provide a vague response regarding their degree helping them to communicate in general, whereas seniors were able to provide specific assignments, courses, or experiences that strengthened their communication skills. A few responses from both classifications demonstrating this shift are given below as examples:

Freshman:

  • Prepare me by teaching me good communication skills
  • It will give me knowledge to be able to best communicate with others so that together we can accomplish a goal or task.
  • Being a nurse, that is what you do everyday therefore I know that my degree will help me immensely interacting with others

Senior:

  • As an Art History major, I have learned to look at life and situations from different perspectives. It has enabled me to broaden my understandings of different cultures and how to interact with other people with different views. It has also taught me to be considerate of where they came from.
  • My degree, CFAS, has prepared me greatly to interact with others. So many classes I have taken in this major are all about communication, especially with individuals from different cultures. Even my minor, psychology, has given me valuable tools of communication! I feel confident in my ability to be an effective and conscience communicator.
  • My degree is in business management and international business. My classes have taught me about ethics in the U.S. as well as abroad and my study abroad experience has helped me understand cultures. My degree will be very helpful in my career.

Office of Planning and Assessment