Understanding Factors Leading to College Classroom Engagement for Millennials: Development of the College Classroom Engagement Scale

This aim of this research study sought to define college classroom engagement from the perspective of Millennial students. A mixed-methods study was utilized to understand both quantitative and qualitative findings for the purpose of developing a measurement scale. The study was comprised of two different phases. In phase 1, students (n = 68) completed a demographic survey from which researchers invited students (n = 20) to participate in a focus group. Data analysis led researchers to identify five themes from the focus groups (relevance, instructional practices, class climate, professor traits, and student traits) to create a scale for measuring engagement in the classroom. The 30-item scale was comprised of five Likert statements for each of the six themes from phase 1. In phase 2, the scale was distributed to students (n = 177) on a university campus in the US. An exploratory factor analysis was conducted from the student responses and revealed 2 factors (Teaching Persona, α =.83 and Learning Environment α =.82) describing Millennial students’ reasons for engagement in the college classroom. The final scale of 18 items suggests that college students in the research study believe that they are more engaged in their college courses when they believe that their instructor exhibits traits demonstrating care and concern for students and when the instructor creates an environment conducive to learning. This research study and resulting scale may be helpful for guiding college faculty to develop their courses to better engage their college students in the classroom.


Introduction
Student engagement, defined as a learner's initiation of action, persistence in school work, and positive affect during learning activities [1], has become increasingly important in higher education because of the relationship of student engagement to increased retention in college and to students' greater focus on learning outcomes [2]. A significant body of research has long documented the relationship between greater student engagement and positive learning outcomes, such as higher achievement [3][4][5], use of learning strategies [1], and greater self-regulation [6]. Although researchers in the field have defined student engagement differently, most agree that engagement consists of both personal factors (i.e. cognitive factors) and situational factors (i.e. SES and ethnicity) that are related to student academic outcomes [7].
Research in college students' engagement suggests that students who are academically engaged have an increased likelihood of persisting beyond the first year of college [8], and that students consistently engaged in college courses attain higher grades [2,9], especially when they have a rationale to do so [10]. In addition, Hunzicker and Lukowiak [11] propose the importance of both faculty and student roles in increasing students' level of engagement. O'Connor [12] suggests that through class participation, students are more likely to engage in their college classes. The Millennial Generation students seem to prefer more interaction from their classroom experiences [13], and evidence supports the benefits of participating in class [2,8]. Although some research has focused on college student engagement in the classroom, student reported data suggest additional research is needed to understand needs of college students of the Millennial Generation.

Personal and Situational Factors
Although the influence of student engagement can be explained by factors within the student, student engagement also might be influenced by a variety of external factors. The literature suggests that various motivational factors presented in the classroom might lead to greater engagement from students [14][15]. In order to better support and engage the Millennial college student, understanding the situational and personal factors that impact student learning, performance, and success remains important.
Because of the complex nature of understanding motivational factors leading to college students' classroom engagement, a variety of approaches have been used to examine factors in college student classroom engagement. A recent explanatory mixed methods research study indicated that personal factors, such as students with a greater sense of belonging in the classroom were more likely to report greater engagement in their college courses [16]. In addition, specific teaching strategies, such as higher order thinking group activities and lower order teacher-led activities might be more likely to lead to greater student engagement in the classroom [11].

Measures of Engagement
Reschly and Christenson [17] suggest that measurements of student engagement are in the early stages and greater theoretical exploration is needed to understand the construct better. To date, a variety of measures have been utilized to assess student engagement, such as self-report, teacherreport, and observation [18]. However, concerns exist in the literature because of the potentially biased sample of those returning parental consent forms to participate in the research and the potentially biased nature of self-report data [19]. Observational measures might be less biased, but these measures are unable to assess emotional engagement because emotional engagement is defined as using covert strategies, which take place within the students' thinking processes [20]. Most existing measures are derived from researcherdeveloped tools administered to various groups of students, instead of an exploratory mixed methods research directly to gain a conceptual understanding from the students. Although a variety of methods have been used to assess different aspects of engagement in classes, scant research has focused on a specific measure to define engagement variables from the Millennial college students' perspective.
The purpose of the current research was to gain multiple perspectives and provide researchers with a greater understanding of Millennial students' perceptions of college classroom engagement by identifying and measuring variables from the student perspective. Because the Millennial college student might differ from previous college students, gaining an insight into possible teacher actions, classroom structures, and student factors leading to engagement is important to consider. The purposes of this study were divided into two studies to: (1) determine what Millennial college students perceive to be the causes of their engagement in the college classroom and (2) to create an instrument for evaluating college classroom engagement.
Research questions in Study One: Q1: What demographic differences exist between students on campus?
Q2: What situational reasons do students have for engaging in their college courses?
Q3: Does their level of situational and personal engagement differ from one course to the next?
Research questions in Study Two: Q4: What statements can be created to better understand college students' reasons for engagement? Q5: What factor structure is revealed to create a scale for determining college student classroom engagement?

Methods
The current mixed methods research study involved a twophase research design that took place over two semesters. Participants were recruited from a liberal arts institution in the South and might have participated in both studies during the two semesters. Researchers selected a mixed methods exploratory sequential design for the two phases of the study for the purpose of developing an instrument for measuring college student classroom engagement. Creswell and Plano Clark [21] explain that in the exploratory sequential design, qualitative data first collected in Phase 1 (Study 1) for the purpose of conducting exploratory research. Then, a second quantitative phase (Study 2) is used to generalize the initial findings. The researcher then interprets how the quantitative results build on the qualitative results for the purpose of testing or measuring the qualitative findings. To answer the research questions, researchers used a sequential parallel sampling design, in which the researchers first conducted a qualitative phase to understand causes of student engagement from the students' perspectives and then utilized the findings to create an instrument to generalize results to a larger population. Students in both phases were drawn from the same general student population [22].
The first research study (Phase 1) was a mixed methods study using sequential exploratory methods [21]. Participants on campus were given an opportunity to complete a demographic survey to determine student differences. Then, selected participants from the sample were chosen to participate in a focus group. The second study (Phase 2) was a quantitative study that used themes from the focus groups to create a scale for student engagement.

Study One
Undergraduate students were recruited from all colleges across campus via distribution of a survey link emailed to class instructors across campus. Students responding to the survey were mostly female (n = 46, 68%) but were otherwise representative of the diverse group of students on the college campus in ethnicity, major, classification, gender, living situation, and age. Descriptive analyses were conducted to identify the differences in students responding to the survey. Then, the researchers selected 20 participants, via purposeful sampling, who were representative of the university's student body to participate in focus groups to discuss factors influencing engagement in the college classroom [23].
In the second phase of Study 1, five of the twenty students selected attended and participated in both the initial and follow-up sessions. Three students identified as White, one as African American, and one as Hispanic. Participants' ages ranged from 19-22. The two 1-hour undergraduate led focus groups contained questions that were semi-structured and audio recorded. In the first focus group, the undergraduate researcher asked participants to discuss reasons that they were engaged or unengaged in their college classes on campus. Guiding questions were created by the researchers and based on previous research in student engagement. During the follow-up focus group, the moderator validated initial responses with the participants.

Quantitative Analysis
A total of 68 students responded to the demographic survey sent to all students on campus (12.78% of the total undergraduate population). Respondents represented all seven colleges on campus. Students reported ethnic backgrounds as Hispanic (n=11), Caribbean (n=3), or Not Hispanic or Caribbean (n=54). A Total of 40 students identified as white.

Study One: Qualitative Analysis
The transcript data were analyzed using constant comparison analysis. Following the two focus groups, the authors, a trained graduate student, and two trained undergraduate students separately read through the data to induce codes. Following the initial reading the research team met to discuss codes, and reread the data to triangulate identified and coded portions. Using the mutually agreed upon codes, the research team identified five themes affecting student engagement: relevance, instructional practices, class climate, professor traits, and student traits.
All of the students in the focus group described the importance of relevance of the content as an important factor in their class engagement. They explained that when college instructors relate the content to their lives or future careers, they are more likely to engage in the course, as demonstrated in (1).
(1) He makes me want to come to class, he really does…And he doesn't just teach, he gives us life experiences to relate to the subject matter.
The students in the focus group also discussed specific types of instructional practices that led them to be more engaged in their classes. Students discussed instruction leading them to be more engaged, as shown in (2).
(2) So every other class we get in a circle or something and we talk. I come to that class because I know I'm going to get something out of it.
Students in the focus group also mentioned particular components of the class climate that helped them to feel at ease in the classroom, thus leading to greater student engagement in the course. Students explained that they were more engaged when instructors created an inviting atmosphere and demonstrated care about students, as seen in (3) below.
(3) I like that I know my professors. Like, they know who I am; I've had professors that talk to me in class, like "I haven't seen you in a while", "how are you doing", and stuff like that.
In addition to creating an inviting atmosphere, students in the focus group also commented on specific traits of the instructor that caused them to become more engaged, such as instructors who are passionate and approachable. These instructor traits are evidenced below in (4).
(4) He's probably my favorite professor…He just, like, is so passionate about what he is teaching, and he gets really excited about things. And he draws you in.
Though much of the conversation in the focus groups discussed the classroom and instructor, students did mention some activities that students may do to lead to greater engagement. Students offered that they were more engaged when they prepared for class ahead of time and attended regularly and showed up to class on time. These student traits can be seen below in (5).
(5) I like to have it fresh, so I'll usually set aside like two hours before to read or re-read material to make sure I understand what's going on.

Study Two: Creating a Scale to Measure College Student Classroom Engagement
The researchers used the themes from Study 1 to design a Likert-format scale for determining students' engagement in the college classroom. Once statements had been created by each member of the research team, member checking was used to determine a set of 30 statements (six for each theme from Study 1) to measure student engagement in the college setting. See Appendix A for final scale items.
In the spring semester, the lead researchers emailed all college instructors across campus to ask whether they would allow researchers to solicit participants from their classes to take the survey. A total of 177 undergraduate students enrolled in face-to-face classes across campus participated in Study 2. Participants represented each classification of students, and 73% of the participants identified as female and 64% indicated their ethnicity as White. With a sample size of 177, the case-to-variable ratio is 5.9, which falls within the recommended range and indicates that the factor analysis solution is trustworthy with stable structure/pattern coefficients [24].
Because of the exploratory nature of the data and the new approach to student-driven scale development, the Engagement for Millennials: Development of the College Classroom Engagement Scale researchers first conducted a parallel analysis to determine the number of statistically significant factors for the scale [25]. At the 95% confidence interval, two factors emerged as statistically significant with Eigenvalues of 9.05 and 1.99. See Table 1 for factor loadings of items. .45 4. The professor makes me feel welcome at each class meeting. a .59 6. The instructor encourages class discussion of course material.
.50 .44 15. I feel that during class time, the professor teaches more than just the textbook.
.68 18. I feel the instructor is approachable.
.78 20. The instructor makes the course interesting.
.65 22. The professor consistently has a positive attitude.
.71 29. I feel comfortable in this class. b .62 30. The instructor uses real world examples when explaining class material.
.49 3. I prepare for class because I know I may be called on.
.58 8. My professor knows the names of most students in the class.
.63 11. During the class the professor uses group activities that enhance/improve our understanding of the material.
. Two subscales, labeled as Teaching Persona (12.92% of the variance) and Learning Environment (10.83% of the variance), were revealed in the final factor analytic structure. Teaching Persona comprised nine statements describing the instructor's personality and characteristics, such as approachability, positive attitude, and traits describing students' favorable impression of the instructor's willingness to make the class interesting. Results suggested that college students in the research study believe that they are more engaged in their college courses when they believe that their instructor exhibits traits demonstrating care and concern for students and when the instructor creates an environment conducive to learning. Learning Environment also contained nine statements and described the structure of the class itself, such as group activities, class discussions, and accountability.

Discussion
The purpose of the current research was two-fold: (a) to determine student identified indicators of classroom engagement and (b) to develop a measure of college student classroom engagement in the college classroom that yields reliable and valid scores. In Study 1, students in the focus group indicated greater engagement in classrooms wherein faculty members used a variety of instructional practices, created a positive classroom climate, made the material relevant, and were approachable. In addition, students indicated student traits which might lead to greater student engagement. These results support previous research in that interesting and relevant coursework yields greater student engagement [9,11] and that greater student preparation and attention during class may lead to greater student engagement [26].
In Study 2, the researchers developed a Likert-format scale based on emerged themes in Study 1. Results yielded an 18item scale to assess student classroom engagement in measuring Teaching Persona and Learning Environment. Previous research in K-12 student classroom engagement indicates that engagement is defined by behavioral and emotional components, such as an instructor's assistance with incorporating learning strategies and students' own use of such strategies [1,7]. However, the results from the current research offer a different perspective on factors for engagement in the college classroom.
The students indicated that the role of the college instructor in effectively delivering the content, making the course interesting, and demonstrating care and concern were important factors in keeping them engaged. Recent research suggests the use of teacher characteristics, such as a fun delivery of course content [27], humor [28], and teacher enthusiasm [29] may be related to greater college classroom engagement. The results of the current study support the importance of the professor's attitude and instructional strategies to create student engagement in courses. Professors who produce a fun learning environment using humor to engage the students are likely to have a positive impact on student enjoyment and retention of the subject matter [30]. By providing students with a caring, flexible, and supportive environment, college professors might lead students to adopt motivational variables to influence their engagement in the classroom Gonida et al. [14], which may result in greater academic outcomes [2].
The findings from the current study also suggest the importance of the learning environment to greater engagement. College instructors, who create an environment with collaborative learning to encourage participation among students, might be more likely to increase engagement in the college classroom. Zumbrunn et al. [16] indicated the importance of creating a sense of belonging in college classrooms to increase students' motivation. The results of our study support these findings, indicating that specific instructional strategies may increase group activities and student interaction. As suggested by Zumbrunn et al. [16], this greater sense of belonging and engagement might be predictive of greater grades and persistence through college. Thus, college instructors should consider structuring learning environments to create activities to promote this sense of belonging through group work or other types of collaborative activities. When students work together to understand concepts of the course, they not only develop the sense of belonging with one another but also have a greater opportunity to engage in the course, which likely leads to positive academic outcomes [3][4][5].

Conclusion
We believe that this research, while necessarily limited in scope, will inform university instructors as they reflect on their own teaching and contemplate best practices for engaging the Millennial student. Limitations exist regarding the small size of the focus group in Study One and further validation of the scale will be required to assist researchers in understanding its applicability to the Millennial student. Once score-validated, future researchers might utilize the scale to measure engagement in individual college classes to assist professors in creating environments to enhance student learning and to aid students in persisting beyond their first year of college. Having a greater understanding of the college instructor's role in influencing student engagement might help college instructors to create a learning environment conducive to learning.
Despite the limitations, the results of this study offer of Millennial college students a greater understanding of factors deemed important to engaging students in the classroom. By considering one's own personal teaching practices and delivery of course content, the way that courses are structured to create a warm classroom climate and learning environment, and the way that students are prepared and participate in class, educators can better research and engage Millennial college students in college classrooms.

Appendix
College Student Classroom Engagement Scale Relevance (α =.72) 10. I must be present in class in order to understand the course material 15. I feel that during class time, the professor teaches more than just the textbook 19. The assignments in this course are not relevant 24. I feel that what we are graded on represents the skills/knowledge we will need in our professional lives 25. I do not know why the material in this course is important 30. The instructor uses real world examples when explaining class material Instructional Practices (α =.71) 6. The instructor encourages class discussion of course material 7. My professor is never prepared to teach the class 11. During the class the professor uses group activities that enhance/improve our understanding of the material 16