Triangulation of Feminist Methods : Uncovering Discrimination Against Female Graduate Teaching Assistants a Case Study

Discrimination against women and the devaluation of their work occurs within our society. Academe is not immune to sexist ideologies toward female faculty, students and graduate instructors. A previously conducted study questioned whether students within female graduate instructors’ classrooms express this form of discrimination via hostile, dominant, disrespectful behavior, and devaluation. This study utilizes a triangulation of methods, both quantitative and qualitative. Unobtrusive observations, surveys, and consciousness-raising debriefings were used in order to examine the issue of sexism in academe and grant voice and merit to the experiences of female graduate teaching instructors. This study examines the methods utilized and questions whether such collection of data would be pertinent during the Me Too Movement Era.


Introduction
The goal of this paper is to re-evaluate a prior study's method of data collection. In doing so, the researcher intends to glean a better understanding of methods utilized to unearth the discrimination that may occur in Higher Education. The prior study was to contribute to the literature and research regarding women's experiences in academe, particularly female graduate instructors. This was done by studying students' behaviors within the classroom, student evaluations of scholarship, and attitudes toward women and Feminism prior to the Me Too Movement. This examination was divided into three methodological stages. Stage I included unobtrusive observations and the tallying of verbal and nonverbal hostile behaviors of the students. Stage II included a two-part survey. The first part of the student survey consisted of four article abstracts and evaluations statements for responses. The second part of the survey consisted of the FEM Scale, which measured attitudes toward women and Feminism. Stage III included a consciousness-raising debriefing of the survey population. Discussed in this article are the issues that were experienced during this study and explores potential improvement in the collection of data.

Method
The original study examined a population of undergraduate students enrolled in sections of Introduction to Sociology taught by graduate instructors only. This was done to determine the impact of the graduate instructor's sex and the students' attitudes toward women on students' behaviors in class and assessment of scholarship. Introductory classes were chosen in order to access student attitudes prior to sensitization of issue of discrimination. This was done in order to minimize potentially confounding factors.
Prior research and literature has abundantly supported the notion that students devalue women in higher education. There is what is known as a "chilly climate" for females, both faculty and students, in academe [1]. This study questioned whether this devaluation by students is expressed through hostile, dominant, and/or disrespectful behaviors within the female graduate instructors' classrooms. The research posed the following questions: 1. Is women's scholarship devalued by students? 2. Are students' attitudes toward women and/or Feminism sexist?
3. Is there a hostile classroom climate for female graduate instructors? 4. Does devaluation lead to hostile behavior? 5. Are attitudes toward women and/or Feminism factors that lead to the hostile classroom climate for female graduate instructors? The terms chilly climate and hostile climate have been used interchangeably. Both terms incorporate many behaviors, such as devaluation, dominance, and disrespect. All these forms of behavior have been explored in the study.
The population sampled included students in an institution of higher education. Specifically, a convenience sample was taken from the population of undergraduates in the Department of Sociology at Western Michigan University. The sample included students enrolled in four sections of Principles of Sociology, an introductory course taught by two male and two female graduate instructors.
The four graduate instructors' classes were chosen to participate in the study because of access, as well as the similarity of pedagogy of the instructors. All were white and similar in age. This was done to control for as many variables as possible, other than gender, which may affect student behavior. Approaching the four graduate instructors was not a simple task. The researchers experienced some resistance. One male and one female were very excited and willing to grant access to their classes; however, the other two instructors were suspicious of the study. Both were concerned that the study was going to criticize their teaching abilities. The male was concerned with the ethics of the study and wanted to see the research instrument and Human Subjects Review approval before he would agree to have his class participate. The researchers sent all four of the graduate instructors the material requested along with a letter explaining the focus and nature of the study. The cover letter reassured the instructors that the researchers was to observe and analyze student behavior only and not instructor performance. The study was a replicating previous studies.
Trust between the researchers and the four graduate instructors was important to establish due to the nature of the study and the dynamics of the participants. The graduate participant along with the researcher held similar status as graduate students and instructors. Although the graduate student/instructor culture at the institution was supportive, competition existed. This is noted for two reasons. First, to explore the relationship between the researcher and the participants of a study. The analysis of said relationship is a long ruminated topic in the field of qualitative inquiry [2]. Questioned are the issues of representation and measure of validity upon which research rests. Second, trust during the observation stage was imperative. The graduate instructor participants needed to carry out class lectures as if the researcher was not present. The researcher trusted that they did not alter their behavior in any way. This was to be established in order to achieve a neutral position. The neutrality of scientific methods allows researchers to capture the fundamental nature of the research subject and establish universal understanding of what has been observed [2].
However, Feminist empiricists argue that this is impossible and silences women.
Within all four graduate instructors' classes, 238 students were registered. Therefore, a possible 238 students could have been observed during the unobtrusive stage of the study. During the survey stage of the study, 187 students participated. Fifty-one students that were register for the classes did not fill out the survey. Fifty students were absent across the four classes and one student did not complete the questionnaire. Therefore, there were 186 completed surveys out of 187 collected. The sample population for the survey and consciousness-raising debriefing stages of the study were the same.

Stage I: Unobtrusive Observations
The study was based up two previously conducted studies. Virginia R. Brooks did the first study entitled, "Sex Differences in Student Dominance Behavior in Female and Male Professors' Classrooms" in 1982, prior to the Clarence Thomas / Anita Hill hearings which brought to light issues of sexual harassment at the workplace. Brooks sought to provide a comparison of male and female dominant behavior in the classroom and a measure of the contextual effect of sex of professor on the interrelatedness of sex, status and dominance [3]. The main difference between this study and the present study was that I used graduate instructors. Brooks' method of research assumed that "quantification of spontaneous verbal behavior in a natural competitive setting would be a more accurate index of male and female dominance behavior and of interact effects of sex and status than either self-report, role-play, or videotape methods, which might allow misrepresentation of actual behavior [3]." The verbal behaviors that were recorded by Brooks, as well as this researcher, included the following: 1. Number of times students speak, 2. Length of time students speak, 3. Number of times female students interrupt other male and/or female students, 4. Number of times male students interrupt other male and/or female students, 5. Number of times male/female students interrupted professors.
This research had added to this list a number of other verbal behaviors. This list was developed through personal experiences as a graduate instructor, as well as, through extensive discussions with other graduate instructors and professors. The result is a list of common experiences and behaviors that perceived as hostile, disrespectful behaviors. The added verbal behaviors are as follow: 6. Talking to other student during lectures, 7. Laughing at statements made by the instructor (when not applicable), 8. Sighing at statements made by the instructor, 9. Direct verbal confrontation with the instructor, 10. Speaking without raising their hand (if applicable). The nonverbal behaviors added were developed similarly to those added to the verbal behavior list. The nonverbal behaviors recorded included: 1. Tardiness (without prior faculty  The sex of the instructor, the number of female and male students present in class, the lecture topic and the time of the lecture were also recorded. The researcher recorded similar student behavior as Brooks, however, the scope was broadened by examining hostile student behavior within the classroom directed toward the female graduate instructors. Observed were three classes of each of the four graduate teacher participants, two male and two female. For the purpose of comparison, 12 classes were observed. Once data was collected, chi-square analyses was done based on grouped data from the monitored classes to find statistical significance and allow for the sex ratio in male and female professors' classes to be determined. Cramer's V was also conducted to assess the level(s) of association between frequencies and percentages. T-tests were used and data cells collapsed because of the relatively small data sets.
The unobtrusive observations were done prior to the surveying of the sample because the researcher did not wish to affect the students' behavior or sensitize the students to the issue of gender and/or discrimination. The students' exposure to the survey may heighten their awareness of what was to be observed.

Stage II: Survey Data, Abstract Evaluations
Philip Goldberg (1974) conducted the second study that was drawn upon. Goldberg investigated whether there was prejudice by women against women. He also explored how women perceived their own sex [4]. In this current study, males were included in order to see if there is any difference between the prejudice, perceptions and ultimately the devaluation of female scholars by male students versus female students. Although the literature clearly suggests that women are devalued within higher education, I wanted to build upon this by exploring the hostile, chilly climate for female graduate instructors. Goldberg's study questioned if women value the professional work of men more highly than that of women. In testing this, Goldberg compiled six articles from varying fields. His participant were asked to evaluate the scholarship of the authors [4]. For this study, the researchers complied four article abstracts related with the field of Sociology. The abstracts were taken from American Sociological Review and displayed authorship by two females and two males. To eliminate any other effects on the student evaluations, the authors did not bear any status cues, such as, "Dr.", nor school affiliation. The author's names were fictitious and culturally neutral to preclude any possibility of recognition and/or bias.
The criteria used to select the article abstracts reflected the findings in Elaine Martin's article entitled, "Power and Authority in the Classroom: Sexist Stereotypes in Teaching Evaluations." In the article, Martin discussed the findings of many studies. One study in particular found that reviewers of articles tended to evaluate journal articles more favorably when the sex of the author was consistent with the stereotypes of the discipline [5]. A study conducted by Harriet Mischel found evidence that women may be considered superior workers only when they stay within their traditional roles [5]. Therefore, I did not choose article abstracts that fell within areas viewed as male or female dominated. The abstracts selected covered the gamut of Sociological theory, statistical data and references to previous studies, but avoided gender associated topic.
The subjects in my study, as well as in Goldberg's study, were to read the booklet given to them and critically evaluate the scholarship of the articles by rating each article's value, persuasiveness, writing style, intellectual depth of article, professional competence, professional status, and ability to sway the reader [4]. No mention of the author's sex was made. This was up to students to notice or ignore. Six evaluation statements were posed to the subjects. They are as follows: 1. The author is persuasive. 2. The author has a scholarly writing style.
3. This author demonstrates professional expertise within the field of Sociology. 4. The author was able to hold the reader's interest. 5. The author holds a high professional status position within the field. 6. I believe that the article has value within the field of Sociology.

Stage II: Survey Data, FEM Scale
The researcher wanted to repeat Goldberg's study with a slight twist. The independent variables included the gender of the scholar (X1) and the gender of the student (X2). However, the field of study of the scholar as a variable was not taken into account because only Sociology articles were included. An additional difference from Goldberg's analysis was the introduction of another independent variable to the study. The additional independent variable was Attitudes toward Feminism, or FEM Scale score (X3).
The FEM scale, developed by Eliot Smith, Myra Marx Ferree and Frederick D. Miller in 1975, deals primarily with traditional sex role norms and anti-Feminist stereotypes. The scale is a measure of sexist and authoritarian attitudes toward women and toward Feminism [6]. A Likert Scale is utilized (1 = Strongly agree, 2 = agree, 3 = neutral, 4 = disagree, 5 = strongly disagree). A low score indicates high sexist attitudes. A high score indicates low sexist attitudes. According to the research hypothesis, a low score (high sexist attitudes) or high score (low sexist attitudes) may have an inverse relationship to the students' evaluation of females in academe and ultimately create a chilly/hostile classroom climate for female graduate instructors. The subjects were asked to complete the FEM Scale after they had already evaluated the article abstracts, so as not to raise awareness of the research question.

Stage III: Consciousness-Raising Debriefing
Stage III of this study, comprised of the debriefing, occurred after all of the observations and surveys were completed. A consciousness-raising approach was taken during this stage. This was done because this study is concerned with the experiences of women. It is research by a woman, about women, and for women. Therefore, this is Feminist research that uses Feminist methods. As Catherine MacKinnon argues, "consciousness-raising is the unique Feminist method because it embodies principles such as enabling women to discuss and understand their experiences from their own viewpoints" [7]. Although consciousnessraising as a Feminist method usually refers to "meetings by small groups of women over an extended period of time for the purpose of discussing personal experiences without professional leadership," for this study, it was defined simply as an "educational activity" [7]. Many Feminists view consciousness-raising as an educational activity because its product is "a new way of thinking, relating, naming, or acting" [7].
In viewing the consciousness-raising method as an educational activity, the researcher was able to be more inclusive than the usual definition of this method allows. As a part of the extension of consciousness raising, the researcher incorporated males into the study and exposed them to the experiences of women in academe, hopefully to heighten their consciousness to the discrimination faced by women. Consciousness-raising also allowed complete honestly between the researcher and the subjects eliminating hierarchy within the study. Participants were informed of the purpose of the research, the hypothesis, studies in which the research was built and the literature. The students were able to ask questions about my own views, experiences and the study. No information concerning the unobtrusive observations, elements of the survey and previous studies concerning the discrimination of women in academe was denied to the respondents. The researcher replied to their questions and comments by citing the studies and literature. Since the literature indicated that women are devalued in higher education, it was essential to establish credibility by citing references. Any statements that were made were supported by studies. Therefore, the debriefing responses were scripted as much as possible. This type of debriefing of subjects must inform and expose the subjects to women's experiences in higher education, particularly to female graduate instructors' experiences.
During the debriefing stage, an anti-sexist approach was utilized. An anti-sexist approach is usually associated with Feminist Pedagogy. It is a teaching "strategy which takes gender (race and class) firmly into account" [8]. It does not ignore the existence of inequality, but begins with its recognition. Therefore, time is not spent on "if" oppression existence, but spend on "how" it exists and "what can be do". An anti-sexist approach to debriefing respondents of a study makes gender a central issue in social experiences and validates these experiences in order to bring them into consciousness. It "shifts the focus from the realm of morality (I am not a sexist) to the realm of political practice (what can I do about sexism?)" [8]. This grants voice and opportunity for activism. The researcher did not want students to think that they were being called sexist. This would only block the students' willingness to expose themselves to the silenced experiences of women. The topic of this study was and still can be considered quite political. "The personal is political" and even potentially explosive. Thus, the researcher felt it best to conduct the debriefing with the approach of antisexist, consciousness raising. In turn, the research becomes a vehicle for change.

Practical Lessons Learned: Current Reflections
As mentioned, the researcher had combined aspects of Goldberg and Brooks's studies and built upon their research for this study. Because the studies were dated, the researcher questioned whether the findings would still be relevant in a post Clarence Thomas / Anita Hill hearings era. Are things the same for female graduate instructors in higher education as Goldberg and Brooks found them to be for female professors? Currently, it could be argued that the previous research did glean issues of sexism in higher education and would support the probability for similar results to be found in a post Me too Movement era.
The researcher concurs that the previous study's use of a triangulation of methods within the study granted a holistic exploration of the chilly climate for females. The triangulation of methods referred to the "multiple operationalize or convergent validation, the uses of three (usually) data-gathering techniques to investigate a phenomenon" [9]. The researcher concurred with Charlene Depner's argument in support of triangulation when she stated that research: "must move outside the limitations of the androcentric research tradition-to recognize what is a value in it and to move beyond this evaluation to the development of new constructs, new methods and new frameworks Feminist psychology must implement every tool at its disposal-and create new ones-rather than reject any out of hand [7]. It is believed by the researcher to be true for all disciplines and not just psychology, to which Depner was referring. The use of multiple methods in any discipline expresses a "commitment to thoroughness, the desire to be open-ended and to take risk" [7]. In putting forth an argument for the use of multiple methods, the researcher suggested that "combining approaches will ensure the increased validity" of data collection no matter which era it may occur [10]. The researcher is not suggesting that, "information constructed through different methods can simply be aggregated to produce a single unitary picture of the truth" [10]. Rather, the researcher is suggesting, along with many other Feminist researchers that "the differences generated from different research techniques are likely to be as illuminating as the similarities" [10].
It is concurred that the triangulation of methods enabled the researcher to "link past and present, data gathering and action, and individual behavior with social frameworks" [7]. Through a triangulation of methods, the study was able to "illuminate previously unexamined or misunderstood experiences" and offer some suggestions on how to overcome some of the problems women face in academe [7]. However, this research was not without its limitation.

Limitations of Stage I, Unobtrusive Observations
A major limitation of the study being examined is the paucity of data concerning race, and the effects of racism on minority female graduate instructors. No minority graduate instructors' classes were included because there was only one black female graduate instructor and no black male instructors. Therefore, the researcher could not make a comparison. This does not diminish the issue of race as it pertains to this topic by excluding it from the study. The researcher did not feel comfortable, as a white woman, interpreting the experiences of minority women without proper representation of minority women and men within the study. However, a sample was not available.
Many traditional, Feminist theories and research have rendered people of color invisible. It has been argued by black Feminist, such as bell hooks, that "the word women is synonymous with White women, for women of other races are always perceived as Others, as de-humanized beings who do not fall under the heading woman [11]. The generalizations of the experiences of the white middle-class women that participated in the study should not be applied to the experiences of women of color, nor should they be trivialize. Women of color in higher education experience discrimination based on race as well as gender. For example, black women are more often placed in the spotlight, given more responsibilities that white colleagues, have their competence questioned more often, experience external and internal conflict, and are expected to make all problems racial [12]. Studies have clearly demonstrated that "black female faculty represent a very small part of the instructional personnel in higher education" and "have not succeeded as well as black men, in terms of absolute numbers" [8]. This is indicative of the impact of both sexism and racism. As Yolanda T. Moses (1989) stated in her study entitled, Black Women in Academe: at the intersection of race and gender stand women of color, torn by the lines of bias that currently divide white from nonwhite in our society, and male from female. The worlds these women negotiate demands different and often wrenching allegiances. As a result, women of color face significant obstacles to their full participation in and contribution to higher education. [13].
It is for this reason that the previous study does not assume that the experiences found to be true for white women are true for women of color. Although the study is a Feminist study, done by a woman, about women, and for women, it is limited. Clearly, the study refers solely to white female and male graduate instructors, since their experiences are included. The researcher continues to argue that attention should not be deflected away from the fact that black women continue to be "extremely victimized by both racism and sexism" [11]. Commodore, Baker and Arroyo in their book, "Black Women College Students" (2018) support the previous findings and argue for a specific guide to success for women of color [14]. Therefore, this researcher supports the emphasis that the previous study does not pretend to represent the experience of all female graduate instructors and must be taken as heuristic [14].
This researcher continues to agree with Alcoff and Potter (1993) and Bagihole (2004) in their perception of a strong consensus among Feminist today. No matter what the theory or methodology, the consensus is the commitment to Feminism to assist in the liberation of women. If Feminism is to take on this task, "it must address virtually all forms of domination because women fill the ranks of every category of oppressed people" [15]. This will be the project of future research of female graduate instructor experiences in the classroom. Since the "personal is political", this researcher will have no choice but to assist my sisters in seeking "to unmake the web of oppressions and reweave the web of life" [15].

Limitations of Stage II, Survey Data
A limitation of the second stage of the previous study was that the respondents to the survey may have determined that the survey was designed to evaluate their prejudice against women and, therefore, may have responded in a less genuine and less valid manner. The researcher attempted to control for response bias by placing the FEM Scale at the end of the survey, requesting that the respondents complete the survey in the order that it was placed, and requesting that once the survey was completed that they hand it in and not go back to adjust any of their responses. Nonetheless, some respondents reading the FEM Scale statements prior to the evaluation of the article abstracts. The researcher also found a few respondents changing their responses to the evaluations after they had completed the FEM Scale.
Another limitation of this stage of the study concerns the article abstracts. In retrospect, the article abstracts that were chosen may not have been as gender neutral as hoped. The subject matter covered in the abstracts was also an issue. For an introductory Sociology course, the topics may have been too complex and indigestible for first year college students. If the students were unable to understand the subject at hand, how could they be able to evaluate the abstracts? The respondents may not have been able to understand the subject matter of the article abstracts, and were afraid of pointing out the negative when they were not sure what the negative was. This may have resulted in a large segment of the sample population reporting neutral responds.

Limitations of Stage III, Consciousness-Raising Debriefing
A major limitation of a consciousness-raising debriefing is its potential of becoming an explosive situation. When discussing subject matters such as sexism and discrimination, it is easy for individuals to become indignant and defensive. This will only preclude informative discussions. A possible solution may be a structured outline of possible topics and studies to discuss during the debriefing. This may assist the researcher in directing the discussion to enable consciousness-raising to occur. This outline can be referred to if the researcher finds it problematic to initiate discussion, or if the discussion becomes heated. However, avoiding issues may not be conducive to consciousness-raising. If the participants are willing to discuss an issue then it becomes open for exploration even if it gets heated. The researcher believed that a structured outline could be useful, since consciousness-raising may be quite difficult. For three of the four debriefings, an unstructured outline was utilized, using solely notes from the literature review. The researcher experienced many occurrences of loss of control, while using the notes. This shut down discussion and is currently argued to silence of voices. During the fourth debriefing, a very structure outline was used. This method granted validity to the study. However, it did not open the discussion to experiences. This contradicted the intent of the methods used for data collection.

Conclusions
Both qualitative and quantitative methods provide insight into research questions. However, many of the qualitative finding in this study gave us more insight on the experiences of female graduate instructors. For example, the observations and consciousness-raising debriefing experiences demonstrated that the idea that women have made it and are no longer subjected to a climate of oppression within higher education is a myth. This buttresses the ideological hegemony that dominates our society by understanding the discrimination against women in academe. Such disregard for the discrimination that occurs against 52% of our population only perpetuates sexism within the very place where we should know better. The devaluation of women's ideas, methods, theories, and overall contributions to their field allows for, if not encourages, the silencing of an entire group. By viewing research conducted by women as worthless and/or unworthy of attention, as did some of the male students in this study, we continue to view women as secondrate scholars. The researcher believed this to be reflective of the negative association with anything female in our patriarchal, misogynistic culture.
The findings, quantitative and qualitative, initiate an abundance of other research questions and may also be utilized to grant voice to those usually silenced within academe. Further inclusive research must unearth experiences that are more diverse. Once uncovered and included, we can begin to understand the variety of experiences. Many more voices will be and must be heard so that the silencing and marginalization can end.