A Teacher and Researcher: A Scratch on the Science Community and Meaning of Evaluation with the Research Doctoral Programs Ranking
Kiyoung Kim1, *, Hyun-Meong Ju2, Marium Khatun2
1The Legal Research Institute, College of Law, Chosun University, Kwang-ju, South Korea
2The Legal Research Institute, Chosun University, Kwang-ju, South Korea
To cite this article:
Kiyoung Kim, Hyun-Meong Ju, Marium Khatun. A Teacher and Researcher: A Scratch on the Science Community and Meaning of Evaluation with the Research Doctoral Programs Ranking.International Journal of Philosophy.Vol.3, No. 4, 2015, pp. 34-46. doi: 10.11648/j.ijp.20150304.11
Abstract: The epistemology and phenomenology of contemporary society tend to be deepened, and the philosophical challenges never are minimal that we may be called to face with the kind of post-modern chaos from the rapidly changing phenomena of the global community. The ballast held on the identity of faculty members as a teacher and researcher now turns due so as to be recast with our intrinsic of routine performance. I considered their quality as bent on the intellectual strife on the method and the kind of attitude, say, evaluation and consultation. In this paper, the authors have presented some thought and implications that triangulated the triad, i.e., research methodology, program evaluation and consulting illustration on the college research program rankings. The author is hoped that the discourse can help the academicians to share the attribute of different methods as well as the dimension of evaluation and consulting, perhaps essentially related with their work role in terms of teaching and researching.
Keywords: Research Method, Qualitative Studies, Mixed Method, Program Evaluation, Theory and Philosophy, Program Rankings, Consulting, Journal Writing
1. On the Research Method
One can ponder on the act or attitude, "why we research?" There would be a plenty of response. Somebody may say the organization he manages recently received a funded project on the poll of coming election to the profits of his survey firm. Others would admit, "He now prepares his credentials of tenure review in which the research performance and scholarly activities are crucial component for final decision." Some others would introduce his clinic for consulting and program evaluation dealing with a particular conditions or specific purpose of in-depth situation. The graduate students should do it to complete his degree requirement (Kim-2, 2015). On the common attribute of whole of these cases, we see that they treat them a body in pursuit of some dimension on intelligent curiosity. They are driven to strike that agonistic into the kind of rational order. The subjective and objective conundrum involving the quantitative and qualitative studies partly relates with this backdrop (Patton, 2002; Saldaña, 2011). The body, mind, and spirit—such trilemma in the view of classic philosopher-- also has pertinence in terms of understanding the methodological debate. It is, for this reason, a threshold question to ask, "What do we like to know if we begin our research?" It is also in this context that the researcher himself is most determinative as someone like a seaman on the steering key of how to design a research plan and what method he chooses. As Patton guides, the status of researchers is one factor to choose among the traditional triad --quantitative, qualitative, mixed-- that the graduate students need to consider the supervisors of competence and auspice in progressing his dissertation work (2002). Nevertheless, the nature of inquiry and attribute of topic -- most importantly the "curiosity of researcher to know what"-- would preferably govern a selection from methods.
In the cross-section of different methods, there are a scope of points or views already debated on the trait, strengths and weakness. Even a contention is well noted in which each side could be skeptical of other way of knowledge in terms of the scientific force to vindicate on phenomenon or occurrence. I have some thought rather intuitively. As we see in the science citation index, most researchers work on natural or engineering science. The medical science had flourished and perhaps continues or will continue to prosper, which perhaps could be captured under the umbrella term of natural science. The two most leading journal titles, "Nature" and "Science," are not irrelevant with this present status. The methods on social science, neighborly with the humanity, would actually be less sizable or organized—at least pluralistic in conception of researchers—enabling to stand on different approach to resolve the curiosity and to expel the thirst of our sensory agnosticism (Hunt & Colander, 2015). The reality of UFO is some of most popular thirst whether it actually exists or the aliens of other universe would live like us. We never undertake – or at least dislike -- the survey method in resolving such thirst which is perhaps because the survey method is received so as not to completely solve the question. This example thrusts several implications (i) the natural science or its method would be most accurate and convincing as we often agree (ii) the quantitative method is related with political or social context of knowledge which we could share with possible imperfection (iii) there are a wider scope of knowledge province actually blurring the traditional notion of method proponents.
2. An Excuse for Qualitative Studies
While we acknowledge the principal service of research is to compile the data basis and scientific knowledge, the evidence often is alleged a key strand to support the research work which point has been intensely argued by the quantitative circle (Laureate Education, 2010e). It is seemingly undeniable that the qualitative truths are some taste of literature or novel-like understanding of exterior world, which, however, differs from its systemic analysis of interview result and the kind of coding system with the aids of computerized program. Given the literature can possibly satisfy the curiosity of, and thirst for the deep humanity, it could stand alone on the utility as university department, but would be made a borderline case with the social science because of evidence. We normally would not expect an evidence for the novel writers. With a similar thought of dealings, the history and literature researchers would often be more descriptive and autocratic than evidence-reliant or without the quantitative information—but comparative in cases-- except for the sensitive issue of controversies. The setting of literature or novel writers would provide a useful comparison that brings our think-point around the strands of research method, such as fact, belief and knowledge. The literature and novel often would not be a fact -- of course with exceptions-that lacks the quality as science or social science (Gardner, Lawn, Ridi, Schakel, 2012). It also can be seen as separated from the normal understanding of knowledge if it is not fact- based. Generally we could not draw upon that source to form our belief system although we may get hallucinated with a fantastic love scene of Gone with the Wind or ego forming of juveniles with the mighty Robocop in the cinema. Nevertheless, the human agent affected from the literature and cinema personally will experience or share the same intelligent process with the reality—in some deep dimension of his ego. It could be a fact, belief and knowledge—of course in his subjective dimension—to be utilized to determine his personality lifetime. As one fusion of our notions, we may illustrate the case of "science fiction" which is a popular source of Hollywood cinema. It combines the scientific backdrop with our imaginary story which shows the current intellectual taste of people.
Around these examples, perhaps extreme as bootstrapped with the methodology of social science (Hunt & Colander, 2015), we can imply a relativity and balance apart from any absolutism on methodological query (i) the scientific truths are constructive in concept and explains a part of human dimension besides the society—most immediate object to be investigated (ii) both methods probably are not definite, but instrumental to provide a professional platform for the social scientists. Given the imperfection around statistical assumption or bias of in-depth inquiry, we consider it nearest to the truth earned from the natural science, and we would be dormant to stress the use of natural knowledge for the better picture of scientific living. Nevertheless, it is surprising that EU and US, most advanced countries of science, do not agree on the impact of hormone-growing cows and meat on human body. It is one of medical issue, but had been disputed in the shoes of WTO laws-- perhaps pivotally related with the evaluation of desired human condition—the kind of social standard and professional belief. In Wyeth v. Levine, the use and labeling of gangrene injected with Phenergan, an anti-nausea drug made by Wyeth Pharmaceuticals, were debated that the social standard to provide a warning would differ varying with the locale and people—even the pharmacy or medical experts themselves for the extent of public interchange. This illustrates that even the absolute truths of natural science would turn to become relative in the force of persuasion given its application into the social and human dimension. The scientific truths have a meaning only when we can be constructive with the compromise and common assumption. It is a part of human dimension as we see them left with other working professionals or researchers, who make the people knowledgeable or create a belief system as in the case of literature or history students and judicial bench tasked with the comparative examination of documents and prior cases in relevance (Laureate Education, 2010e). I am not sure, nevertheless, if the history and judicial science can have a class as qualitative studies whatsoever. In this stream of understanding, we may also illustrate the psycho-therapy or analysis as a broach within the qualitative method. After all, however, we would be unwise if to miss emphasizing the thought process and general attitude embedded within the traditional two methods on social science. The attitudes "symbolize the community of social science as us" and allow the platform as a social scientist in the society. It needs no further mention that the training on this method is any more important for the research students and interested professionals.
3. A Comparative Thought on Two Methods
One stigmatic focus on the methodological reflection can come contrasted between the number and story as Patton guides (2012). He also illustrated historic figures often held greatest in the world history, in which Marx on English factory, Darwin on Galapagos tortoise, Jung on dream and so, would be composed into the understanding of humanity and society (2012). The number is generally conceived as most accurate and certain that we even need not cite the greatness of Pythagoras. This probably would be a most element that the quantitative researchers could convince himself his intelligent process to prove his research question (Creswell, 2009). On the while, the qualitative researchers would focus directly on the human agent to unearth truths (2009). This leads to a comparative difference on the aura and propensity of research work.
First, the quantitative studies stretched outwardly -- hence more societal -- while the qualitative studies tend to project into the human agent, what is a determined component of society and end meaning of social science research.
Second, the scope of research coverage possibly could have a different characteristic between the average and selected class.
Third, as aforementioned, the quantitative investigation could bring a consequence that it would be more easily adaptive with the political and social context of research issue. This does not necessarily mean that the wider exposure always is guaranteed of quantitative studies because, for example, the media may intervene for the issue of African poverty or Sepp Blatter with the bribed FTO leaders in 2015—a good source of qualitative research-- more in focus leading to public awareness. Any popular research findings would not only stem from the quantitative investigation, but from the qualitative inquiry.
Fourth, the strengths of each method depends on the nature of topics and research design—hence, case by case basis to select for most effective research outcome. For example, the deep investigation of Supreme Court justices on his or her propensity could be more properly framed with a qualitative approach utilizing in-depth interviews and documentary examination with his timeline of significant decisions. The behavior and living mode of "aboriginal tribes" could be delved more convincingly in ground theory and long observations than scaled survey inquiry since they would often not be susceptible of usual generalization from "original countries," what we say of Europeans (Creswell, 2013).
Fifth, the difference in major tone of each method would explain its use that the quantitative studies would serve the test of existing theory while the qualitative researchers employ the method to develop into a theory building as we read in the article on negative leadership.
Sixth, both methods can share an eventual destination as we often encounter in the abstract of journal articles and key terms. The qualitative researchers seem to have a more trait in affinity with this style of presentation—i.e., abstract and key terms, since he primarily works on his key thesis and with a long indulgence or observation as illustrated in Patton’s historic scientists. In this phase, we may note the importance of lead author on the articles and linguistic differences in terms of the implications of scientific study. In this dimension, the qualitative methods could be convoluted more than quantitative one, but also could be a solution for any meaningful deals with the difficult process of coding or word magic in the research operation. As the terms of art are uttered to ascribe the work of Supreme Court justices, the qualitative researchers would be stuck on the key words and seek to supplement for the abnegation or reluctance in the interview process.
4. A Focus on Qualitative Inquiry from Patton
Given the distinction between two major traditions, the qualitative studies would be exposed to several tips for effective research operation (Patton, 2002). First, the qualitative researchers have to be more minded and focused, "how illuminate the meanings." Second they need to study how things work as we see the evaluation of program. Michael Scriven gave an insight, "evaluation is the process of determining the merit, worth and value of things, and evaluation are the products of that process." Third, capturing stories to understand people’s perspective and experiences has to constantly be minded which characterizes the qualitative studies qualitative. As Patterson cited from Rukeyser and Gottschall, the ending point would be a universe and human beyond the politics, "the universe is made of stories, not atoms," and "stories make us human (p.7, 2002)." Of course, this is not to say the qualitative studies are totally irrelevant with the politics and social activism. Fourth, it is no fullest merely to take a part of system, but the qualitative studies are required to elucidate how systems function and their consequences for people’s lives. Fifth, the context has a central importance than numerical order that the qualitative researchers investigate how and why it matters. Sixth, as the kind of toppling, the qualitative researchers take an extra process or final touch so as to identify the unanticipated consequences (2002). This is the point that we admit an imperfection of social science research and honestly open the question for the future consideration. The attitude of federal government in their work process also underlies same element, for example, "niggardly but expansive on regulating the FCC authority by Congress" or waits to see on "deference rule" for the net neutrality policy by Supreme Court in the NCTA decision (National Cable & Telecommunications Association et al. v. Brand X Internet Services et al., 2005). This attitude is progressive and incremental on the kind Darwinian beliefs. Seventh, the qualitative researchers make a case comparison to discover important patterns and themes, which emanates a same feel in terms of judicial analysis of precedents in the creation of judge-made law.
These days I had time on my video lecture conducted in English. Since we teach in Korea, the Korean language is the most popular and natural medium of instruction. The Korean Open Course Ware is the public program managed by the government support foundation, which provides lectures of Nobel Laureate, members of the Korean Academy, university professors and lecturers. It collected over thousands of public lecture and scholarly articles at the public availability. It is the kind of Korean MOOK, the concept toward the universalized public education without a barrier. A small number of lectures are available at non-Korean language and subsidized from the foreign source, such as Indiana University and UC Santa Barbara. Some of Korean professors contributed his or her English lectures, one of whom is myself. The lecture was originally recorded with less than sound that embarrassed me about its disservice for the audience. I have not known if it could be cured. My wife readily helped that we could buy an amplifier from the computer store. It then could be audited in normal sound that we had an unusual time to listen the English-based lecture. It perhaps would be impressive if a mid-aged madam usually does not encounter the lecture of professors, even seldom with that of spoken English. Now it is the time of her precious comment, "It likely sounds your English." I originally had expected to hear "It seems likely from native speaker" or "it is less proficient to allow a guess if the speaker is foreign-educated other than English speaking countries." Her response was unique, but I soon realized in the context of weekly course objective.
The qualitative researchers are often described as subject oriented and put an emphasis on human’s whole story as a component of universe. It is projected and enduring, a part of deep dimension less frivolous nor rectifiable comparatively than the assumption of research sample of quantitative researchers (Kim-1, 2015). Simply, we may be less waked when we reply for the five scale of surveys. We would be more certain and affected deeply with the gunman incidents involved with narrative studies or ethnographic research. We generally ascribe as "intact cultural group" when we begin with our qualitative design of research. We perhaps speak "intact" to describe "unstudied or unearthed." Ironically, "intact" seems more adequate to describe the subjects of quantitative studies if received in usual use. The participants of quantitative studies would truly be intact as if we respond with the Walden survey of class evaluation at the end of quarter, election poll or public survey about the policy aftermath upon the occurrence of Sewol ferry tragedy of Korea last year. The in-depth investigation of victims’ family and close friends concerning the essence of Sewol incident would undertake a different quality of research (May & Malcolm, 1996). In this case, the scope of participants would not be intact, but deeply affected, many of whom suffer from trauma and embarrassment. Some of them would feel like better to commit a suicide, but also truly intact if they are not studied. It seems the kind of equivalent with the gun shooting problem in US settings.
We often tell that the victims of this kind would be our neighbors and precious components of society. We often have a focus if the qualitative research has a worth of study more frequently than those of quantitative approach (Laureate Education, 2010e). Of course, professionally speaking, it may be contested in the competitive process of NSF grants or other funding institutions. My point is that they are not only neighbors of sharing, but also reveal the kind of intellectual destination that we are said to be truly intellectual. I have once introduced my studies on Korean constitutional court, and we know the modern focus of European philosophical curiosity, such as post-modernism, existentialism and so on. Creswell also illustrated the intellectual stream from Husseri through Heidegger, Sartre, and Merieau-Ponty as to relate with the phenomenological research (2013). The modern being may be extant under the circumstances of affectation that the risk society from Ulrich Beck is not the story of others. We Korean people had long been affected to have a judiciary of advanced modality, and 1987 Korean reform of constitution had truly been momentous. The focus group or ethnographical scope would be less general nor normalized given its intensity on Korea, but thankfully was considered a worth of study. Without a deep awareness of affected people and sharing, the research scheme may eventually go futile (May, Malcolm, 1996).
The context above sketched can be summarized (i) in-depths dimension of truths (ii) affectation, rapport or sharing (iii) intellectual standard as pertinent to understanding the status of qualitative studies. In terms of sharing and intellectual standard, one note needs to remark on the current transformation of e-age that the rapid growth of on-line journals would be illustrative. For example, India or Chinese based on-line journals likely become rampant to create their own circle of professional communication beyond the traditionally indexed journals. The open access movement for professional sharing on articles and books is another trend as notable. KOCW is one of example now in service for the public.
Let me return to the first example to reflect on the philosophy of disciplines—perhaps – and basis of qualitative method. As known, Creswell perceived that, in terms of qualitative method or phenomenological studies, the subjective and objective dichotomy prevailed over the enlightenment age as a intellectual basis would be less adequate to understand the humans and universe, say, the kind of society on which we often elaborate if setting aside the inside space of Space-shuttle or Moon Explorers (2013). Plainly we can retrospect how the work of geography department, one of social science disciplines and adjacent with anthropology, would have an interest and we come to realize that their concerns are not merely a work of cartography or description of physical trait between the urban and rural areas. The element of human is common across the disciplines of social science although the assumption of human is made a little different between two methods (Creswell, 2009; Patton, 2002). In the qualitative studies, the assumption would be thicker, particular and dimensional that may be ascribable to the Greek paradigm, say, whole of being, but more prototypical than social. This may be used as a basis of skepticism from the quantitative circle of adherents. The qualitative researchers may counter if the essence of humans could be so abstract and neutral between the subject and object. For them, the exterior world, perhaps object for the subjects, is likely consumable to the life and humanity. The object is theirs as affected and less separable, perhaps said less cool, but heated. A journal writing in the previous lesson would have a focus on this aspect. One other factor could be related with the medical facts of human element, who would be aged, infirm and eventually die. The phenomenological studies would have a service for the discipline of nursing science, public health and education (Dahnke & Dreher, 2010). In this aspect, we may see if the practical reality of humans would be more persuasive with the interviews and lengthy observations than computer aided marking of public survey.
The instant utterance of my wife is very interesting to disappoint my expectation. It was "your English" than general comment. The object and subject are immersed in this case as the qualitative focus would highlight. The comment also survive many potential Korean English speaking persons, perhaps herself, who should speak English in this highly globalized community. The comment entails "sharing of English" and "affectation" as a Korean foreign. It showed a "deep engagement" since we had time for twenty minutes in listening. Most importantly, the object and subject were not coolly separated that "immersed me and general context of English-based lecture." My expected comment would go otherwise to make me stand among the two objects --lectures of native speaker, English lecture generally, and one subject -- myself. In her comment, no objective scale can intervene about proficiency or native manner of lecture, but merely one man of existentialism. Her comment eventually made me laugh, but it seems to locate the attribute of qualitative research, and the kind of philosophical understanding. The qualitative research is surely to be attested to by multivocal discourse.
6. About the Program Evaluation
In the strategic changing process, the managers or leaders wish to know how they go or what programs are implemented in a satisfactory or unsatisfactory manner (Merriam, 1998). This would be important to assess the wake of progress, the present status of organizations or programs, as well as to explore any plan to improve them. The concept of evaluation recently turned to be highly attended in the face of increasing complexities of human, organizational or public performance. The concept may begin with a class evaluation of instructors at colleges and universities, rating of assembly or congressmen, evaluation or rating of countries and firms for their credit and so on. The work on evaluation, most powerfully organized into the work frame of program evaluation, is seen, in my view, to be most proximate with the intrinsic and attribute of teaching and researching. It likely is the culmination of methodological theory and concerns that was applied to the community. Hence, my focus turns on the program evaluation and their role of student counseling or consulting for the further study within the college and graduate programs
The philosophy and frameworks are crucial in studying the social science since the researcher is an agent, not a discoverer, who is neither absolutely intact nor prototypical and innate to approach, interpret, analyze, constructive and should be consistent and integral through the end of research (Patton, 2002). He himself is one of social constituents, and learned being (Hayes, Barnes-Holmes, Roche, 2001). Two points are remarked (i) the issue of philosophy and frameworks generally influence or govern, in many senses, not only part of research -- such as literature review and headfirst discussion on the independent section of philosophical assumptions or interpretive frameworks -- but the whole of research process, say, data collection, analysis, discussion and suggestions (ii) the philosophy as a lens for the scholarly taste would not only be variable objectively, but also possibly change subjectively with the career development of individual researcher (iii) the claims of grounded theory research would take a researcher more seriously that generally desires of him as one active participant and that is expected of no bias or predisposition from the standing mainstream of knowledge (Creswell, 2013).
For example, we can consider the first point relating with the data collection. The feminist or disability researchers may prefer a photo or audiovisual materials to make their assertion more concrete. The pragmatist may prefer the case study to solve a practical solution for the bounded system. The post-positivism user will be more oriented to the documentary examination, for example, as related with the legal research, than other approach of data collection since the elements would be reductionistic, logical, empirical, cause and effect-oriented and deterministic on a priori theories. On the second aspect, I may illustrate between the juridical scientists and jurists. Within the system of law school, the doctoral degree on juridical science is a highest degree that the JD degree holders will study after his graduation. We say, therefore, a graduate or research degree in law for the students who study within those programs -- LL.M and SJD – other than JD program, often called as law school. Both degree holders conduct a legal research. The frameworks or philosophy may keenly be related each other on one hand since they treat the law or legal subject commonly, but little differs from other aspect. The statistical data and interdisciplinary perspectives are more demanded of such higher degree, and the style of research product may apparently be discriminating in cases. The diverse lens can be employed and encouraged to employ by the supervisors, ironically normally professors with the JD degree. A later development of law faculty in his career path may breed them to be interdisciplinary and diverse in terms of scholarly lens, but vastly unlikely in reality, which implies that the LLM and SJD are principally a foreign purported degree by teaching the basic of American concept of law, expecting the art and science intelligence than professional education as comingled with the basic legal knowledge, and finally seeing them to become a professor in their home countries. In other cases, the growth of scholarly career would often allow a wider and open or interdisciplinary perspective in dealing with the philosophies and interpretive frameworks (Gardner, 2011). I may further be on the LLM/SJD and PhD (International relations and diplomacy) studies with an illustration concerned of program evaluation and consulting of prospective students. In this case, two theories can be referred as most adequate lens to research, which of course works as a basis of evaluation and consultation (Bogdan & Biklen, 1992; Mertens, 2009; Phillips & Burbules, 2000). One article is very pertinent with my scheme that I summarized the perspective of author to guide on the work of my illustration. As I said, I also remind that these two qualities -- evaluation and consultation -- are intrinsic for their work, i.e., teaching and researching (Kincheloe, 1991).
First, the transformative framework seems to inform his research that he charted vast data over period and was enthusiastic to argue on the inseparable relationships between the psychotherapeutic research and practice. This perspective often focuses on the marginalized individuals or groups rather than imposing structural laws and theories. Although the clinicians may not be such discrete group within the circle of clinical psychology, it was taken to be challenged by the author from the mainstream of knowledge since their belief, value and knowledge are highly individualized or particular. In the introduction, it seems conceded, "as a practitioner myself, it occurred to me that perhaps one of the reasons psychotherapy research is often ambiguous and inconclusive is that it was trying to model itself on the quantitative investigatory paradigms of the physical sciences (1996)." In this framework, the basic tenet is that knowledge is not neutral and it reflects the power and social relationships within society. This lens seems to highly influence the author’s attitude through the article standing between the subjective knowledge on therapeutic practice and quantification-oriented general knowledge from the therapeutic research. For example, he introduced his methodology, "The use of my own personal and professional experience as client, teacher and supervisor of psychotherapists as locus of exploration… (1996)."
Second, the author partly employed the post-modernist frame as explicit in his statement, "to explore a postmodern qualitative research methodology, context and content which was grounded in a moral universe where issues of values, ethics …. (Clarkson, 1996)." According to Thomas, the postmodernist are "armchair radicals" who focus on changing ways of thinking than calling for action based on these changes (Creswell, 2013). This can make as distinct from the transformative framework where the latter goes far enough in advocating action to help individuals. This aspect of frame is fairly penetrating through the article, but eclectic by relating the effect and utility through the theory, supervision and practice. This is so even while he placed the heart of study with the felicitous phrase, the therapeutic relationships—the focus for the case under investigation in the instance. He also seems to be influenced from the post- modern way of thinking, as we read in the Discourse analysis, "First, there was the thorough exploration of the diversity of meaning, the different contradictory ways of speaking that govern what we do (Clarkson, 1996)." He also was expressly iterative of his position, by commenting, "I would submit that this study has not only been post-modern in the diversity and particularities of its components drawing from a multiplicity….." His main suggestion on the enduring and reinforced ties between the research and practice had been stressed in a sense of diversity within the universe. In his belief, the client is a major source of new or confirmable knowledge "Learning with the client in such a way introduces a praxis of the recovery of knowledge which is surely at the very heart of the therapeutic endeavor itself (1996)."
7. Evaluation or Consultation and Research Methodology
We generally, however, do not include all of those rating or evaluation activities in the strict sense of evaluation. Evaluation, in a meaningful term, needs to be systematic in the least, but often is treated as scientifically by using a criteria governed by a set of standards, hence, closely entwined with the three methods, i.e., quantitative, qualitative, and mixed methods (Creswell, J.W., 2009). Therefore, when we talk about evaluation, its quality tends to be scientific and generally exhaustive about the subject’s merit, and the aim, objectives, results of program are considered to ascertain and assess. It could help a decision making, enable reflection, and identify a future change (Laureate Education, Inc., 2008). In practice, we can see two forms of evaluation which are formative and assumptive. The formative evaluation precedes the programs, events or organizations to develop the concept or proposal. The assumptive one primarily takes place to draw lessons upon the completion of project or implementation of programs. What, then, is the main purpose of an evaluation or program evaluation? As Marthe said, the purpose can be defined in view of the systemic process to "determine the quality of a program by formulating a judgment" (Hurteau, Houle & Mongiat, 2009; Patton, 1980).). The essences of evaluation in its definition would be (i) structured interpretation, (ii) giving of meaning, (iii) comparison with the original objectives, and (iv) understanding of what and how. A more fine definition may further include these; (i) systematic, rigorous, and meticulous application of scientific methods, (ii) resource-intensive process (such as, evaluate expertise, labor, time, and a sizable budget) (iii) critical assessment and objective manner (iv) attainment of objective knowledge (v) scientific or quantitative measuring (vi) objects merit and worth and assistance of audience (evaluand: lient).
For the rising attention to the field of evaluation, we can see a tremendous progress of theoretical and methodological developments during the last three decades (Babchuk, 2011; Reynold, 2007). For example, the role of the Joint Committee on Standards for Educational Evaluation and the American Evaluation Association is notable. A set of Guiding Principles for evaluators developed by the latter elicited several of important elements to be respected by the evaluation researchers (i) systematic inquiry: evaluators conduct systematic, data-based inquiries about whatever is being evaluated. (ii) competence, (iii) integrity/honesty. In this guideline, the evaluation research is required of quality data collection, defensible indicators, and eventually the credibility to findings. It is a duty for the evaluators to provide competence performance to the interested parties. All these elements described have a bearing to be interconnected with the aspect of three methods.
8. On the Professional Competence Through the Process
Although we said the three methods are viewed to fit or be required within the field of evaluation, this does not necessarily mean that any method will yield a best result for the evaluators. As we had learned through the course, each method has the strengths and weaknesses as submissive to the professional way of suitable selection (Creswell, J.W., 2009). The experimental approach, for example, can be best to disclose the causal relationships of event or program evaluation. The quantitative study may be matched with the management information system and can serve more effectively for the dynamic operations of complex programs. The content analysis may be undertaken as related with the qualitative study, and provides an evaluation if to include a judgment. The mixed method can be employed, in any most effect, to provide a "model building" for the programs evaluated. That is partly because the mixed method often begins with some predisposed stance of researchers. Consumer-oriented studies generally would be conducted on the basis of quantitative method, which can be seen "objectivists, mass and true evaluation." The objectivists and elite perception in program or organizational evaluation can be viewed as quasi-evaluation since it tends to entail a high share of data itself or knowledge other than a value or assessment. The qualitative or mixed methods probably can be connected with the subjectivist’s perception. It also is classed into the elite and mass aspects like the objectivists. The subjectivists and elite perception provides a true evaluation, which is typically represented in the certification and accreditation process. The connoisseur studies would be one branch to yield a more nuanced and refined findings to address the client’s needs, which usually may be seen in the qualitative or mixed undertaking of evaluation research. The adversary approach is interesting to unearth the truths, which would impliedly be embedded on all the three methods (Maxwell, 2005; Mills, 1959). In the quantitative, this way of thinking could enable a current status of theories or tenets, and facilitates the understanding of vast literature. This point would also be true with the mixed method when they consider a grand theory or frame of the themes or propositions. The adversary approach would be a little less minded if the natural settings are primary to begin with the qualitative method. However, the coding work or documents review may require this basic of mindset. The adversary approach, as occasioned in a mock of legal proceedings, represents the dialectic exchange of ideas to inner-subsidize the three methods. It, nonetheless, independently provides a subjectivist, mass, true evaluation by exposing the two opposing positions. One illustration involved with the program evaluation and consultation was presented herein forth.
9. Problem Statement
In a variety context of public institution, the program evaluation is practiced. For example, the famous magazine of The National Jurist in the US legal education would produce the useful information for the legal education besides the US News and Report or those of global rating institutions. A rating for the best public service law schools, practical training program or clinical learning program would be the kind of examples. Some concerned lawyers or experts may individually rate the program, for example, the ranking of LL.M program on the basis of recruitment statistics for the major law firms upon graduation. The evaluation expertise seems rapidly be made abundant over various sectors and interests which explains for our contemporary public lives. In this trend, the research degrees in law (LLM or SJD and MA/PHD in Law) other than JD had not been specifically addressed, which I enchanted to exemplify (Stringer, 1993). That is also the case, for example, about my research doctorate in the "international relations and diplomacy" although the adjacent area, such as "political science" or "international studies"-- perhaps massively language or history and oriented of each nation and in coverage of the whole of three level of degrees – may appear in the NRC or QS ranking. In this concern, many experts would stress on the importance of consulting process as the QS graduate guide suggested. The illustration now onward has been prepared to give a formula for the consulting process and one ranking source for the programs given no perfect ranking source is available or inadequate as a matter of the degree's trait. It will likely be the kind of rankings on the business doctorate of Financial Times along with the traditional MBA-focused business school rankings. In use of the ranking within this illustration, we may situate the consulting students for his years relevant with the base year of 2007. Since the quality of information is longitudinal, we can suppose if 1993 through 2014 graduates with the degree of research master or doctorate in law and PhD in the international relations and diplomacy can be covered. This kind of temporal factor in the evaluation setting can be applied in this way for various events of evaluation project. The issue of evaluation and consulting subject is related with the kinds of discipline, such as the program evaluation, education, sociology, psychology, legal education, and therapeutic studies.
10. A Mixed Research and Consultation with the Forms of Use
Two students, described B type, asked to seek a guide for their further studies in above two research programs (Hatch, 2002). I made an initial contact, and audit their gist of referrals, which appears to be very concerned on their part. I thought that the research was necessary, and the core issues had been summarized as in the problem statement. One week research seems deemed that I scheduled for the day of three months later to give the final result of investigation and outcome of evaluation. As a focus of evaluation, I considered several important themes that most pertains to the problem and solution. First, they are exploring the study opportunity as a research student, not a college or law schools generally bending on the education of JD students. Second, they would be flexible in their final selection decision between the popular law school or LLM rankings and research-oriented ranking. These two basic qualities of evaluation lead to many subtlety of considerations about the factors of evaluation. The usual rankings, for instance, are massively based on the academic credentials of admitted student, such as GPAs or scores of law school admission test, which is not relevant with the research programs. In the case of B-2, such data are even unavailable or less immediate given that his plan is suited with the study abroad. The challenge also arises in the B-1 since the other rating, such as QS is massively faculty oriented, although the student is much excited with the performance of alumni trained from same degree courses. You also may consider my note on the transformative framework or post-modernism with this aspect of challenge. Now we live on the post-modern context of global community with the rapid rate of technological advancement and new mode of communication, and the individualization or vulnerability of ego seems starker (Barritt, 1986; Bloland, 1995; Bogdan & Biklen, 1992). Most importantly, with the increasing exchange of students in the global context, the foreign educated graduate will subsist as even looks like the people of Diaspora across the global jurisdictions. In their life path perhaps not easy on the long way, the degrees achieved in the foreign countries or in the graduate and research programs are the kind of source of enjoyment or even life-time meaning in reflection (Husserl, 1931; Hunt & Colander, 2015; Spiegelberg, 1982). This type of personal development, often within the foreign research students as said, needs to be cast on the framing of evaluation with the factors assigned with different values. They had a focus most sensitively with the degrees they expect to obtain relatively very higher than other side of considerations, such as labor market and settlement in that country (Colaizzi, 1978). This seems a typical phenomenology involved with this kind of cultural group. The reflection on this point leads me to yield a sharp focus on the degree-based impact ranking as considerably of higher value than other factors. Also the framing and assignment of value has relevance with the quality of research programs. For example, 3.8 GPA students may perform well in the taught based program, but is not always true if the creative work on research is a trait in the programs (Fay, 1987). Of course, this is also because the comparison of undergraduate achievement generally is not practiced only with a rare exception, such as NRC rankings from the US source. The small nature of class would be irregular in view of student population and yield a less meaningful consequence as distinct from the law school or national business school rankings. The data collection was performed based on the examination of documents and records, in which the existing data from the sources are utilized. As the degree-based research impact ranking is rarely compiled that exhaust me too much work for independent investigation that I decided to exploit the data compiled by Shapiro from the Yale law school. Other sources are plentiful and easy access was made, for example, peer review result of law schools in the USNWR or QS research quality of faculty and many others (Barbour, 2000; Lather, 1993). On the process, I was impressed that the degree-based research impact ranking seems most direct and immediate to my case beyond the ratings of other factors, say, one reason to assign a high value for the factor. The data collection and analysis as well as preparation of forms devoted to the practice of consulting on this issue were finally made ready on the sixth day. The compilation of data on the productivity and citations from Shapiro’s, for example, was conducted with the aid of my assistant for 20 minutes of exhaustive search about the background of scholars within the top 100 all time list. In order to ensure the accuracy of data, all the ways possible were used. In the stage ahead, the journal writings and reference to the memos of stakeholders were analyzed, and the consulting day was full to share much time of exchanging opinions and views with the students. Through the process, the rigor and rapport are a crucial touchstone for the credibility and trustworthy of qualitative or mixed method that the "best possible way" standard and "nothing to be left unlearned" often held as a principle of qualitative method always guided my research and evaluation. Also very importantly, the purposive sampling or bracketing of research through the data collection and analysis had been minded and held importantly that is the kind of major attribute of qualitative or mixed inquiry and evaluation as Patton guided. This aspect is reflexive with the same eventual destination as you see in the Exhibits between the degree-based research impact ranking and pro-choice one. Some sources of data I utilized through the process, one journal writing in my previous travel – hence research is also a participant in this data – findings of degree- based research rankings -- were shown as Exhibits on the back of this article and the forms for the future use also was attached (Connelly & Clandinin, 1990; Strauss, 1987; Neyman, 2011).
Exhibit I. A Piece of Journal Writing
A meditation July 2013
Having a busy schedule last week in Houston, Texas, I am now enjoying a slight margin with my family in Los Angeles. In the afternoon, we plan to visit the Hollywood, and I am thought to spend true holidays for a few days. When I return home this Saturday July 6th, my time schedule is full of publication contract, preparation of exam questions for the national bar examiners and other backlogs that it probably seems hard to enjoy this summer vacation. The United States seems to have become quiet and of serenity around the upcoming Independence Day because it actually is a few days holiday for the Americans. Perhaps because I stay here in the US, it seems natural to be reminiscent of students and peer professors who wrestled with the west law, and books and articles in law for their research and preparation for the bar exam and various national civil service examinations. Actually I am seemed that we, the law professors, have to be responsible for the future of legal culture and system. Although we are part of a stately Republic of Korea, we may be evoked if we are any kind of minority in the world that we think the superpowers in the world, including the US, China and Japan, As I stay in the US for two weeks in this vacation, I likely come to sense that the minority issue is not the story of others, but also of a great significance to our nationals. We are dignified and proud to possess the national territory and independence, and the stay in this foreign country reminds me of the preciousness of home.
We, Korean legal system, had come under a lot of German influence since we were acculturated with the legal implantation of German laws through the rule of imperial Japan. Therefore, in selection of graduate school, we have preferred Germany and the students, who wish to study abroad, often were destined at the country. However, it has been trending that the recent students also prefer to study law in the United States much in share, and virtually all of law libraries in Korean law schools have a subscription to the West Law or other paid basis of on-line provision of law. As I stay in the United States, it is also natural to reflect the days of graduate studies in the United States around the mid of 1990’s and with some impulse to remark the minority issues. That is to bring up this piece of journal writing in my old bulletin board of personal webpage available with the Chosun Law.
The law professors in the United States mostly are JD degree holders. However, the foreign students interested in studying abroad and ultimately wanting to become the law school professors and researchers generally enter the LLM or SJD degree programs. The JD program is taught-based for the three-year course, and LLM program often seminar based for scholarly experience with some depth of specialization on law and advanced concept. In some cases, it would be research-based as in the LLM program of UW-madison law school. The SJD program had been available with the 20-30 among more than 200 law schools in the United States, and the number has slightly increased over the decades. It is, of course, research-based that we class one type of research doctorate in the educational awareness. The United States is a typical country with the spirit of minority protection, Protestantism, the virtue of frugality and fidelity, which is admirable and can be helped to learn. Notwithstanding my alumni status, the graduate studies in the Wisconsin Law School seems to show the context of their public spirit and philosophy a whole. The campus news told that the Hastie fellowship program, one of LLM course, had a memorial reunion to celebrate the 40 years of anniversary in worship and enjoyment. I did not attend, but in an effort spanning 40 years by professor E. Jones, the program was known to produce the largest number of minority law professors with the advanced degree of law among the law schools in US. Another recent study, such influential work by Shapiro, Yale professor of law and citation studies, about 2012 ranking on most cited law review articles shows that the two articles made a top 100 all- time list by the alumnus of Hastie fellowship with an LLM degree. It is amazing given it may well be comparable with the Nobel Prize for the professors of law in the world. While most of 100 articles were authored by the JD degree holders of Harvard and Yale and other prestigious law schools, it is a significant achievement that only could be feasible with the kind of respectable American spirit of E. Jones (The Shapiro’s research impact studies in law are similar to the general citation studies in basic quality, such as Leiden ranking or others, but differ and are interesting to show the ranking of degree production institutions. It is one of influential and authoritative studies that provide the landscape of legal research in the United States and its trend). Since I usually hold a common interest to legal scholars, I made the time to investigate the fare of LLM and SJD degree holders in the top 100 list on the basis of Shapiro’s studies, and the outcome turned out that they are truly the minority in that share as compared to the JD degree holders. That's because the LLM / SJD degree holders are those of minority in the US law schools. Provided that the number of LLM / SJD students is small, the result should be viewed that could not be made any reality without the deep concern and over 40 years of undisclosed effort by professor E. Jones. I also consider it relevant that the entering into the job market of legal teaching by LLM degree holders in the United States and their scholarship is a very good sign to protect the rights and status of minorities. If we plan to rank the quality of research doctorate in law, then the criteria and context are considered to produce important elements as a factor (For example, the business PhD ranking compiled by the Financial Times, unlike the conventional MBA Ranking, is based on the "post-doctoral degree entrance into the teaching position of business schools, i.e., the number of recruitment as a business professor/number of PhD graduates).
Studying within the graduate program of law in the US law schools is learning the American pragmatism and the Protestant spirit. If you want to learn the spirit to protect the minority, perhaps the ideals of law ultimately, and pragmatism and Protestantism of Americans it is good alternative to study law in the US. Often the applicants consider the rankings of law school as a single variable if they are thinking to study in the US law schools as a single variable (the usual ranking source of US law schools are JD-oriented, or entirely in coverage of whole three degrees and faculty. Of course, some ranking source is specialized in LLM. Nonetheless, the research doctorate in law has not been treated independently even in 2010 NRC rankings, buy only with some statistical data. That is perhaps because the program is small and oriented to the minority education). Of course, a focus in the selection of SJD program can be different depending on the context and preference of individual, but if you think of the studies in the research degree in law, the implications that is shown by the Shapiro’s citation studies in 2010 and 2012 are thought very significant to select the programs that you wish to study.
What is good and advisable is to select the programs appropriate to their own beyond the consideration of usual law school rankings. For example, the UW-madison law school manages the east Asian legal studies center, and the university has the US or possibly world top record of 78 programs registered as ranked in the 2010 NRC studies on the assessment of research doctorate programs. Also the LLM program is research based as said. Given the interdisciplinary studies of law are stressed to quality legal research, this backdrop is one important aspect that the consulted students and parents share and be informed adequately.
Tomorrow, I am going to visit the Hollywood with the family, and have the time of enjoyment for repose and reinvigoration. Early in the morning today, I was sudden to recall on the peer professors, who are to be connected to the west law portal and my dear law students, junior researchers now in the graduate programs of law and prospective students for those programs. What do we think between the minorities and law? This is perhaps the eternal question that the inquirers of law are to be challenged constantly.
Exhibit II. Degree-Based Research Impact Ranking of LLM (MA in Law) and SJD (PhD in Law)
~ LLM/SJD or MA/Ph.D in law: Research Impact Ranking Based on the Degree Indication * Not Faculty Based * and Expanded to Cover the World Universities.
* The data are based on the two 2000 and one 2012 studies of Shapiro from Yale Law School, "Most cited legal books (2000), authors (2000), legal articles (2012): All Time List.
* The data can be partially produced on the raw basis from the HeinOnline or other websites (For example, it is notable that one scholar with a doctoral degree in law from the German university and now in teaching position for the US law school made a list as one of top cited authors in the HeinOnline/And Prof. Coffee from NYU would be stark in this method). But Shapiro's method is distinct to keep on integrity and consistence to measure.
* Total Number of LLM/SJD: Compared in the median year of three articles: based on 2007 LLM/SJD entering class of the law schools (One LL.D degree researcher from Edinburg was excluded since it is not-training based).
* If in conflict of rank order among two factors, per capita citation had been considered in priority.
* The category "Additionally" "Younger" "Very Younger" was excluded in consideration of fairness, consistency, and integrity.
Exhibit III. Degree-Based Research Impact Ranking of PhD in the Humanities and Social Science
|Frequencies of Author||Total citations|
|1. Academie de Paris (France)||6||1874 + 2521 + 2465 + 897 + 662 + @|
|2. Harvard (USA)||3||694 + 596 + 519 + @|
|3. Cambridge (UK)||2||1303 + 723 + @|
|4. Freiburg (Germany)||2||874 + 566 + @|
|5. Iowa (USA)||1||1536 + @|
|6. Chicago U. (USA)||1||1066 + @|
|7. Goethe U. (Germany)||1||1049 + @|
|8. Berlin U. (Germany)||1||971 + @|
|9. Yale (USA)||1||960 + @|
|10. Vienna (Austria)||1||903 + @|
|11. Konigsburg (Germany)||1||882 + @|
|12. U. Penn (USA)||1||812 + @|
|13. U. Munich (Germany)||1||733 + @|
|14. U. Neuchâtel (Swiss)||1||725 + @|
|15. Princeton (USA)||1||708 + @|
|16.Groningen U. (Holland)||1||700 + @|
|17. Heidelberg (Germany)||1||593 + @|
|18. U. Bern (Swiss)||1||583 + @|
|19. Columbia (USA)||1||577 + @|
|20. MIT (USA)||1||577 + @|
|21. Johns Hopkins (USA)||1||575 + @|
|22. Cornell U. (USA)||1||573 + @|
|23. Yena U. (Germany)||1||566 + @|
~ MA/Ph.d (Humanity and Social Science): Research Impact Ranking Based on the Degree Indication * Not Faculty Based* and From the 2007 Citation Information from Thomson Reuter.
* Barthes, Tajefel, Wittggenstein, and Niezschete are hard to confirm and thus unclear if they graduated with a master or doctorate. Barthes had undertaken as the research officer in the CNRS over the long period time, but did not obtain the graduate degree. Tajefel is known to obtain the bachelor degree from the Birbeck college, London university, and his career can only be made clear that he taught the social psychology at the University of Cambridge for the long term. Wittgenstein is just as well that he studied in the Yena University of Germany and Cambridge University in England, who later taught at that university. It is unclear if he is a holder of master or doctorate degree. Nietzsche also seems to have not acquired a master or doctorate, but merely known to study at the University of Leipzig.
* If the number of author is equally among the institutions, the ranking is discriminated on the basis of citation. " + @" indicate the annual amount of citations added, thus, uncertain but on some steady rate of increase, as assumed that it would increase every year at a constant rate (In the case of law review articles or books, the citation tends to increase at more than constant rate than other context of disciplines. In the case of the humanities and social sciences, the annual trend of citation increase is less predictable, but seems to be increasing each year with a significant correlative).
Exhibit IV. Pro-choice Ranking for the B-1 Student
|Institution||(Summary of Consultation) "As seen beneath"|
|1. Wisconsin (Madison)|
|4. U. of Chicago|
|7. Nation-Based Rankings will be sectioned here: Stanford/NYU/Virginia/Duke/Michigan/Berkeley/ Georgetown/Cornell/Northwestern/Vanderbilt and so on (The order of rank will be marshaled on the basis of USNWR law school rankings because of no meaningful data in this scheme of rating. In the cases of other countries as pertain to 8 and 9 below, the national or regional rankings of QS data may be utilized in identifying their specific ranking).|
|8. Non-English Speaking Universities in the West|
|9. Asian Universities|
(Summary of Consultation) B-1 student finished the LL.M course, now is considering to attend PhD studies in law or SJD. Because he focused on the research impact on the degree-based citation indicators than the faculty members, the above Research Impact Ranking (it is related with the rankings of LLM or MA in law and SJD or PhD in law as combined – hence graduate programs in law -- and is considered as any most proximate data in considering the quality of research doctorate program in law. It is because the pattern and structure of legal academia are close to be interwoven with both degrees besides the major workforce of JD degree holders ) can have a share of 55 % as a factor and 15 % from the measure of faculty members based on the USNWR or QS rankings of law school and law subject. The remaining share can be composed of overall research capabilities of university such as NSF and the overall reputation of law schools (30% for their share to explain for the final ranking), which eventually yields the pro-choice ranking of consulted student. In this process, the attribute of research degree in law is contingent and volatile that the range or scale of distribution to be assigned with the score can be classed possibly at considerable margin (for example, the overall reputation of law schools may assign a value with one point in discrimination, such as 10 and 9, for the rankings 30 or 50 of USNW in margin; that could be wider in the case of QS considerations; the consulting process can be done with either option). This concept is relevant with the intrinsic of studies of the research degree in law program and the GPAs or undergraduate academic credentials, often importantly referred to the law school admission and taught-based instruction, could significantly turn as less a factor for the new mode of scholarship on the research-based work. On the other hand, the citation and productivity of unit indicators in the above RIR can be estimated in a range more closely.
Exhibit V. Pro-choice Ranking for the B-2 Student
|Institution||(Summary of Consultation) For the B-2 student, although the idealistic road of international politics is important, he prefers to increase his viewpoint of realist international politics, such as the essence of the state power, as well as the diplomatic importance of such realist international politics. It is to be studied based on philosophy and in order to deepen his undergraduate studies dealing with the international relations and foreign affairs -- hence, his wish was for the course titled "PhD in international relations and diplomacy." As the degree name implies, the philosophy is elementary to gear up with the research doctoral studies and was encouraged to think of the importance of interdisciplinary research. B-2 student also put an emphasis on the citations of degree holder more than faculty members in exploring the selection of programs. So the 2007 statistics of Thomson Reuter was the basis of consultation, which comes as proportioned in 55% of share. Other recent criteria, such as most cited scholars of articles in the SSCI comes into consideration as a factor with the assigned share, 15% and other 15 % may stem from the subject ranking as most proximate with his degree name. The latter two ratios are less than the former since the research impact from the book authors in the humanities and social sciences is more significant, and the ranking of international relations and diplomacy is not directly related to the subject ones ("international studies" - language and history oriented -- or "political science"). The subject ranking also needs to be considered that it is not focused on a research PhD, which, however, will be common to bachelor, master and PhD. Furthermore, since the United States and the European perspective of the diplomatic analysis tends to expose the different lens and frames of understanding so that the student was advised to think about the country of study in the first. The various factors in this kind explained for the 15% of parameters, and the final outcome for the Pro-choice ranking of B-2 student was yielded at left column.|
|1. Academie de Paris (France)|
|2. Harvard (USA)|
|3. Cambridge (UK)|
|4. Freiburg (Germany)|
|5. Iowa (USA)|
|6. Chicago U. (USA)|
|7. Goethe U. (Germany)|
|8. Berlin U. (Germany)|
|9. Yale (USA)|
|10. Vienna (Austria)|
|11. Konigsburg (Germany)|
|12. U. Penn (USA)|
|13. U. Munich (Germany)|
|14. U. Neuchâtel (Swiss)|
|15. Princeton (USA)|
|16.Groningen U. (Holland)|
|17. Heidelberg (Germany)|
|18. U. Bern (Swiss)|
|19. Columbia (USA)|
|20. MIT (USA)|
|21. Johns Hopkins (USA)|
|22. Cornell U. (USA)|
|23. Yena U. (Germany)|
Exhibit VI. Forms Usable for the Future Consultation
|Institution||Left Blank for the Consultants or Counselors|
|Institution||Left Blank for the Consultants or Counselors|
|Institution||Left Blank for the Consultants or Counselors|
|Institution||Left Blank for the Consultants or Counselors|
|Institution||Left Blank for the Consultants or Counselors|
* Forms of Other Category of Stakeholders
|Institution||Left Blank for the Consultants or Counselors|