Research Method Evaluation

Assessment 1

Option A: A critical review of research methods in context ..................... By Nusrat Naz ID # 1158097 .....................

Over View of Research Topic

In TESOL research, a variety of research methods are used. Being a TESOL practitioners and researchers we need to evaluate the research methods to check the suitability and validity in relation to a particular research topic. This evaluation of the methods involves critical analysis in all their respective aspects including, sampling techniques, data collection tools, coding and analysis of the data, and advantages and disadvantages of the selected methods in the context.

Many issues in language learning and pedagogy have great potential In TESOL research. Code-switching (Alternative use of L2 and L1) is one of the very crucial issues in multilingual and bilingual EFL settings. Many researches have been carried out on the topic so far. Zentella (1981) defined code-switching as “ bilinguals’ ability to alternate between the languages in their linguistic repertoires”. Code-switching is considered as a popular practice because it enables learners to harness their main language as a learning resource (Milroy and Muysken 1995).

My personal affiliation with the topic is based on my own background of learning English in the same EFL setting, where code-switching in teaching English has been far too common, because almost all English language teachers in Pakistan are non-native speakers of English, and they also share their first language with their learners.

Many researchers have investigated the topic from multiple perspectives using different research methods. For evaluation of the research methods in context I chose three research articles on more or less similar topics on code-switching that I will analyse in the following paragraphs one by one.

1. Issues of Language(s) Choice and Use: A Pakistani Perspective

A Qualitative Research

This research was conducted by Malik Ajmal Gulzar and Samina Amin Qadir and published in Pakistan Journal of Social Sciences (PJSS, Vol. 30, No. 2) in December 2010. The research population consisted of seven expert PHD scholars from different areas of Pakistan. The purpose of the study was to observe the teachers’ views regarding the code-switching in the bilingual classroom in Pakistani context focusing on the aspects like;

  • The role of L1 in improving teacher’s performance and as an aid for learners
  • Issues involved in bilingual classroom regarding choice and use of the language (Gulzar, Qadir 2010).

This research was conducted through qualitative research methods, which is based on individuals’ personal views within a selected situation according to their feelings and experiences. As Schwandt (2000) defines:-

Qualitative inquiry is a 'home' for a wide variety of scholars who often are seriously at odds with one another but who share a general rejection of the blend of scientism, foundationalist, epistemology, instrumental reasoning, and the philosophical anthropology of disengagement that has marked 'mainstream' social science (p. I90).

In this research, interviews were used as a tool for the collection of qualitative data from the participants. Overall ten (20 to 25 minutes long) interviews, consisting open ended questions were conducted from the participants, over a month. The advantage of gathering data through interviews is as Zoltan Dornyei (2007) states, “multiple realities exist which are not observable and can only be assessed through talking with the participants”. As interpretation of the data is subjective in nature, it is naturally influenced by the researcher’s own views as well. Huberman (I994: 7) states; “The researcher is essentially the main "measurement device" in the study'. Accordingly, in qualitative research, the researcher's own values, personal history, and 'position' on characteristics such as gender, culture, class, and age become integral part of the inquiry”. However in qualitative research descriptive data based on participants’ views indicates the equal importance of both the researcher and the researched (Haverkamp 2005). Semi-structured type of interviews selected in the study allows space for modification in interview questions and their arrangement to get a better insight into the participants thought, and also suits to the research objectives in the study.

The qualitative data collected in this research was non-numerical data as opposed to the quantitative research methods, which was then, analyzed and interpreted through non statistical method. For analysis, questions in the interviews and their answers were carefully read. The recordings of the interviews were transcribed, coded and categorized for the production of the themes and, finally for drawing conclusions (Gulzar, Qadir 2010). The research concluded the following points;

  • Undecided use of language is because of unspecified educational policies regarding to medium of instruction
  • Resolute use of code-switching can facilitate both teaching and learning
  • Considering the use of code-switching as a compulsory technique in EFL classroom was not agreed
  • Sufficient use of English was recommended as a technique to evade CS in EFL context

One of the overt advantages of the employed research method in the study is; the sample population for the research was selected from different regions of Pakistan to create variety of regional contexts within one national context in order to maintain diversity. According to Martin and Jones (1995:7 as cited in Dörnyei 2007) social context of different classrooms plays a vital role in shaping code-switching in bilingual discourse.

In general qualitative research deals with generating an average statement for a larger number of people, and generalizing the views based on personal ideas of the different persons, that Dell and Hymes (1994) considered as; “undesirable reduction process because in QUAL terms the real meaning lies with individual cases who make up our world”. But in the discussed study, as findings show a uniform view regarding the use of language, based on expert views could not be established, because most of the participants were not determined about use of CS in the classroom.

At the same time this study being qualitative research used limited number of participants, limited data and contexts, unlike quantitative method. So generalizing the views of limited participants for a very large group of people may be far from reality.

2. Linguistic Features of Code-Switching:

A Study of Urdu/English Bilingual Teachers’ Classroom Interactions

A qualitative Research

This second research was carried out by Liaqat Iqbal, on the lectures delivered by both male and female lecturers, at the postgraduate levels in six recognised local universities in Lahore, a city in Pakistan. This study aimed to check the ratio of code switching from English to Urdu in the teachers’ language and its different linguistic features. This research was published in the International Journal of Humanities and Social Science (Vol. 1 No. 14) in October 2011.

In this study using the qualitative research method of discourse analysis, its authors tried to find the ratio of code-switching in teachers’ language as well as its different features like intra-sentential code-switching, inter-sentential code-switching and code-switching at word, phrase and clause level (Iqbal 2011). This Qualitative research is carried out empirically as it was done in a natural context without trying to control the context. For the detailed study of the natural context, data is collected through longitudinal and deep contact with the real research setting (Haverkamp 2005).

For the collection of data in this study, initially sixteen HEC recognised universities were selected which were later further short listed to only six universities through purposive sampling. For the detailed study on the research topic empirical data was collected through observation and audio recording of the lectures delivered at postgraduate level in the selected universities. This data included two to three lectures from each university and altogether fourteen lectures were recorded and observed. The purpose of recording and observing classroom lectures was to analyse the corpus critically for the use of code-switching and its structural and linguistic features in its social context of bilingual society through descriptive study. Advantage of the recoding in the method is highlighted by Eisner (1991), that “note–taking and audio–taping are crucial tools in conducting any qualitative research because they provide the researcher with reminders, quotations, and details for both descriptions and interpretations”.

As far as data analysis is concerned it should be noted that this research aimed at observing only the linguistic feature of code-switching and not any other issues related to its causes, objectives and functionality, therefore, long statements were avoided. Audio recording of the lectures were transcribed with the main focus on the examples of code-switching in lecturers’ corpus for analysis purpose.

One of the major advantages of the qualitative method, which is also evident in this study, is the data consisted of the naturally occurring samples of language used in context. Linguistic features of teacher’s discourse and degree of alternating between languages can only be observed in naturalistic and controlled observation (Reeves 2011). For some reasons this study was not done over the long time to get deep insight into the context for analysis of code switching. In other words, limited data collection is a disadvantage in the method which was practised in this research, as data was collected from few participants from few institutions only. Therefore it is hard to generalise its findings for other varieties of contexts.

3. The Iranian EFL Students’ and Teachers’ Perception of Using Persian in General English Classes

A case study-Mixed method (Quantitative-Qualitative) Research

This final study was conducted by Seyed Mostafa Hashemi at the University of Guilan, Rasht, Iran to examine the concept of Iranian EFL teachers and learners regarding the use of Persian in English classroom in higher education (Hashemi 2013). This research was published in the International Journal of Applied Linguistics & English Literature in March 2013.

This research is basically a case study. Case studies deal with understanding and analysing specific features or issues related to persons, or a group of people in language learning and pedagogy. An advantage of this method is that, “it enables researchers to carry out in-depth studies of an issue in its natural environment and it is one of its strengths that it observes effects in real contexts while explaining the complexities involved in real situation”(Cohen, Manion and Morrison 2005), which may not be observed through survey or experimental research. The discussed study mainly focused on investigating both teachers’ and students’ understanding about the use of L1 and L2 classroom at university level.

Mixed methods involving a combination of qualitative and quantitative research were used to carry out this study. A major advantage of using mixed methods is that “quantitative researchers follow a 'meaning in the general' strategy, whereas qualitative researchers concentrate on an in-depth understanding of the 'meaning in the particular” (DORNYEI 2007). Therefore, the author got more balanced results as a result of using one research method and then in initiating and corroborating findings from the other method (Rossman and Wilson 1994).

The sample selected for this case study consisted on 10 English teachers selected out of 25 teachers at the University of Guilan, Rasht, Iran. Besides them, 345 students (195 female and 150 male, 18 to 35 years old) from General English classes were selected through clustering sampling from different faculties at the University of Guilan, Rasht, Iran (Hashemi 2013).

For gathering data a triangulation technique was used involving questionnaires and interviews as tools for quantitative and qualitative data collection. Both tools were used for getting deep insight into teachers’ and students’ perception regarding L1 use. Interviews were taken from teachers as “interviews can allow researchers to investigate phenomena that are not directly observable, such as learners' self-reported perceptions or attitudes (Mackey, A. & Gass, S., 2005). On the other hand students ideas were collected through questionnaire for collecting quantitative data which is more concerned with the common features of the large group as compared to individuals (Tellis1997). This questionnaire was developed by the teachers in the department of English at the same university as a localized tool. It included 26 items with 4-point likert scale for pilot study. Both teachers and students answered the same questionnaire (with little amendments for students and teachers) which was initially developed in English and later translated into Persian for students’ convenience and assed for validity.

Data analysis was done through transcription of the recorded interviews and classification of the qualitative data according to the relevance, and comparing the score of both groups in quantitative data through Mann-Whitney to indicate statistical difference of concepts between both groups. The whole information was combined together for the final interpretation through comparison and triangulation of data. The analysis of data was done with statistical software SPSS16 (Hashemi 2013). The accuracy of the analytical tools was checked at different levels which highlight the authenticity of research, though there was a risk of data mismanagement if it was not organized systematically (Zaidah 2007).

A case study normally develops a structure for analysis or utilises an existing structure for further deeper analysis of a specific issue. At university level only a few studies were carried out in the given context regarding the use of L1 especially combining both teachers’ and students’ views. This research was a further extension of the old studies, since it dealt with the topic in more detail for sorting out the salient feature of the issue, and revealed both teachers and students had different views regarding the use of L1 in FL classroom. Case study method is becoming very popular as it gives deep knowledge about a topic. On the other hand, the case study method can be a long and labour-intensive process to carry out the research which makes it a disadvantageous method in one way (Yin 1984). However, in this case study no the time spent on the study is not mentioned so that we have no idea that how long it took to complete the study.


The leading objective of discussed studies was to develop a good understanding about the fact of code-switching, its significance in bilingual classroom and impacts (both positive and negative) on teaching and learning a language. Over all these studies aimed to analyse what learners and teachers perceived about its effectiveness in learning and teaching a language.

Through critical review of the employed research methods in context, I tried to analyse all the significant aspects of method application involved in the chosen articles. This evaluation enabled me to identify the research methods, differentiating these methods from each other, their validity for the chosen research topic and their advantages and disadvantages within the context. Some of the methods used in the above researches were similar but not the same in terms of tools for gathering data and analysing the data. These methods included quantitative, qualitative and mixed method case study. Profound understanding of the research methods and their rightful application allows TESOL practitioners as researchers to single out the issues faced in language learning and teaching context, and offering their appropriate solutions to facilitate a better teaching and learning experience.


Brown, G. and G. Yule. (1983). Discourse Analysis. Cambridge: Cambridge University Press.

Cohen, L. Manion, and Morrison. 2005. Research method in Education. (5th ed.). London: Routledge.

Dörnyei, Z. (2007). Research methods in applied linguistics: Quantitative, qualitative, and mixed methodologies. Oxford University Press.

Griffee, D. T. (2012). An introduction to second language research methods: design and data. TESL-EJ Publications, Berkeley.

Gulzar, M. A., & Qadir, S. A. (2010). Issues of language (s) choice and use: A Pakistani perspective. Pakistan Journal of Social Sciences (PJSS), 30(2), 413-424.

Hashemi, S. M., & Sabet, M. K. (2013). The Iranian EFL students’ and teachers’ perception of using Persian in general English classes. International Journal of Applied Linguistics and English Literature, 2(2), 142-152.

Haverkamp, B. E. (2005). Ethical perspectives on qualitative research in applied psychology. Journal of Counseling Psychology, 52(2), 146.

Huberman, M. (1994). Research utilization: The state of the art. Knowledge and policy, 7(4), 13-33

Iqbal, L. (2011). Linguistic features of code-switching: A study of Urdu/English bilingual teachers’ classroom interactions. International Journal of Humanities and Social Science, 1(14), 188-194.

Mackey, A., & Gass, S. M. (2005). Second language research: Methodologyand design. Routledge

Milroy, L., & Muysken, P. (1995). One speaker, two languages: Cross-disciplinary perspectives on code-switching. Cambridge University Press.

Nunan, D. (2003). The impact of English as a global language on educational policies and practices in the Asia-Pacific region. TESOL Quarterly, 37, 589-614. doi:10.2307/3588214

Yin, R.K. (1984). Case Study Research: Design and Methods. Beverly Hills, Calif: Sage Publications.

Schwandt, T. A. (2000). Three epistemological stances for qualitative inquiry.Handbook of qualitative research, 2, 189-213.

Yin, R.K. (1984). Case Study Research: Design and Methods. Beverly Hills, Calif: Sage Publications

Zainal, Z. (2007). Case study as a research method. Jurnal Kemanusiaan, (9), 1

Zentella, A. C. (1981). Tá bien, you could answer me en cualquier idioma: Puerto

Zivkovic, J. (2012). Strengths and weaknesses of business research methodologies: Two disparate case studies. Business Studies Journal, 4(2), 91-99.


David Humffary wrote;

Hi Naz

I looked at your assignment and the content and summaries are good. It's an interesting topic and you have chosen 3 varied research studies that tackle it.

I found it really interesting reading yours and Carmen's assignments to get an insight into Non native English teachers experiences in being a teacher. These are issues which I don't get any exposure to in my teaching context.

In the third study I wonder if you should give a brief description of what the Mann-Whitney is to help a layperson understand and make your assignment more understandable.

Congratulations for finishing assignment 1.


Carmen Shiu wrote

Hi Naz

What a great idea by using a digital poster!

Code-switching in language teaching and learning is a very common practice in many TESOL context. It's an interesting topic !

Your analysis is thorough and organized. I particularly like the third one - case study analysis. I think it's clear and logical, full of information,sound like you put a lot of effort in it! well done....other than that I don't have much to say really!

however, there are some typo (as we all have) Overview, current (1st line)

keep up your good work!


Oliver Smith wrote;

Hi Naz

I just skimmed your work and found it interesting in regards of selected research topic. The evaluation of the research methods you have done is quite systematic and organised. By discussing two different studies involving similar research methods, differing from each other in terms of tools for collecting data and data analysis in your evaluation highlighted one of the significant aspects of research methods. You have put great effort in discussing importance of each research element.

Well done!


Muhammad Shabbir wrote;


This research review of different research methods in TESOL is very comprehensive and easy to understand. It discussed most popular research methods and techniques with practical examples in the form of selected research papers. Use of L1 in L2 classroom is interesting topic in English language teaching as it had been very popular in Grammar translation methods and then lost its importance with the advent of Direct method and communicative method. Now recently, by this review, I learnt that limited use of L1 has been proved as a useful technique in L2 classroom. Selected research papers in this review mainly used qualitative research method and I can easily understand that how interviews and observations have been used to conduct qualitative research. Similarly I found that this review discusses the quantitative research method very nicely with solid examples from the papers.

It is interesting part of review that it gives examples of mixed method and triangulation. I easily understood that how many researches used mix methods including both qualitative and quantitative methods of research in order to get more authentic and balanced results. Overall, I see that review is simple, direct and comprehensive as it describes major research methods in TESOL research with good examples.