64 - 109 An investigation of learning · Resumen En 2011, el Ministerio de ... ing practices, or socially-oriented, in boosting levels of harmony between inter-ethnic groups” (Marsh,

64 LACLIL / ISSN: 2011-6721 / Vol. 9 No. 1 January-June 2016 / doi:10.5294/laclil.2016.9.1.4 / 64-109

Received: 2015-08-05 / Sent for peer review: 2015-08-09 / Accepted by peers: 2015-10-12 / Approved: 2015-11-09

To reference this article in APA style / Para citar este artículo en APA / Para citar este artigo

Yang, W. (2016). An investigation of learning efficacy, management difficulties and improvements in tertiary CLIL (Content and Language Integrated Learning) programmes in Taiwan: A survey of stakeholder perspectives, LACLIL, 9(1), 64-109. doi:10.5294/laclil.2016.9.1.4

* National Kaohsiung University of Hospitality and Tourism, Kaohsiung, Taiwan. Corresponding author: [email protected]

An Investigation of Learning Efficacy, Management Difficulties and Improvements

in Tertiary CLIL (Content and Language Integrated Learning) Programmes in Taiwan:

A Survey of Stakeholder PerspectivesWenhsien YANG*

AbstractIn 2011, Taiwan’s Ministry of Education conducted a national-scale appraisal of 92 CLIL programmes. However, we lack an effective model for examining by preci-sely how much improvement in the quality of the CLIL programmes will rise as a consequence of the increased language proficiency and the acquisition of dis-ciplinary knowledge. To gain greater insight into the relationship between the execution and appraisal results of CLIL and the facilitation of content and foreign language acquisition, we researched the stakeholders’ perceptions of and attitu-des towards CLIL in order to create a reference for the national appraisal results. Our aim was to collect both quantitative and qualitative data on the programme managers, teachers and learners in 12 CLIL programmes nationwide by way of a questionnaire survey and interviews. We assessed these data to answer our main research questions regarding the efficacy and effectiveness of employing CLIL edu-cation in higher education in Taiwan. In total, 53 undergraduates and postgradua-te CLIL students completed a self-designed questionnaire survey, investigating their perceptions of and attitudes towards CLIL education. In addition, interviews with CLIL programme managers and student focus-groups were also conducted to further probe their opinions on CLIL. The findings mainly revealed that the lear-ners’ satisfaction with the CLIL approach is greatly affected by their level of lan-guage proficiency. Our findings can significantly advance our understanding of the current situation of CLIL education and the likely effects of changing the cu-rricula and directions of delivering content and foreign language courses at the tertiary level in Taiwan.

Keywords: CLIL evaluation; Taiwan; tertiary education; stakeholders’ perspective.

65LACLIL / ISSN: 2011-6721 / Vol. 9 No. 1 January-June 2016 / doi:10.5294/laclil.2016.9.1.4 / 64-109

YANG

Investigación de la eficacia en el aprendizaje, manejo de dificultades y mejoras en educación terciaria de los

programas en AICLE (aprendizaje integrado de contenidos y lenguas extranjeras) en Taiwán: encuesta sobre las

perspectivas de los agentes involucrados en la educación

ResumenEn 2011, el Ministerio de Educación de Taiwán realizó una evaluación a nivel na-cional a 92 programas de aprendizaje integrado de contenidos y lenguas extran-jeras (AICLE). Sin embargo, carecemos de un modelo efectivo para examinar con exactitud el nivel de avance en la calidad de los programas de AICLE que tenga como consecuencia el aumento en la proficiencia en la lengua y la adquisición de conocimiento en esta disciplina. Con el fin de entender mejor la relación entre la ejecución y los resultados de la evaluación de AICLE y la facilitación del conte-nido y la adquisición de una lengua extranjera, se han investigado las percepcio-nes de los agentes involucrados y sus actitudes con respecto a AICLE, con el fin de tener referencia para los resultados de una evaluación nacional. El propósito fue recoger datos cualitativos y cuantitativos de los directores de programas, profe-sores y aprendices de 12 programas de AICLE a nivel nacional por medio de una encuesta y entrevistas. Se analizaron estos datos para responder las preguntas de investigación en relación con la eficacia y eficiencia de la enseñanza por me-dio de educación en AICLE en la educación superior en Taiwán. En total, 53 estu-diantes de pregrado y postgrado en AICLE completaron una encuesta que indaga sobre sus percepciones y actitudes hacia la educación en AICLE. Adicionalmente, también se hicieron entrevistas a grupos de enfoque conformados por directores y estudiantes de programas AICLE para confirmar sus opiniones acerca de AICLE. Los resultados revelaron, principalmente, que la satisfacción de los aprendices con la metodología AICLE se ve afectada, sobre todo, por sus niveles de proficiencia en la lengua. Nuestros resultados pueden aportar, significativamente, en nuestro conocimiento de la situación actual de la enseñanza en CLIL y los posibles efec-tos en el cambio del currículo y las directrices para impartir cursos de contenido y lengua extranjera en la educación terciaria en Taiwán.

Palabras clave: evaluación de AICLE; Taiwán; educación terciaria; agentes invo-lucrados en la educación.


An Investigation of Learning Efficacy, Management Difficulties and Improvements in Tertiary CLIL...

Pesquisa sobre a eficácia na aprendizagem, manejo de dificuldades e melhoras na educação superior dos

programas de Aprendizagem Integrada de Conteúdos e de Língua em Taiwan: enquete sobre as perspectivas dos

agentes envolvidos na educação

ResumoEm 2011, o Ministério da Educação de Taiwan realizou uma avaliação nacional a 92 programas de Aprendizagem Integrada de Conteúdos e de Língua (AICL). No entanto, carecemos de um modelo efetivo para examinar com exatidão o nível de avanço na qualidade dos programas de AICL que terá em consequência do au-mento na proficiência na língua e da aquisição de conhecimento nessa discipli-na. Com o objetivo de entender melhor a relação entre a execução e os resultados da avaliação de AICL e a facilitação do conteúdo e da aquisição de uma língua es-trangeira, investigamos as percepções dos agentes envolvidos e suas atitudes a respeito da AICL a fim de ter referência para os resultados de uma avaliação na-cional. Nosso propósito foi coletar dados qualitativos e quantitativos dos diretores de programa, professores e aprendizes de 12 programas de AICL no âmbito nacio-nal por meio de uma enquete e entrevistas. Analisamos esses dados para respon-der a nossas perguntas de pesquisa com relação à eficácia e à eficiência do ensino baseado na AICL na educação superior em Taiwan. Em total, 53 estudantes de gra-duação e pós-graduação em AICL completaram uma enquete que questiona so-bre suas percepções e atitudes sobre a educação em AICL. Além disso, foram feitas também entrevistas a grupos de enfoque formados por diretores e estudantes de programas de AICL para confirmar suas opiniões sobre ela. Os resultados revela-ram principalmente que a satisfação dos aprendizes com a metodologia AICL se vê afetada principalmente por seus níveis de proficiência na língua. Nossos re-sultados podem avançar significativamente o conhecimento da situação atual do ensino em AICL e os possíveis efeitos na mudança do currículo bem como nas di-retrizes para ministrar cursos de conteúdo e de língua estrangeira na educação superior em Taiwan.

Palavras-chave: avaliação de AICL; Taiwan; educação superior; agentes envol-vidos na educação.


YANG

INTRODUCTION1

Content and Language Integrated Learning (CLIL) is a generic term, refer-ring to an innovative educational approach in which a subject is taught in an additional language (Lasagabaster & Sierra, 2010). CLIL is always du-al-focused on both content and language. Depending on the model used, its aims can be very subtle in terms of helping learners understand the point of learning a language and developing their ‘can-do’ attitude as language learners (Marsh, 2009). This purpose is crucial for EFL contexts such as Tai-wan or Japan where English is mainly used in classrooms rather than in the students’ real lives, and where many EFL learners are eagerly attempt-ing to advance their English proficiency. On the other hand, its aims can be subliminal in having both language and content teachers “change teach-ing practices, or socially-oriented, in boosting levels of harmony between inter-ethnic groups” (Marsh, 2009: 1). This purpose is also vital for those contexts in which CLIL programmes have a combination of internation-al and local students such as in the cases researched in the present study.

Contributed to by the strong driver of socio-economic globalisation, the number of CLIL programmes has increased rapidly in the last decade, and CLIL has become a mainstream form of education in many European countries (Maljers, Marsh, & Wolff, 2007; Gefaell & Unterberger, 2010). Simi-larly, in tertiary education in Taiwan, the establishment of CLIL programmes has been encouraged by the educational authorities, mainly the Ministry of Education (MOE), to push the goal of internationalised higher education forward. The preeminent additional language taught by CLIL programmes is English (Lasagabaster & Sierra, 2010). In order to evaluate the effective-ness and management of offering CLIL education, the MOE conducted its first large-scale appraisal in 2011, which is exceptional in that it was a na-tionwide appraisal conducted by the top educational authority of the coun-try rather than by teachers, schools, researchers, or local governments, as in the European context.

1 Parts of the sections including the introduction, literature review, and background to the study are derived from the author’s previous works and research grant proposals (Yang, 2014 a.b., 2015, 2016 a.b., Yang & Gosling, 2013, 2014) and are partially modified for the present report.



Previous studies on evaluating CLIL education have mainly been conducted from three dimensions: firstly, an examination of the effec-tiveness of CLIL education by comparing learners’ progress in the (addi-tional) language proficiency before and after the implementation (e.g., Connon, 2007; Dalton-Puffer, 2008, 2009); secondly, an evaluation of the CLIL implementation, mainly in secondary schools in European contexts, by applying a sole research method (e.g., Jappinen, 2005); and thirdly the evaluation of CLIL education mainly initiated by the micro-involvers, namely, CLIL teachers and researchers (e.g., Perez-Canado, 2012).

However, apart from European contexts, investigations of implement-ing and evaluating CLIL programmes on a large scale are apparently rare-ly documented in the literature (Perez-Canado, 2012), in particular, in Asian EFL situations where CLIL education is increasingly obtaining the attention of educational authorities who hope to establish close connections to glo-balisation by integrating English, the most dominant foreign language, into disciplines at the tertiary level. Therefore, this study aims to bridge the gap by examining how EFL learners view the new learning approach of CLIL based on their expectations and experiences with a quantitative survey, and probing CLIL learners’ and practitioners’ deeper attitudes and perceptions of CLIL during qualitative interviews to see if their opinions correspond to the official results and previous studies.

LITERATURE REVIEW1

As Coyle (2007b) suggested, effective CLIL takes place “through progres-sion in knowledge, skills and understanding of the content, engagement in associated cognitive processing, interaction in the communicative con-text, the development of appropriate language knowledge and skills as well as experiencing a deepening intercultural awareness” (p. 550) and thus much current research focuses on evaluating the relationship be-tween input and outcome in both content knowledge and linguistic competence, while still other studies spotlight the evaluation of CLIL pro-grammes as a whole in various settings, and identify their potential prob-lems and the difficulties of implementation.


YANG

In the first instance, a number of studies in European contexts have examined the effectiveness of using the CLIL approach to teach subjects other than language courses at different levels of education. For instance, in Jappinen’s (2005) study, the results reveal that the students in the ex-perimental CLIL group had more significant cognitional development in learning mathematics and science than those in the control group, and the research confirms that “the Finnish CLIL environments in public main-stream L1 education have succeeded” (p. 161). However, she also warned that abstract topics may not be well suited for young CLIL learners in her context.

Similarly, Burston and Kyprianou (2009) also concluded that their CLIL programmes in teaching architecture, biology and law in a Cyprus university were holistically successful in that the participants were very motivated and satisfied with the programmes, the departments were sat-isfied, and most importantly, the students’ grades had improved.

In contrast to the above two studies, Dalton-Puffer (2008) examined the outcome of CLIL education from the perspectives of language compe-tencies and use in the classroom. She argued that “CLIL significantly en-hances the language skills of the broad group of students whose foreign language talents or interest are average” (p. 5) where the favourably affect-ed language competencies include receptive skills, vocabulary, morphology, creativity, risk-taking, fluency, quantity and emotive/affective outcomes.

In addition, in CLIL classrooms, teachers tend to provide elaborate feedback and additionally reserve more space and time for students’ com-ments and interaction. At the end of her paper, Dalton-Puffer (2008) pin-pointed the fact that investigation of students’ perceptions of how they view their CLIL teachers’ (including content teachers and EFL teachers) lan-guage problems and corrections in instruction is still lacking, so the pres-ent research aims to attend to this concern.

Coonan (2007), on the other hand, studied the outcomes of both lan-guage competence and non-language effects in CLIL classrooms through teachers’ self-observation and introspection. Her results are similar to Dal-ton-Puffer’s (2008) in that she found that learners’ linguistic receptive skills are promoted more than their productive skills. Besides, she emphasised that CLIL positively affects the way students learn the content because



extra care is taken by teachers to help them overcome the hurdles and to nurture language growth through the content.

Furthermore, a great number of studies have also confirmed the positive improvement in learners’ linguistic production in a CLIL context, particularly in terms of their receptive skills and lexical richness (see Alonso, Grisalena, & Campo, 2008; Dalton-Puffer, Huttner, Schindelegger & Smit, 2009; Infante, Benvenuto & Lastrucci, 2008; Loreanc-Paszylk, 2007, 2009; Lorenzo, Casal & Moore, 2010; Olavide, 2009; Trento, 2008; Zabore, 2008), except for in Airey’s (2009) study where the students who were taught in dual languages per-formed better than those taught exclusively in English.

Nevertheless, the claimed dual effects of CLIL are also questioned in some studies, and some disadvantages have been unexpectedly generat-ed in some European contexts. For example, Seikkula-Leino (2007), who researched CLIL learners’ affective performance and attitudes, noted that although pupils in CLIL programmes were more motivated to study and to use the foreign language (FL), they sometimes felt incompetent and in-adequate in learning, a situation brought about by the complexities and difficulties of learning through a FL. CLIL caused low self-esteem regarding the students’ FL ability. Bruton (2011a. b.) not only questioned the results of improved FL ability in the experiments comparing students’ performance in CLIL and non-CLIL groups, but raised critical concerns about the real ef-fect of the development of the dual focus for average students (not select-ed or elite students) in CLIL education. Furthermore, he also highlighted the concern of quality control in CLIL, including CLIL teacher training.

In addition to the European cases, a number of CLIL studies have also been conducted in Asian contexts, but their attitudes are more cau-tious. For example, in Japan the implementation of CLIL has aroused con-cerns that FL may replace the priority of learning their mother tongue for young learners (Sasajima, Ikeda, Hemmi, & Reilly, 2011). Likewise, a num-ber of studies (Lee & Chang, 2008; Mackenzie, 2008; Marsh & Hood, 2008) have also contended that as a potentially new teaching approach in Asian contexts, CLIL has motivational advantages, develops multiple intelligenc-es, and acts as an enabler for Asian EFL learners to achieve positive atti-


YANG

tudes towards the relevance of English, but still, CLIL teacher preparation is apparently insufficient in this area.

Feng (2010) also described the diverse models of CLIL adopted in the greater Chinese circle (Hong Kong, Mainland China, Macau and Taiwan). He argued that CLIL exists as a continuum from language-driven to con-tent-driven ends, and each context is mapped diversely on this continuum according to differences in cultural, governmental or economic factors. Sim-ilar discussion of the probabilities of implementing CLIL in Taiwan and Chi-na has confirmed that, in contrast to the traditional ELT approaches, CLIL can be an alternative for accommodating students’ subject knowledge and linguistic competence, although there is still the issue of training qualified CLIL teachers as in other contexts (Hou, 2007; Lo, 2007; Luo, 2006). Besides, the nature of the different subjects taught in English, the English acquired in the English language classes, and the students’ English proficiency en-try level can also greatly affect the teaching efficacy in Asian CLIL educa-tion, as in the case of Hong Kong (Marsh, Hau, & Kong, 2000). To the best of my knowledge, currently, the only empirical study investigating CLIL education in Taiwan is Yang and Gosling’s research (2013). After examin-ing a single highly-recommended CLIL programme, they concluded that CLIL teacher development is an urgent need in Taiwan, further consider-ation of whether all students in EFL settings can equally learn or thrive in a CLIL programme is warranted, and accommodation should be made for language support. However, compared to the rich CLIL research document-ed in European contexts, the practical outcomes of CLIL education in Asian contexts, in particular Taiwan, are still relatively understudied.

Regarding the evaluation of CLIL in various settings, Perez-Canado (2012) conducted an extensive review of the cases across European coun-tries. She concluded that, not surprisingly, a great majority of the CLIL pro-grammes have been championed in Europe, but unfortunately the fact is that these programmes are full of methodological flaws in their design. Most of the evaluations are stand-alone quantitative studies which do not take dynamic variables or statistical analysis into consideration. Thus, she advises that future studies evaluating CLIL should rigorously investigate “whether the gains observed are truly ascribed to CLIL practices“ (p. 16).



So far, most of the experimental studies used learners’ language output and knowledge competency as the measures to establish the effectiveness of the programmes. However, students, as “the target group and intend-ed beneficiary of CLIL education, have at this point not had much voice in the development of and discourse on CLIL in general” (Dalton-Puffer et al., 2009). Students’ attitudes and perceptions of CLIL education are relatively understudied, except for a few studies (e.g., Dalton-Puffer et al., 2009; Ge-faell & Unterberger; 2010; Yassin, Marsh, Tek, & Ying, 2009).

Although CLIL is still in its infancy in Taiwan, several studies have been conducted by the author and colleague to unveil its current imple-mentations and effects. They, for instance, studied why a CLIL programme would be highly recognised or not recommended by Taiwan’s MOE after its first official appraisal of all the CLIL programmes in Taiwan universi-ties (Yang & Gosling, 2013, 2014). They concluded that CLIL learners’ atti-tudes towards CLIL education can be a key factor, affecting the results of the programme appraisal. Those who hold a relatively positive attitude towards the CLIL approach may help the programme pass the apprais-al and make it highly recommended by the authorities, and vice versa. In addition, Yang found that proficient CLIL learners tend to use indirect lan-guage learning strategies relatively more frequently in order to adapt to the highly collaborative learning environment of tertiary CLIL education (Yang, 2016 b). His studies (Yang, 2014a, 2015), furthermore, confirmed that CLIL undergraduates did outperform their non-CLIL counterparts in both content achievement and linguistic outcomes, and thus also showed high-er mobility and employability in the globalised job market. These studies, so far, have clearly evidenced the positive benefits of CLIL education real-ised in an Asian EFL context.

However, in Taiwan, although CLIL programmes are encouraged by the MOE, currently an objective and systematic evaluation of both students’ and teachers’ performance in CLIL education is still seriously lacking. Thus, the issue of whether or not the quality of CLIL lessons can be controlled is doubted by the public (Chen, 2012). In other words, the issue of how stake-holders such as students or programme managers evaluate their CLIL edu-cation is generally overlooked. Thus, to bridge the gap, this study attempted


YANG

to combine both qualitative and quantitative methods to evaluate CLIL ed-ucation along with considering the stakeholders’ variables, and to triangu-late the evaluation results from various sources of data by comparing and contrasting a number of CLIL programmes in Taiwan. Hopefully, the study can highlight implications for managing CLIL programmes, and provide practical suggestions for how to evaluate students’ performance in terms of both content and FL in EFL contexts in Taiwan.

To be specific, this research mainly explores: firstly, how the CLIL learners at tertiary level view the effects of CLIL education; secondly, how the stakeholders of the CLIL programmes, namely, the managers and learn-ers, view the claimed dual benefits of CLIL. To be specific, the research fo-cuses on the following questions:

What are the students’ perceptions of and attitudes towards CLIL education?

What are the differences or similarities in the views of the students and managers of the implementations of the CLIL education in Taiwan?

METHOD

A mixed-methods research framework

As suggested by Perez-Canado (2012), the research design and methodolo-gy of future studies on CLIL should combine mixed methods, adopt trian-gulation, and employ multivariate procedures. Hence, the present study used both quantitative and qualitative methods to collect and analyse data, as the study’s focus lent itself to a mixed methods approach using qualitative and quantitative approaches in combination, and drew upon the advantages of each. This method can serve to provide a better under-standing than either approach alone (Creswell & Plano, 2007). Using the notation proposed by Morse (1991, 2003), this is a QUANQUAL explanato-ry mixed-methods design. The results are discussed using the quantitative results in combination with the qualitative data to enhance the analysis and further illuminate some of the findings. This methodology allows the integration of data and responses from the various stakeholders, from



the MOE, through the programme directors, the teaching staff and the stu-dents as outlined below.

A quantitative approach was employed with the students who were given a Chinese close-ended questionnaire with a Likert scale five-point response (from 5, strongly agree to 1, strongly disagree) designed for the purpose of the study. The question items in the questionnaire are divid-ed into two major sections and five sub-categories, that is, linguistic im-provement, content knowledge learning, classroom practices, supports and strategies, and affective attitudes. Before the students completed the questionnaire, it had been piloted by one L2 teacher, one L1 teacher and a number of students in the selected programmes to ensure its comprehen-sibility and readability. The reliability of Cronbach’s alpha reaches .856.

The qualitative data came from several sources in addition to a refer-ence to the MOE appraisal report. Interviews were held with the teaching staff, where each teacher and programme director was individually inter-viewed for about half an hour in their respective first languages. Chinese interviews were transcribed into English and checked by independent translators. In addition, the students responded to open-ended questions on the given questionnaire.

The researched context

In a globalised society, politics, culture and economics are becoming close-ly dependent on each other, and higher education, which is responsible for cultivating professionals and transmitting professional knowledge, is also facing competitiveness and challenges from other countries. Consequent-ly, to facilitate cultural exchange, to publicise the features of Taiwanese higher education, to increase the standards of tertiary education, to solid-ify the competitiveness of universities, and to broaden college students’ global vision and language proficiency, the MOE of Taiwan has long en-couraged institutes in higher education to set up courses which use English as a medium of instructing disciplinary knowledge. The final purposes of these programmes are to recruit foreign students to study in Taiwan, to increase the competitiveness of Taiwanese higher education, and to devel-op professionals with the ability of international mobility.


YANG

In addition to the numerous CLIL courses, there are 92 CLIL degree programmes in 29 universities2 conferring bachelor, master and doctoral degrees. The earliest programmes were established in 1979 and the latest were set up in 2010. All of the programmes are expected to use English as the only language of instruction in the classroom. The MOE provides in-centives of up to 10 million NT dollars per university to encourage the es-tablishment of more CLIL degree programmes, while also conducting a national-scale appraisal of all the programmes to ensure their quality in October 2011. The 92 programmes were classified into three different rank-ings, namely, highly recommended, recommended, and not recommended.

Sampling CLIL programmes and participantsThis research sampled the programmes based on several categories. Firstly, all of the current CLIL programmes were divided into undergraduate (UG) and postgraduate (PG) programmes. Then, the results of the national ap-praisal were used to further classify the two major categories into highly recommended (HR), recommended (RD), and not recommended (NR) pro-grammes, which resulted in six categories in total. Next, one programme (G) was randomly selected as the study context from each of these six cate-gories; therefore, six CLIL programmes were investigated in this study. Ap-proximately ten learners in each programme were invited to complete the questionnaire, and the return rate reached 90% (i.e., N=54 in total). Of the participants, 38 were undergraduate students while the rest were post-graduates (including master and doctoral students) from six different uni-versities island-wide (5 comprehensive, 1 polytechnic, 2 national, 4 private), all of whom are Taiwan nationals. A majority of them (40.7%, N=22) were in the final year of their programme, the first year (7.4%, N=4), the second year (33.3%, N=18), and the third year (18.5%, N=10); 81.5% (N=44) of them came from normal senior high schools while only 18.5% (N=10) had studied at a vocational high school. Normally, senior high school students’ English competency is higher than that of polytechnic students in Taiwan; therefore,

2 CLIL degree programmes include bachelor, master and doctoral programmes where all disciplinary courses are conducted in the major foreign language i.e. English; indivi-dual CLIL courses offered by universities are not included in this study.



CLIL programmes are more commonly implemented in the former than in the latter. Hence, over half of the respondents (57.4%) had reached equiv-alent of the CEFR B2 level in an English proficiency test (over TOEIC 750) while 40.7% had reached CEFR B1 (TOEIC 500-750), and only 1 participant had lower competency (less than TOEIC 500). Generally, the present partic-ipants had better command of English compared to their non-CLIL peers, and had achieved the benchmark of English language proficiency required by most Taiwanese universities.

Finally, the directors (D) and also the teachers of the six programmes were interviewed together with four randomly-selected students (S) who helped complete the questionnaire survey from each programme. Total-ly, 24 students, making up six focus groups, joined the interviews for this study. Since the CLIL education investigated in the present study was fo-cused on degree-based programmes rather than departments, all the pro-gramme managers (or chairs) and teachers came from other departments to support the programme, so there were no full-time teaching faculty Similarly, all the programme chairs (4 males, 2 females) were Taiwanese teachers with proficient English skills due to their education background in English-speaking countries. All of them, during the research stage, had taken the position for one term (i.e., 3 years). The student interviewees constituted four undergraduate groups and two postgraduate groups. All the interviews lasted for about one hour and were conducted in Manda-rin Chinese to facilitate contribution and avoid any misunderstandings.

Data analysis

To answer research questions 1, a Chinese close-ended questionnaire (see Appendix 1 for the English version, Yang & Gosling, 2013) on a five-point Likert scale was designed, and the collected data were analysed with the help of the statistical software, SPSS 14.0. In addition to the descriptive analysis, t-tests and one-way ANOVA were also performed to gauge any significant differenc-es in the variables. Then, to answer research question 2, further interviews with the directors, teachers and students were conducted. The interviews were recorded and then transcribed into English by the researcher and two research assistants together with the help of NS colleagues’ proof-


YANG

reading. Then, all the English transcriptions were processed with the as-sistance of the qualitative data analysis software, WeftQDA. The study applied grounded theory, which is extensively used to analyse qualitative data by coding data with similar concepts into various categories (Glaser & Strauss, 1967). The coding was checked and confirmed by the research assistants of this study.

RESULTS AND DISCUSSION

Descriptive results of the questionnaire survey

This following part answers research question 1: how Taiwan CLIL learn-ers perceive the effects of the current CLIL education, and is divided into five sub-categories. Linguistic improvement includes question items 1, 2, 3, 4, and 5; content knowledge learning: 6, 7, 8, and 9; classroom practices: 23, 24, 28, 29, and 32; supports and strategies: 10, 11, 12, 15, 16, 17, 18, 19, 20, 21, 26, 30, 32, 34, and 35; and affective attitudes: 13, 14, 22, 25, 31, 33, 36, 37, and 38 (Yang & Gosling, 2013).

Tables 1 to 5 display the results of the perceived linguistic improve-ments under CLIL education. Surprisingly, different from the previous stud-ies, a majority of the present participants generally did not show high agreement with the increment in linguistic skills under the CLIL approach. CLIL learners usually have better performance in receptive skills than in productive skills, which has been extensively documented in previous re-search (see Yang, 2014a for a detailed discussion). Yet, the current partic-ipants, in contrast, did not hold similar attitudes. It is assumed that the participants’ overall English proficiency was already high; thus, it is dif-ficult for them to perceive any significant change. In addition, it is likely that teaching linguistic elements is not stressed as equally as instructing content knowledge in Taiwan CLIL programmes; hence, the participants may not be clearly aware of the improvements in their English language skills. CLIL is sometimes criticised as an elitist approach (Burton, 2011a. b.) and Taiwan CLIL programmes also purposefully recruit highly proficient English learners due to the fear that lower English achievers may not be able to adapt well to courses instructed only in English. Ironically, this



selective recruitment seems to lead to a lack of general agreement regard-ing linguistic improvement as a result of CLIL education.

Table 1. My writing proficiency has improved (Q1)

Frequency Percent Valid Percent Cumulative PercentValid 1.00 1 1.9 1.9 1.9

2.00 13 24.1 24.1 25.9 3.00 27 50.0 50.0 75.9 4.00 12 22.2 22.2 98.1 5.00 1 1.9 1.9 100.0 Total 54 100.0 100.0

Table 2. My reading proficiency has improved (Q2) Frequency Percent Valid Percent Cumulative Percent

Valid 1.00 4 7.4 7.4 7.4 2.00 23 42.6 42.6 50.0 3.00 21 38.9 38.9 88.9 4.00 5 9.3 9.3 98.1 5.00 1 1.9 1.9 100.0 Total 54 100.0 100.0

Table 3. My speaking proficiency has improved (Q3)

Frequency Percent Valid Percent Cumulative Percent

Valid 1.00 14 25.9 25.9 25.92.00 17 31.5 31.5 57.43.00 17 31.5 31.5 88.94.00 6 11.1 11.1 100.0Total 54 100.0 100.0

Table 4. My listening proficiency has improved (Q4) Frequency Percent Valid Percent Cumulative Percent

Valid 1.00 19 35.2 35.2 35.2 2.00 20 37.0 37.0 72.2 3.00 12 22.2 22.2 94.4 4.00 3 5.6 5.6 100.0 Total 54 100.0 100.0


YANG

Table 5. My overall English proficiency has improved (Q5)


Valid 1.00 11 20.4 20.4 20.4 2.00 17 31.5 31.5 51.9 3.00 19 35.2 35.2 87.0 4.00 7 13.0 13.0 100.0 Total 54 100.0 100.0

The following Tables 6 to 9 exhibit the participants’ opinions on learn-ing content knowledge via the additional language, English. However, the results show a different picture from the previous discussion. The major-ity of the respondents admitted to having difficulties understanding the courses delivered in English. In other words, half of the students indicated that learning content knowledge through an additional language is not as easy as they had believed, even though their command of English was gen-erally rather good. There are various possible reasons for this. For instance, learning content and language simultaneously brings students double pressure, which may hinder the acquisition of content knowledge com-pared to learning in Chinese-speaking classrooms. Another reason might be the different purposes of using English in the classroom. Most Taiwan-ese EFL learners learn English for the purpose of taking tests, and the En-glish curriculum is designed based on learning the language for general purposes, which does not sufficiently prepare learners for using English for academic purposes (EAP) at tertiary level; neither are there many EAP courses provided to bridge this gap in the CLIL programmes. Although CLIL is not regarded as providing language courses, and CLIL and ESP (English for specific purposes) are seemingly two extremes of the language learn-ing curriculum, additional ESP courses can be offered as supplements and supports for CLIL learners (Yang, 2016 a), in particular in higher education where CLIL students have to learn academic disciplines via very academic English which clearly has some language conventions which must be fol-lowed but which are seldom taught in Taiwan’s English learning context. Thus, it is advisable that tailor-made ESP courses be designed and provid-ed as additional support for CLIL learners.



Table 6. I have no difficulty reading articles (Q6)


Valid 1.00 3 5.6 5.6 5.6 2.00 23 42.6 42.6 48.1 3.00 16 29.6 29.6 77.8 4.00 10 18.5 18.5 96.3 5.00 2 3.7 3.7 100.0 Total 54 100.0 100.0


2.00 15 27.8 27.8 29.6 3.00 22 40.7 40.7 70.4 4.00 12 22.2 22.2 92.6 5.00 4 7.4 7.4 100.0 Total 54 100.0 100.0


2.00 22 40.7 40.7 46.3 3.00 20 37.0 37.0 83.3 4.00 6 11.1 11.1 94.4 5.00 3 5.6 5.6 100.0 Total 54 100.0 100.0


2.00 26 48.1 48.1 70.4 3.00 11 20.4 20.4 90.7 4.00 4 7.4 7.4 98.1 5.00 1 1.9 1.9 100.0 Total 54 100.0 100.0

Table 7. I have no difficulty writing my content knowledge (Q7)

Table 8. I have no difficulty orally expressing my content knowledge (Q8)

Table 9. Generally, I can understand most CLIL courses (Q9)


YANG

Regarding the third category, supports and strategies, Tables 10 to 14 show the results as follows. First, in line with the previous discussion, over half of the participants showed a need for extra supportive language courses to assist their CLIL learning. Although CLIL claims a dual-focused ap-proach, catering for both content and language learning, content teaching is usually overemphasised in relation to language teaching. It is therefore argued that when CLIL is implemented in secondary or primary schools, the courses are less academic and extra language support to master the courses is also thus less necessary. Yet, in tertiary CLIL, nearly all the cours-es are academic-oriented and the English required to master the disciplines is more complicated, but also conventionalised with predictable peculiar-ities. Thus, extra academic English courses may be beneficial for lessening learners’ anxiety and pressure of acquiring content knowledge via the addi-tional language in CLIL contexts. Without appropriate scaffolding, learners may translate English into Chinese in order to learn the subject in the CLIL classroom, which is evidenced in the results of question items 28 and 29.

Second, the participants did not expect practitioners’ timely use of the learners’ mother tongue to help their learning and facilitate their un-derstanding in CLIL classrooms, as shown in Table 14. In Taiwan, CLIL is classed as a high-exposure CLIL mode (Yang & Gosling, 2013, 2014) as En-glish is often used for over 60% of the instructional time. Some local CLIL practitioners and learners may misinterpret that CLIL should be instruct-ed using English only, or at least as much as possible; however, appropri-ately using the L1 (mother tongue) is helpful and is in fact encouraged in CLIL classrooms to keep learners motivated and engaged in CLIL learn-ing (Lo, 2015), thus increasing the effect of CLIL education. Hence, timely code-switching while instructing or explaining difficult concepts in the disciplines should be adopted by local CLIL practitioners, as it is believed to be able to facilitate the understanding of the content knowledge and increase the satisfaction with the programme.

In the category of examining teachers’ actual CLIL practices in class, a number of questions were asked. Table 15 lists the overall results. In general, the participants agreed relatively less with their teachers’ actual practice in class. First, the present participants would prefer Taiwanese teachers to native English-speaking teachers to teach CLIL courses (Q11, Q12). They also



Table 10. I need no extra English courses to increase my English ability (Q23)


Valid 1.00 9 16.7 16.7 16.7 2.00 16 29.6 29.6 46.3 3.00 12 22.2 22.2 68.5 4.00 15 27.8 27.8 96.3 5.00 2 3.7 3.7 100.0 Total 54 100.0 100.0

Table 11. I need extra English courses to increase my English ability (Q24)


2.00 16 29.6 29.6 33.3 3.00 15 27.8 27.8 61.1 4.00 15 27.8 27.8 88.9 5.00 6 11.1 11.1 100.0 Total 54 100.0 100.0

Table 12. The CLIL programme helps me think in English (Q28) Frequency Percent Valid Percent Cumulative Percent

Valid 1.00 14 25.9 25.9 25.9 2.00 24 44.4 44.4 70.4 3.00 10 18.5 18.5 88.9 4.00 6 11.1 11.1 100.0 Total 54 100.0 100.0

Table 13. I tend to convert Chinese into English in CLIL courses (Q29) Frequency Percent Valid Percent Cumulative Percent

Valid 1.00 5 9.3 9.3 9.3 2.00 18 33.3 33.3 42.6 3.00 14 25.9 25.9 68.5 4.00 11 20.4 20.4 88.9 5.00 6 11.1 11.1 100.0 Total 54 100.0 100.0


YANG

showed less agreement with native English speaking teachers being more capable of teaching CLIL subjects (Q20, Q21). It is believed that local teach-ers share the same cultural and language backgrounds with the learners so interaction with each other would be easier. In addition, postgraduate students may receive many tutorials with their teachers to discuss their dissertations. Teachers who speak the same L1 are thus more popular with postgraduate students as their supervisors.

Second, a majority of the respondents did not agree that the teach-ing methods are different in CLIL class (Q10), which pinpoints a major prob-lem in the current Taiwan CLIL education. That is, most CLIL teachers are content teachers who have a good command of English, but unfortunate-ly lack proper training of teaching language. Thus, they tend to replicate the way they teach content in their L1 in the CLIL context, and make no ob-vious attempt to adjust their English to increase content comprehension, which is also evidenced in Q30 and Q35. In other words, insufficient CLIL teacher training leads to this duplication without proper adjustments to the teaching methods. This insufficiency also causes dissatisfaction with the assessments in CLIL courses. CLIL is dual-focused and differs from tra-ditional education approaches; thus, assessing both language achievement and content learning outcomes is necessary, but this is often neglected by CLIL practitioners (Massler, 2010). Similarly, the results of question items 17, 18 and 19 also indicate that traditional lectures and group discussion are still the most common teaching methods used in the current CLIL programmes, but they are not very popular with the students. All these concerns mirror the necessity of offering CLIL teacher training for practitioners.

Table 14. Teachers need to use Chinese in a timely manner to teach CLIL courses whenever needed (Q32)


2.00 25 46.3 46.3 77.8 3.00 5 9.3 9.3 87.0 4.00 6 11.1 11.1 98.1 5.00 1 1.9 1.9 100.0 Total 54 100.0 100.0



However, nearly 90% of the respondents did not expect teachers’ cor-rection of their errors (Q34). Error correction may prevent learners from en-gaging in linguistic production, and discourage them from studying in a CLIL programme, for fear of losing face in public. Indeed, at the initial stage of CLIL education, fluency is more desired than accuracy in terms of learn-ers’ linguistic production. The respondents held a divided attitude towards Q26. They showed an uncertain attitude towards which language is pre-ferred to learn content knowledge, L1 or English. CLIL is still a rather new educational approach and thus not much positive or empirical evidence can be found locally, except for Yang’s research (2014a, 2015). It is argued that learners are likely to have more positive attitudes towards CLIL once more local studies have confirmed its effectiveness and benefits.

Table 15. Classroom practices (%) (N=54)

Scale Q. item

5 4 3 2 1

10 3.7 11.1 27.8 42.6 14.811 3.7 5.6 31.5 27.8 31.512 3.7 25.9 42.6 16.7 11.115 1.9 1.9 3.7 50 42.616 1.9 0 29.6 51.9 16.717 0 11.1 38.9 40.7 9.318 3.7 9.3 29.6 42.6 14.819 7.4 13 35.2 31.5 1320 13 13 27.8 31.5 14.821 3.7 27.8 50 18.5 026 7.4 22.2 40.7 24.1 5.630 0 16.7 24.1 50 9.334 0 5.6 11.1 53.7 29.635 0 3.7 29.6 44.4 22.2

Table 16 concludes the last category of the questionnaire, i.e. affec-tive attitudes, and presents the overall results. First, the reason why the students decided to study in a CLIL programme may be their parents’ en-couragement or their interest (Q13 and Q14). According to the MOE’s policy, Taiwan CLIL programmes are established not only to attract international


YANG

students but also to develop local students’ content knowledge and lan-guage proficiency, which is supposed to help increase their future employ-ability and mobility in the globalised job market, and this non-linguistic benefit of CLIL education has also been confirmed in Yang’s research (2015). Thus, it is self-evident that parents would encourage students to study in a CLIL programme.

However, after studying a CLIL programme for a period of time, it seems that it does not increase most students’ motivation to learn either English or the content knowledge. It is possible that CLIL education is not as wonderful as they may have expected. Thus, in their overall judgement, they did not show a sufficiently supportive attitude to recommend this in-novative approach to other students. It seems that the present CLIL learners, in general, are not very satisfied with the claimed benefits of CLIL educa-tion. Probably, the fact that they did not perceive a significant increase in their English proficiency is one of the reasons. Yet, in Yang’s (2014a) study, he pointed out the interesting fact that CLIL learners tend to devalue the actual increment of their language achievements and content knowledge performance while in fact showing an increase in both areas according to tests. This also echoes the previous issue, namely the use of appropriate assessments in CLIL education.

Table 16. Affective attitudes (%) (N=54)

Scale Q. item

5 4 3 2 1

13 3.7 16.7 27.8 27.8 24.1

14 31.5 33.3 22.2 13 0

22 0 13 37 42.6 7.4

25 1.9 27.8 27.8 29.6 3.7

31 0 11.1 31.5 48.1 9.3

33 0 7.4 24.1 48.1 20.4

36 7.4 5.6 40.7 33.3 13.0

37 0 0 24.1 59.3 16.7

38 0 5.6 35.2 42.6 16.7



Significant differences by variablesDiffering from the descriptive data in section 4.1, the following discussion focuses on the significant differences caused by different variables. In the present study, question items regarding the demographic information of the questionnaire were used as the variables to gauge the statistical signif-icance among various groups by performing t-tests and one-way ANOVA.

First of all, only two question items showed a significant difference between female and male learners, i.e. Q16, satisfaction with teachers’ as-sessments in CLIL courses (t=-.761, <.01) and Q17, the preference of teachers’ lectures in the CLIL classroom (t=-1.175, <.01). Male learners showed signifi-cantly higher satisfaction with teachers’ assessments and lectures than the female students did. It is possible that, since female EFL learners general-ly have a more positive attitude towards English learning and have high-er English proficiency than their male peers (Kobayahsi, 2002), they would naturally care more about how they are assessed. Therefore, when English performance is not as highly stressed and properly assessed as the content knowledge is, as in the present context, they would feel dissatisfied. Sim-ilarly, female CLIL learners may expect more diverse teaching methods to be used in the CLIL classroom, and thus also showed a reluctant attitude if teachers’ lectures still dominated as they do in traditional courses de-livered in the L1. However, this issue may be worth further investigations.

There is only one item showing significant difference between un-dergraduate and postgraduate CLIL learners, i.e. improvement in writing skills (t=.640, <.05). The undergraduates had significantly higher satisfac-tion with this than the postgraduates did. It is assumed that English writ-ing skills are more important for postgraduate study, and learners would show greater dissatisfaction if they felt that CLIL failed to improve their English writing skills, in particular the skills required for dissertation writ-ing. This result indicates that when a CLIL programme is being planned, considering which linguistic elements would be more practically required by different learners is essential. For instance, CLIL implemented in post-graduate study should place much emphasis on integrating English writ-ing with content learning, while listening and reading skills may be more highly required in undergraduate study.


YANG

For the variable of which year they were studying, no significant difference was found, which indicates that the respondents’ concerns and attitudes towards CLIL education are generally consistent across the years of study. However, for the variable of their previous major in senior high school (senior high school without a specific major, vocational high school with a non-language major and with a language major), a num-ber of items show statistical significance among the groups as shown in Table 17. In general, the CLIL students who came from senior high schools had a more negative attitude towards CLIL education. They were less sat-isfied with the improvement in their English skills and with the teachers’ teaching methods in CLIL courses; thus, they showed less motivation and involvement in their CLIL courses, and also believed that Taiwanese teach-ers have difficulty teaching CLIL programmes. In contrast, those learners with a vocational high school background had a more welcoming and ac-knowledging attitude towards CLIL education, but also expressed greater need for extra English courses. The likely reasons might be similar to those discussed previously. That is, the higher English achievers may have a high-er expectation of CLIL increasing their English proficiency, but they easily become dissatisfied if the linguistic elements are not equally addressed in the courses. On the other hand, students with intermediate or lower En-glish proficiency may regard CLIL education as one way to improve their English abilities, so they showed more positive attitudes towards it. Again, the implementation of a CLIL course or programme should take contextual differences into consideration and require necessary adaptation to fit the local settings. This can also explain why CLIL, used as an umbrella term, exists in many different versions worldwide (Coyle, 2008).

The variable of learners’ English proficiency led to the most signif-icant differences in this study (see Table 18). Over half of the participants had a high command of English (CEFR, B2 level or above) while the rest were at the B1 level (with one student below). Those who had better En-glish proficiency showed significantly higher agreement with their own performance and with native English-speaking teachers’ capability in CLIL courses, but in contrast, lower satisfaction with the teachers’ perfor-mance and the CLIL approach overall; thus, it is obviously difficult for them to make a recommendation for future students. Contrarily, those with



lower English proficiency were more motivated to study in their CLIL pro-gramme, acknowledging its effects on helping increase their future employ-ability, and recommending the approach to prospective students, although they had relatively more difficulties understanding the CLIL course mate-rials. They tended to convert English into Chinese in CLIL class and also re-quired additional language courses to bridge the gap in comprehending English-delivered lectures. Moreover, they preferred local teachers to teach CLIL courses probably because local teachers are more empathetic and un-derstand learners’ needs and difficulties when learning with an innova-tive approach (Árva & Medgyes, 2000; Medgyes, 1992).

In the last variable, the time duration of studying English, only one item was found to have significance across the groups, i.e. Q33. CLIL ben-efits to future job prospects (F(2:51)=3.662, P<.05). It was found that those who had studied for longer in the programme showed higher agreement with this benefit than junior CLIL learners did. The former learners were about to graduate from the programme and had to find a job, so the du-al-focused CLIL education, developing both content knowledge and lan-guage competency, was expected to be helpful for their job hunting in a globalised market. Indeed, in Yang’s (2015) research, he argues that in-creasing CLIL learners’ mobility and employability are the two evidenced non-linguistic benefits of CLIL education.

A short conclusion to this questionnaire survey is that the imple-mentation of CLIL education should be contextualised, depending on the learners’ needs and customised curricula to accommodate both content and language learning.

Question item One-way ANOVAQ4. Improvement in listening skills (F(2:51)=3.978, P<.05)Q5. Improvement in overall English proficiency (F(2:51)=5.272, P<.01)Q10. Teachers using different teaching methods (F(2:51)=7.118, P<.01)Q22. Students showing high involvement (F(2:51)=2.449, P<.05)Q23. No extra English course needed (F(2:51)=5.691, P<.01)Q25. Increment in motivation to learn English (F(2:51)=6.480, P<.01)Q27. Taiwanese teachers’ difficulty teaching CLIL (F(2:51)=6.232, P<.01)

Table 17. Items with significance across various previousmajors in high school


YANG

Question item One-way ANOVAQ5. The increase in overall English proficiency (F(2:51)=6.006, P<.01)Q6. No difficulty reading articles (F(2:51)=8.030, P<.01)Q8. No difficulty orally expressing content knowledge. (F(2:51)=4.523, P<.05)Q9. Understand most CLIL courses (F(2:51)=4.828, P<.05)Q10. Teachers using different teaching methods (F(2:51)=3.224, P<.05)Q11. Native English-speaking teachers preferred (F(2:51)=4.665, P<.05)Q13. Interested in CLIL (F(2:51)=3.815, P<.05)Q18. Group discussion in CLIL courses preferred (F(2:51)=3.291, P<.05)Q20. English-speaking teachers more capable of teaching CLIL (F(2:51)=4.084, P<.05)Q22. Students showing high involvement (F(2:51)=5.207, P<.01)Q23. No extra English courses needed (F(2:51)=4.877, P<.05)Q24. Extra English courses needed (F(2:51)=5.510, P<.01)Q25. Increment in motivation to learn English (F(2:51)=3.815, P<.05)Q26. Taiwanese teachers preferred to teach CLIL (F(2:51)=3.811, P<.05)Q28. CLIL helping thinking in English (F(2:51)=3.660, P<.05)Q29. Converting Chinese into English in CLIL courses (F(2:51)=7.697, P<.001)Q33. CLIL beneficial to future job prospects (F(2:51)=4.744, P<.05)Q38. Recommending CLIL to others (F(2:51)=8.010, P<.001)

Table 18. Items with significance across various levelsof English proficiency

Feedback from the interviews

From CLIL programme managers and teachers

In total, 6 programme directors (also CLIL practitioners) were interviewed for the present study, and their qualitative contribution can be discussed under the following headings.

Dual benefits of CLIL: All of the interviewees expressed the opinion that CLIL has two major advantages. From the university perspective, CLIL brings more international students to study in Taiwan’s tertiary education institutions, which is relatively more important for private universities. In-ternationalisation is one of the important indicators to appraise a univer-sity, and the number of international students is an essential criterion for evaluating a university’s internationalisation, which would greatly affect



the annual budget or subsidy a university can obtain from the MOE. From the students’ perspective, CLIL creates an international learning environment for local students, which means that students can acquire intercultural knowl-edge and learn English without travelling or studying abroad. One director said, “The number of international students we can recruit is unlimited and the implementation of CLIL programmes can help me pass the MOE evalua-tion in terms of internationalisation.”

Insufficient and unstable CLIL teachers: This is the major concern raised by most interviewees. They argued that CLIL is the right approach and should be implemented, but the problem is that teachers might not be ready for this change. Qualified CLIL teachers are still rare. Most of the current CLIL teachers are either native English-speaking content teachers or local content teachers with proficient English competency. However, CLIL seems to only mean lectures given in English to them; therefore, the accommodation of both content and language teaching is always over-looked. This probably may be the major reason making the present learn-ers dissatisfied with CLIL education. Overreliance on native speakers also leads to another problem, that is, “they are relatively unstable. I mean, they usually come to Taiwan for a short term and may terminate their contract anytime, which brings much uncertainty for the course director to arrange the future courses,” as another programme director commented.

Problem-based learning oriented: Due to the novelty of the CLIL ap-proach, traditional lectures may not be able to suit learners’ needs and expec-tations. Thus, group discussion, class activities, collaboration, and interaction are the common features of a CLIL classroom. However, compared to a simi-lar course delivered in Mandarin Chinese, these CLIL practices require much longer pre-class preparation by the teachers, so most universities are will-ing to pay double the salary for those CLIL teachers. One teacher remarked, “Unless you are also interested in and enthusiastic about CLIL education, preparing a CLIL course is really high pressure and causes anxiety. Honestly, double the pay is not a very attractive incentive. The time and effort spent on preparing the course is unbelievably immense.”

Unattended accuracy in English production: Nearly all the directors and teachers expressed the opinion that accuracy in learners’ English pro-duction is not what they are concerned about; in contrast, they attend much


YANG

more to the accuracy in communicating content knowledge, not wheth-er the content knowledge is presented grammatically correctly or not. Hence, they hardly ever offer feedback on language use to learners, and also seldom evaluate learners’ English performance in assessments. Iron-ically, receiving teachers’ feedback on their language use is what many learners yearn for in CLIL class. It is argued that this gap may distort the claimed dual-focuses of CLIL education and decrease learners’ motivation and satisfaction with CLIL.

The enhancement of global vision and confidence: Since most of the CLIL programmes embrace international students and local students in the same classroom, they offer chances for local students to learn how to interact with the international students, which greatly helps in develop-ing their intercultural awareness and communication, and builds up their confidence in using English for communication. According to one director’s observation, “CLIL learners comparatively have more confidence in express-ing their opinions in English, and the programme helps them to expand their global views as they spend time interacting with their international peers very often during and after class.” One director was very proud of her stu-dents’ performance and achievements in the CLIL programme and even claimed that CLIL will replace English departments in Taiwan tertiary ed-ucation soon as she believes that CLIL learners perform better at content knowledge and language proficiency while English majors only focus on language skills.

Preparing qualified CLIL teachers: Although the interviewees ex-pressed positive comments on the implementation of CLIL in Taiwan, they all suggested that it is indispensable to train qualified CLIL teachers immediately as the popularity of CLIL programmes is growing quickly in Taiwan’s universities. They suggested that workshops addressing how to integrate language teaching into content teaching should be held, which is particularly urgent for content teachers as they usually have no previ-ous formal training on language teaching. One director said, “A workshop on how to teach CLIL is not only needed but should also be provided period-ically for a long-term period. It is a bit ridiculous that the University some-times holds a workshop for simply two days and hopes that we, the content teachers, can become qualified CLIL teachers all of a sudden.”



All in all, compared to the preceding survey results, CLIL directors and teachers hold a much more positive attitude towards the effect of CLIL ed-ucation. However, if teachers’ anticipations and learners’ expectations of CLIL are not aligned, complaints and dissatisfaction would unavoidably occur. Hence, communication between the two parties and modification of the programmes are essential to increase the future implementation of CLIL and ensure its claimed benefits.

From CLIL learnersIn total, six focus groups of students were invited to join the interviews. Their opinions are categorised as the following headings, some of which are similar to the author’s previous findings (Yang & Gosling, 2013, 2014).

‘Asianalised’ CLIL: Many students expressed the opinion that they supposed there would have been many native English-speaking peers in class, but the fact was that most of their foreign peers come from Asian countries, in particular the South-East Asian region where the first lan-guage is not English. Ideally, all the CLIL programmes would have half native English-speaking students and half local students to create an in-ternationalised learning environment according to the MOE expecta-tion, but in fact native English speakers are still very rare in the current programmes. Thus, students may feel that the actual situation is totally different from what they had been told, and then disappointment and dis-satisfaction may occur. One student commented that “all the internation-al students in my programme come from a single South-East Asian country and their English proficiency is much lower than ours. I cannot see that I can learn any English from them.”

Deficient in CLIL teaching faculty: ‘Teacher and teaching quality’ is an-other major student doubt about the current CLIL programmes. A majority of the interviewees indicated their preference for native English speakers as CLIL teachers and complained about the local non-native English speak-ing teachers’ English accuracy, fluency and pronunciation, which would sometimes hinder understanding of the lectures. Although the differen-tiated CLIL classroom discourse between native and non-native speakers has been discussed in some research (e.g., Dafouz, Nunez, & Sancho, 2007; Llinares-García & Romero-Trillo, 2008), it is difficult to argue who, native


YANG

or non-native teachers, is more qualified to teach CLIL courses. One inter-viewee commented that “Taiwanese teachers have difficulties delivering the course in English but they know our learning difficulties and can offer help. However, if we don’t understand something, native teachers only explain it once again in English and we still cannot get it.” Besides, the interviewees argue that those who can speak fluent English are not necessarily quali-fied CLIL teachers because many of them have no training in how to teach English. Thus, the dual-focused programme would sometimes end with only one focus, i.e. content knowledge.

The timely intrusion of L1 and language feedback: As discussed in the preceding sections, learners’ L1 can play an important role in CLIL educa-tion (Lo, 2015) and the interviewees also expressed a similar opinion that L1 can be appropriately used to facilitate understanding. “Sometimes we need Chinese translation for difficult terminology and concepts in class; oth-erwise, we will be stuck there and cannot move on,” remarked one student. In addition, they argued that they expected some feedback on their lan-guage use so that they can improve it. However, feedback should be gen-eral, and not too specific or frequent. One learner said, “If teachers correct each of my mistakes or errors in English, I think I would refrain from using English next time. This would demotivate me.”

Progressive English receptive skills: Differing from their responses in the questionnaire survey, many of the interviewees agreed that their English proficiency had improved due to their CLIL education, in particular, their re-ceptive skills, i.e. vocabulary, listening, and reading, as evidenced in the lit-erature (e.g., Aguilar & Rodríguez, 2012; Dalton-Puffer, 2008). The materials students used in CLIL courses are authentic English language textbooks, and they are also expected to do a lot of reading after class; therefore, their read-ing speed and vocabulary size is expanded. However, they also hoped that an ESP course, teaching terminology and reading skills, would be offered before the CLIL course in order to shorten their time of adaptation. As one student remarked, “Teachers often use some professional terms in class but we cannot understand what they mean. So, we spend time looking up the meanings after class, but we actually already know these terms in Chinese. So, if teachers can provide us with Chinese translations of the terminology



in advance, this will reduce our wrong guessing and facilitate our content comprehension.”

Divided opinions about the effect of CLIL: Regarding the overall eval-uation of the CLIL approach, the respondents held a divided attitude. It was found that learners from a highly-recommended programme gener-ally liked CLIL and would recommend it for future students, while those from programmes that were not recommended held a relatively neg-ative attitude towards the effect of studying in a CLIL programme, and were hesitant to make recommendations. These responses are similar to the author’s previous findings in a case study (Yang & Gosling, 2013, 2014) that students’ satisfaction and the appraisal result of a CLIL programme are closely related. Furthermore, not all disciplines in universities are suit-able to be conducted with a CLIL approach. For instance, disciplines in the humanities or regarding local features may not be adequately taught us-ing the CLIL approach.

CONCLUSION AND IMPLICATIONS

Conclusion of the researchThis research investigated the effect of implementing CLIL education at the tertiary level in Taiwan. The research adopted a mixed-method re-search design to study the questions. The research instruments included a self-designed questionnaire and in-depth interviews with CLIL programme directors and learners. In total, 52 undergraduate and postgraduate CLIL learners island-wide were invited to join the questionnaire survey, while six programme directors and teachers and six focus groups of CLIL learn-ers joined the semi-structured interviews. In the descriptive analysis, the results show that the present CLIL learners apparently do not have very high agreement or satisfaction with the claimed benefits of their CLIL ed-ucation. The major cause may come from the imbalance in the teaching of the content and language areas. Usually, content teaching is highly stressed but linguistic elements are not, which makes the learners perceive no in-crement in their language proficiency, and thus they feel that the dual-fo-cuses of CLIL are not equally achieved. However, in the interviews with the


YANG

CLIL directors and teachers, they showed a very positive and welcoming attitude towards CLIL. They believed that CLIL brings many advantages to both universities and students, but they also expressed one major concern, that is, the preparation of qualified CLIL teachers for the popularly-growing programmes at the tertiary level island-wide. In contrast with the responses expressed in the questionnaire survey, the interviews with the CLIL learners showed a slightly different attitude. In general, they acknowledged the in-crement in linguistic skills, in particular the receptive skills, preferred na-tive English-speaking teachers, and developed intercultural awareness. In contrast to their less positive responses shown in the descriptive analysis, the learners expressed a more positive attitude towards the effect of CLIL education in the interviews.

Pedagogical implications

The present research also offers the following pedagogical implications. First, communication should be well built between programme providers and CLIL learners. The results indicate that there is a gap regarding the ef-fect and benefits of CLIL education between these two groups of stakehold-ers. The former hold positive attitudes towards CLIL while the latter do not. Any educational innovation should be mutually communicated, understood among all the stakeholders and then implemented smoothly. CLIL, a new educational approach, required the involvement of the authorities, univer-sities, programme directors, learners and their parents to make it a success. Hence, good communication with all the stakeholders about what CLIL is and how it differs from traditional approaches is indispensable. If learners per-ceive that they are placed in an educational experiment without knowing what CLIL is, or if they are told how great CLIL is and what they will achieve, they would show great dissatisfaction with CLIL if the claimed benefits are not sensed or realised.

Next, teachers are a vital factor affecting the success or failure of CLIL education. Both the learners’ and teachers’ opinions show that there is a lack of qualified CLIL teachers in Taiwan’s tertiary education sector. Most of the programme providers use the easiest way to staff the teaching faculty,



that is, CLIL teachers are either local content teachers with proficient En-glish or native English-speaking content teachers. Very few of them have a background in English language teaching (ELT). Hence, it is suggested that the programme providers offer potential CLIL teachers periodic training in CLIL education, in particular, the theories of ELT and how ELT can be well integrated with their content teaching. In addition, since the popularity of CLIL is growing dramatically in Taiwan’s universities island-wide, some ‘CLIL Centres’ (in Northern, Central, Southern and Eastern Taiwan) can be in-stituted to be in charge of training certified CLIL teachers, providing teach-ers with resources and support, conducting CLIL research and exchanging experiences with other neighbouring countries where CLIL is also rapidly gaining in popularity, such as Japan or Hong Kong.

Furthermore, since CLIL teaching is always dual-focused, CLIL as-sessment is supposed to be dual-focused in the classroom, too. Too often, CLIL practitioners over-emphasise the assessment of content knowledge while overlooking the assessment of learners’ linguistic performance. One major reason is that the content teachers may not have the appropriate knowledge of ELT assessment, and the assessment of CLIL is especially different and complicated (Massler, 2010), so assessing leaners’ linguistic performance seems purposefully avoided. Yet, CLIL learners hope to know not only if they have learned the content knowledge, but also how much their English proficiency has progressed as a result of their CLIL education. Indeed, CLIL assessment should accommodate both elements and again proper training on how to assess learners’ performance in both content knowledge and language competency is needed.

Lastly, appraising the effect of managing CLIL programmes should be multi-dimensional and contextualised. In the first nation-wide CLIL ap-praisal, the indicators mainly focused on the curriculum structure and ad-ministrative support; however, the teaching and learning performance in the CLIL context were not included as indicators. Hence, it is argued that in future appraisals, practitioners’ practice in the classroom and learners’ performance should also be evaluated, as these two dimensions reflect the core values of this innovative approach. Besides, the standards of CLIL appraisal could be amended in accordance with various CLIL settings. The


YANG

focuses of the appraisal should be differentiated, depending on the status of the programme providers, i.e. national vs. private universities and com-prehensive vs. polytechnic universities. Different programme providers have various purposes of setting up CLIL programmes, and the sources of learners are also diverse, with different entry levels of English proficien-cy. Catering to both contextual diversities and learners’ differences in ap-praisals can help avoid the criticism of CLIL being an elitist approach. After all, one suit cannot fit all.

Limitations and suggestions for future researchSince the present research may be one of the very few studies investigat-ing the implementation and effect of the CLIL approach at the tertiary level in Taiwan, some future studies can be done to complement it. First, inter-national students’ opinions can be examined. They are not all native En-glish speakers, and in fact most are also non-native English speakers like the present participants, and are mainly from Southeast Asian countries. How they see the actual effect of CLIL can provide another perspective on its implementation in Taiwan. Next, due to the small size of the current sample, and in order to generalise the findings, more participants should be invited to join future research. The CLIL learners from polytechnic uni-versities seemed to be more satisfied with CLIL education, so they may be able to provide better reasons for why CLIL has become popular and suc-cessful if more of them are included in a future study. Finally, to better un-derstand why there is reluctance about the adoption of CLIL, it is suggested that researchers investigate CLIL practitioners’ actual practices in the class-room. Classroom observations may unveil the real teaching performance and learning outcomes in class, which would help researchers clearly iden-tify how content and language teaching is executed in the classroom and what kinds of supports teachers and learners may need in order to imple-ment CLIL successfully.

ACKNOWLEDGEMENT

This paper is under the sponsorship of Ministry of Science and Technolo-gy, Taiwan (Grant number: 103-2410-H-328 -002 -).



REFERENCES

Airey, J. (2009). Estimating undergraduate bilingual scientific literacy in Sweden. International CLIL Research Journal, 1(2), 26-46.

Allan, G. (2010). CLIL: Teaching science and English to young learners in Japan. In S. Sheehan (Ed.), Teacher development and education in context: A selection of papers presented at IATEFL 2010 by the British Council (pp.53-57). British Council.

Alonso, e., Grisalena, J. & Campo, A. (2008). Plurilingual education in secondary schools: Analysis of results. International CLIL Research Journal, 1(1), 36-49.

Árva, V. & Medgyes, P. (2000). Native and non-native teachers in the classroom. System, 28(3), 355-372.

Aguilar, M. & Rodríguez, R. (2012). Lecturer and student perceptions on CLIL at a Spanish university. International Journal of Bilingual Education and Bilingualism, 15(2), 183-197.

Bentley, K. (2010). The TKT teaching knowledge test course: CLIL module. Cambridge: Cambridge University Press.

Burston, J. & Kyprianou, M. (2009, September). Integrating task-based and content and language integrated learning. Paper presented at the TBLT 2009: The 3rd Biennial Conference on Task-based Language Teaching. Lancaster University, UK.

Burton, A. (2011a). Are the differences between CLIL and non-CLIL groups in Andalusia due to CLIL? A reply to Lorenza, Casal and Moore (2010). Applied Linguistics, 32(2), 236-241.

Burton, A. (2011b). Is CLIL so beneficial, or just selective? Revisiting some of the research. System, 39, 523-532.

Carrio-Pastor, M. L. (2009). (Ed.). Content and language integrated learning: Cultural diversity. Bern, Switzerland: Peter Lang.

Chang, T. (2012). English Career, 41, 84-87.

Chen, C. M. (2012). English Career, 41, 10-15.


YANG

Coonan, C. (2007). Insider views of the CLIL class through teacher self-observation- introspection. The International Journal of Bilingual Education and Bilingualism, 10(5), 625-646.

Costa, F. & D’angelo, L. (2011). CLIL: A suit for all seasons? Latin American Journal of Content & Language Integrated learning, 4(1), 1-13.

Coyle, D. (1999). Theory and planning for effective classrooms: Supporting students in content and language integrated learning contexts. In J. Masih (Ed.), Learning through a foreign language. London: CLIL.

Coyle, D. (2005). CLIL: Panning tools for teachers. University of Nottingham. Retrieved 06 January, 2012, from http://blocs.xtec.cat/seaeangles/2009/03/16/clil-planning-tools-for-teachers-pilar-olivares-training-session/

Coyle, D. (2007a.). Research on content and language integrated learning (CLIL). The International Journal of Bilingual Education and Bilingualism, 10(5), 541-542.

Coyle, D. (2007b.). Content and language integrated learning: Towards a connected research agenda for CLIL pedagogies. The International Journal of Bilingual Education and Bilingualism, 10(5), 543-562.

Coyle, D. (2008). CLIL: A pedagogical approach from the European perspective. In Encyclopaedia of Language and Education (pp. 1200-1214). Springer US.

Coyle, D. (2009). Promoting cultural diversity through intercultural understanding: A case study of CLIL teacher professional development at in-service and pre-service levels. In M. L. Carrio-Pastor (Ed.), Content and language integrated learning: Cultural diversity (pp.105-124). Bern, Switzerland: Peter Lang.

Coyle, D. (2010). Foreward. In D. Lasagabaster & Y. Zarobe (Eds.), CLIL in Spain: Implementation, results and teacher training, (pp. vii-viii). Newcastle upon Tyne, UK: Cambridge Scholar Publishing.

Coyle, D., Hood, P. & Marsh, D. (2010). CLIL: Content and language integrated learning. Cambridge: Cambridge University Press.

Creswell, J. & Plano, C. V. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.

Dafouz, E., Nunez, B., & Sancho, C. (2007). Analysing stance in a CLIL university context: Non-native speaker use of personal pronouns



and modal verbs. The International Journal of Bilingual Education and Bilingualism, 10(5), 674-662.

Dalton-Puffer, C. (2007). Discourse in content and language integrated learning (CLIL) classrooms. Philadelphia, PA: John Benjamins.

Dalton-Puffer, C. & Smit, U. (2007). Empirical perspectives on CLIL classroom discourse. Bern, Switzerland: Peter Lang.

Dalton-Puffer, C. (2008). Outcomes and processes in content and language integrated learning (CLIL): Current research from Europe. In W. Delanoy & L. Volkmann (Eds.), Future perspectives for English Language Teaching, (pp. 139-157). Heidelberg: Carl Winter.

Dalton-Puffer, C. (2009). Communicative competence and the CLIL lesson. In Y. R. Zabore & R. M. Catalan (Eds.), Content and Language Integrated Learning: Evidence from research in Europe (pp. 197-214). Bristol, UK: Multilingual Matters.

Dalton-Puffer, C., Huttner, J., Schindelegger, V. & Smit, U. (2009). Technology-Geeks speak out: What students think about vocational CLIL. International CLIL Research Journal, 1(2), 18-25.

Dalton-Puffer, C. (2011). Content-and-language integrated learning: From practice to principles? Annual Review of Applied Linguistics, 31, 181-204.

European Commission. (2006). Teacher education for CLIL across contexts: From scaffolding framework to teacher portfolio for content and language integrated learning. Accessed 16 February, 2012 from http://clil.uni.lu/CLIL/Project_files/CLIL_Project_Final_Report.pdf

European Commission. (2010). European language policy and CLIL: A selection of EU-funded projects. Accessed 12 January, 2012 from http://europa.eu.int/comm/dgs/education_culture

Feng, A. (2010, May). Diversity of CLIL: Case studies across Greater China. Plenary speech presented at Innovation methods in multicultural education conference. Narva College of the University of Taru, Estonia.

Finkbeiner, C. & Fehling, S. (2006). Investigating the role of awareness and multiple perspectives in intercultural education. In P. Schmidt & C. Finkbeiner (Eds.), ABC’s of cultural understanding and communication: National and International Adaptations (pp. 93-110). Greenwich, CT: Information Age Publishing.


YANG

Gajo, L. (2007). Linguistic knowledge and subject knowledge: How does bilingualism contribute to subject development? The International Journal of Bilingual Education and Bilingualism, 10(5), 563-581.

Garrido, M. (2009). CLIL: A new approach to TEFL. Accessed 06 January, 2012, from http://www.lara25.com/mywebdisk/NA-TEFL/NA-assignment.pdf

Gefaell, C. & Unterberger, B. (2010). CLIL programme evaluation: Deriving implementation guidelines from stakeholder perspectives. Views, 19(3), 29-35.

Glaser, B. & Strauss, A. (1967). The discovery of ground theory. Chicago, IL: Aldine.

Graaff, R., Koopman, G., Anikina, Y. & Westhoff, G. (2007). An observation tool for effective L2 pedagogy in content and language integrated learning (CLIL). The International Journal of Bilingual Education and Bilingualism, 10(5), 603-624.

HEEACT. (2011). The plan of appraising CLIL programmes in 2011. Accessed 06 January, 2012, from http://www.heeact.edu.tw/public/Attachment/1526919367.doc

Hunt, M., Neofitou, A. & Redford, J. (2009). Developing CLIL training fro modern languages teacher trainees. In D. Marsh, P. Mehisto, D. Wolff, R. Aliaga, T. Asikainen, M. Friglos-Martin, S. Hughes & G. Langé (Eds.), CLIL practice: Perspectives from the field (pp. 110-116). CCN: University of Jyväskylä, Finland.

Hofmannova, M., & Novotna, J. & Pipalova, R. (2008). Assessment approaches to teaching mathematics in English as a foreign language (Czech experience). International CLIL Research Journal, 1(1), 20-35.

Hou, Z. X. (2007). An exploration of adopting CLIL to improve English teaching in non-language professionals in universities. Journal of Leshan Teachers College, 22(11),93-96.

Infante, D., Benvenuto, G. & Lastrucci, E. (2008). Integrating content and language at primary school in Italy: Ongoing experimental research. International CLIL Research Journal, 1(1), 74-82.

Jappinen, A. (2005). Thinking and content learning of mathematics and science as cognitional development in content and language



integrated learning (CLIL): Teaching through a foreign language in Finland. Language and Education, 19(2), 147-168.

Jendrych, E. & Wisniewska H. (2010, November). ESP: How to design challenging tasks for adult learners. Paper published at ICT for Language Learning Conference Proceedings 2010. Accessed 12 January, 2012, from http://www.pixel-online.net/ICT4LL2010/common/download/Proceedings_pdf/CLIL01-Jendrych,Wisniewska.pdf

Kobayashi, Y. (2002). The role of gender in foreign language learning attitudes: Japanese female students’ attitudes towards English learning. Gender and Education, 14(2), 181-197.

Lasagabaster, D. (2011). English achievement and student motivation in CLIL and EFL settings. Innovation in Language Learning and Teaching, 5(1), 3-18.

Lasagabaster, D. & Sierra, J. (2009a.). Immersion and CLIL in English: More differences than similarities. ELT Journal, 64(4), 367-375.

Lasagabaster, D. & Sierra, J. (2009b.). Language attitude in CLIL and traditional EFL classes. International CLIL Research Journal, 1(2), 4-17.

Lasagabaster, D. & Sierra, J. (2010). Immersion and CLIL in English: More differences than similarities. ELT Journal, 64(4), 367-375.

Lee, B. C. & Chang, K. S. (2008). An overview of content language integrated learning in Asian contexts. Studies in English Education, 13(2), 166-184.

Llinares-García, A. & Romero-Trillo, J. (2008). Discourse markers and the pragmatics of native and non-native teachers in a CLIL corpus. Pragmatics and corpus linguistics: A mutualistic entente, 2, 191.

Lo, C. R. (2007). Action research on investigating the probability of implementing CLIL onto English education in primary schools. Unpublished mater dissertation. Taipei, National Taipei University of Education.

Lo, Y. Y. (2015). How much L1 is too much? Teachers’ language use in response to students’ abilities and classroom interaction in Content and Language Integrated Learning. International Journal of Bilingual Education and Bilingualism, 18(3), 270-288.

Loranc-Paszylk, B. (2007). Enhancing the effectiveness of CLIL through writing. A review of the second-term programme of the European


YANG

Council for Modern Languages and a summary of the ECML Conference 2007 (pp. 125-131). Council of Europe Publishing.

Loranc-Paszylk, B. (2009). Integrating reading and writing into the context of CLIL classroom: Some practical solutions. International CLIL Research Journal, 1(2), 47-53.

Lorenzo, F., Casal, S. & Moore, P. (2010). The effects of Content and Language Integrated Learning in European education: Key findings from the Andalusian bilingual sections evaluation project. Applied Linguistics, 31, 391-417.

Lorenzo, F., Moore, P. & Casal, S. (2011). On complexity in bilingual complexity research: The causes, effects, and breadth of Content and Language Integrated Learning- a reply to Bruton (2011). Applied Linguistics, 32(4), 450-455.

Lucitto, S. (2008). A model for quality CLIL provision. International CLIL Research Journal, 1(1), 83-92.

Lucietto, S. & Rasom, O. (2011). Consultancy work in an innovative plurilingual CLIL programme: Context, relationships and evaluation. Procedia: Social and Behavioural Sciences, 30, 1438-1446.

Luo, Q. L. (2006). CLIL and its implications for China’s EFL teaching. Journal of Beijing University of Aeronautics and Astronautics, 19(3), 70-72.

Mackenzie, A. S. (2008). English next in easy Asia. In British Council (Ed.), The proceeding of primary innovations regional seminar (pp. 23-30). Bangkok.

Maggi, F. (2011). Assessment and evaluation in CLIL. ICT for Language Learning Proceedings 2011. Retrieved 04 March, 2012, from http://www.pixel-online.net/ICT4LL2011/common/download/Paper_pdf/CLIL03-193-FP-Maggi-ICT4LL2011.pdf

Marenzi, I., Kupetz, R., Nejdl, W. & Zerr, S. (2010). Supporting active learning in CLIL through collaborative search. In X. Luo, M. Spaniol, L. Wang, Q. Li, W. Nejdl & T. Wu (Eds.), Advances in Web-Based Learning – ICWL 2010 (pp. 200-209). Berlin: Springer Verlag.

Marsh, D. & Hood, P. (2008). Content and language integrated learning in primary East Asia contexts (CLIL PEAC). In British Council (Ed.), The proceeding of primary innovations regional seminar (pp. 43-50). Bangkok.



Marsh, H. W., Hau, K. T. & Kong, C. K. (2000). Late immersion and language of instruction in Hong Kong high schools: Achievement growth in language and non-language subjects. Harvard Educational Review, 70, 302-346.

Massler, U. (2010). Assessment in CLIL learning. Guidelines for CLIL Implementation in Primary and Pre-primary Education, 115-137.

Medgyes, P. (1992). Native or non-native: who’s worth more?. ELT journal, 46(4), 340-349.

Mehisto, P. (2008). CLIL counterweights: Recognising and decreasing disjuncture in CLIL. International CLIL Research Journal, 1(1), 93-119.

Mehisto, P. & Asser, H. (2007). Stakeholder perspectives: CLIL programme management in Estonia. The International Journal of Bilingual Education and Bilingualism, 10(5), 683-701.

MOE. (2011). The results and reports of appraising the CLIL programmes in National Kaohsiung University of Hospitality and Tourism. [Document No.: 1000143344A].

Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40, 120-123.

Morse, J. M. (2003). Principles of mixed methods and multimethod research design. In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioural research (pp. 189-208). Thousand Oaks, CA: Sage.

Novotna, J. & Hofmannova, M. (2000). CLIL and mathematics education. Retrieved 06 January, 2012, from http://math.unipa.it/~grim/Jnovotna.PDF

Novotna, J., Hadj-Moussova, Z. & Hofmannova, M. (2001). Teacher training for CLIL- Competences of a CLIL teacher. Accessed 12 January, 2012, from http://people.fjfi.cvut.cz/novotant/jarmila.novotna/Novotna-et-alSEMT01.pdf

Perez-Canado, M. L. (2011). CLIL research in Europe: Past, present, and future. International Journal of Bilingual Education and Bilingualism. DOI:10.1080/13670050.2011.630064

Sasajima, S., Ikeda, M., Hemme, C. & Reiley, T. (2011). Current practice and future perspectives of content and language integrated learning


YANG

(CLIL) in Japan. The JACET 50th (2011) Commemorative International Convention Proceedings. Tokyo: JACET.

Scheneider, K. & McCollum, S. (1991). It’s academic: An integrated skills content-based approach to language learning. Ontario, Canada: Maxwell Macmillan International.

Seikkula-Leino, J. (2007). CLIL learning: Achievement levels and affective factors. Language and Education, 21(4), 328-341.

Serra, C. (2007). Assessing CLIL at primary school: A longitudinal study. The International Journal of Bilingual Education and Bilingualism, 10(5), 582-602.

University of Cambridge ESOL Examinations (UCLES). (2009). TKT-CLIL Glossary. University of Cambridge. Accessed 12 January, 2012 from http://cambridgeesol.org/assets/pdf/resources/teacher/clil_glossary.pdf

Vazquez, G. (2007). Models of CLIL: An evaluation of its status drawing on the German experience. A critical report on the limits of reality and perspectives. Revista española de lingüística aplicada (Vol. Extra 1), 95-112.

Wannagat, U. (2007). Learning through L2-Content and language integrated learning (CLIL) and English as medium of instruction (EMI). The International Journal of Bilingual Education and Bilingualism, 10(5), 663-682.

Wijayadharmadasa, S. (2011). Exploring the implementation of Content and Language Integrated Learning (CLIL) in English for Medicine programmes of a Sri Lankan University. Proceedings of Annual Research Symposium (pp. 40-42). University of Colombo: Sri Lanka.

Wolff, D. (2010). Content and Language Integrated Learning: An evaluation of the German approach. Retrieved 04 March, 2012, from http://share.dschola.it/dd4pinerolo/clil/Shared%20Documents/Theory_strategies/Dieter%20Wolff.pdf

Yang, W. H. (2003). An interpretative analysis of teachers’ perceptions of educational changes. Unpublished doctoral thesis. Exeter University, Exeter, UK.

Yang, W. H. (2014a). Content and Language Integrated Learning next in Asia: Evidence of learners’ achievement in CLIL education from



a Taiwan tertiary degree-programme. International Journal of Bilingual Education and Bilingualism, 18(4), 361-382.

Yang, W. H. (2014b). Higher education perspectives in CLIL appraisals. Ministry of Science and Technology, Taiwan. (NSC 103-2410-H-328-002).

Yang, W. H. (2015). Tuning university undergraduates for high mobility and employability under the Content and Language Integrated Learning approach. International Journal of Bilingual Education and Bilingualism, (ahead-of-print), 1-18.

Yang, W. H. (2016 a). CLIL vs. ESP: Two sides of the same coin or two extremes of a continuum? ESP Today, 4(1), 43-68.

Yang, W. H. (2016 b). The dominance of indirect language learning strategies in tertiary Content and Language Integrated Learning (CLIL) education: Natural evolution or involuntary decision? Paper under review.

Yang, W. H., and Gosling, M. (2013). National appraisal and stakeholder perceptions of a tertiary CLIL programme in Taiwan. International CLIL Research Journal, 2(1), 67-81.

Yang, W. H. and Gosling, M. (2014). What makes a Taiwan CLIL programme highly recommended or not recommended? International Journal of Bilingual Education and Bilingualism, 17(4), 394-409.

Yasain, M. D., Marsh, D., Ong, E. T., & Lai, Y. Y. (2009). Learners’ perceptions towards the teaching of science through English in Malaysia: A quantitative analysis. International CLIL Research Journal, 1(2), 54-68.

Zarobe, Y. R. (2008). CLIL and foreign language learning: A longitudinal study in the Basque country. International CLIL Research Journal, 1(1), 60-72.

Zarobe, Y. R., & Catalan, R. M. J. (2009). Content and language integrated learning: Evidence from research in Europe. Tonawanda, NY: Multilingual Matters.


YANG

APPENDIx 1: THE ENGLISH VERSION QUESTIONNAIRE

Students’ Perceptions and Attitudes towards Content and Language Inte-grated LearningCLIL: A professional content course taught in English

A Background information1. Gender □Female □Male2. Degree □B.A. □M.A3. Nationality □Taiwanese □International 4. Grade □1st year □2nd year □3rd year □ 4th year5. High school major □Related □Languages □Normal6. English proficiency (TOEIC) □<500 □500-750 □>7507. Duration of learning English □1-6 yrs □7-12 yrs □13 yrs

B After studying this CLIL programme, to what extent do you agree the fol-lowing statements?(5: strongly agree ------------------------------------------- 1: strongly disagree)1. My writing proficiency is increased. □5 □4 □3 □2 □12. My reading proficiency is increased. □5 □4 □3 □2 □13. My speaking proficiency is increased. □5 □4 □3 □2 □14. My listening proficiency is increased. □5 □4 □3 □2 □15. Overall English proficiency is increased. □5 □4 □3 □2 □16. I have no difficulty in reading articles. □5 □4 □3 □2 □17. I have no difficulty in writing my content knowledge. □5 □4 □3 □2 □18. I have no difficulty in orally expressing my content knowledge. □5 □4 □3 □2 □19. Generally, I can understand most CLIL courses. □5 □4 □3 □2 □110. Teachers use different methods in instructing CLIL courses. □5 □4 □3 □2 □111. I prefer native English-speaking teachers to teach CLIL courses. □5 □4 □3 □2 □112. I prefer Taiwanese teachers to teach CLIL courses. □5 □4 □3 □2 □113. I chose this programme because I am interested in CLIL. □5 □4 □3 □2 □1



14. My parents persuaded me to study CLIL programme. □5 □4 □3 □2 □115. English is used more than 60% in CLIL courses. □5 □4 □3 □2 □116. I am satisfied with teacher’s assessment in CLIL courses. □5 □4 □3 □2 □117. I prefer teacher’s lectures in CLIL courses. □5 □4 □3 □2 □118. I prefer student’s group discussion in CLIL courses. □5 □4 □3 □2 □119. I prefer student’s individual oral presentation in CLIL courses. □5 □4 □3 □2 □120. I perceive native English-speaking teachers to be more capable of teaching CLIL courses. □5 □4 □3 □2 □121. I perceive Taiwanese teachers to be more capable of teaching CLIL courses. □5 □4 □3 □2 □122. I show high involvements in CLIL courses. □5 □4 □3 □2 □123. I need no extra English courses to increase my English ability □5 □4 □3 □2 □124. I need extra English courses to increase my English ability. □5 □4 □3 □2 □125. CLIL programme increases my motivation of learning English. □5 □4 □3 □2 □126. I prefer teachers to use Chinese in teaching CLIL courses. □5 □4 □3 □2 □127. Taiwanese teachers have difficulty in teaching CLIL. □5 □4 □3 □2 □128. CLIL programme helps me think in English. □5 □4 □3 □2 □129. I tend to convert Chinese into English in CLIL courses. □5 □4 □3 □2 □130. Teachers significantly use different methods in teaching CLIL courses. □5 □4 □3 □2 □131. CLIL programme can increase my motivation of learning content subjects. □5 □4 □3 □2 □132. Teachers need to use Chinese timely to teach CLIL courses whenever needed. □5 □4 □3 □2 □133. CLIL programme is beneficial to my future job. □5 □4 □3 □2 □134. I prefer teachers to provide me immediate linguistic feedback on my language errors. □5 □4 □3 □2 □135. Teachers intentionally use simplified English to teach CLIL courses. □5 □4 □3 □2 □136. My family believe CLIL programme is beneficial to my future job. □5 □4 □3 □2 □1


YANG

37. Generally, I like this CLIL programme. □5 □4 □3 □2 □138. I would recommend this CLIL programme to others. □5 □4 □3 □2 □139. Please write down what you have gained or lost in studying this CLIL pro-

gramme. ___________________________________________________________________40. Please write down any comments about this CLIL programme. ___________________________________________________________________Thank you for your precious time and generous contribution!

Source: from Yang & Gosling (2013).