Posts in methods

Directionality research in translation and interpreting studies: A shorter than short history (1/2)

agosto 26th, 2013 Posted by blog, directionality, interpreting, inverse translation, methods, mother tongue, native speaker, translation No Comment yet


Most generally speaking, directionality research is concerned with the direction from and into which a translation or interpretation is carried out, and related concepts. In a way, the translation or interpreting direction is present in any translation or interpreting research, since a translation or an interpretation can always be construed to have been done from a specific written, spoken or signed language into another. Building on an old debate in the profession and anecdotal comments on the issue, which go back at least as far as Pliny the Younger, who in 85 CE advocated translating from Ancient Greek into Latin and vice versa (Robinson 2002), a specific field of translation and interpreting studies (TIS) has developed to investigate directionality.

Directionality-related questions that have been or could be addressed by translation and interpreting researchers include: What are the roots of concepts central to directionality and so-called inverse translation such as ‘(non-)native’ speaker or signer, ‘(non-)mother tongue’ or ‘foreign’ language? How similar/dissimilar are attitudes and norms towards translation or interpreting direction in the various ‘translation cultures’ (Prunč 1997, 2012; Schippel 2008, Grbić et al. 2010)? How homogeneous are the groups of ‘native’ and ‘non-native’ speakers or signers? How do ‘native’ and ‘non-native’ speakers or signers feel about being explicitly or implicitly classified as such? With regard to directionality effects, how similar/dissimilar are specific language pairs and genres/text types? What is the performance of a translator or interpreter out of his/her ‘mother tongue’ as opposed to into his/her ‘mother tongue’? What are the effects when multiple collaborators with different ‘mother tongues’ (such as translators, revisors or validators) are involved in the translation process? How does translation or interpreting direction affect reception? How useful is it to use directionality as an organizing principle of translation/interpreting courses or entire study programs? How much is the ‘nativeness’ factor stressed in job advertisements, how relevant is it to actual hiring practices, and what reasons are given for that?
Now let me give you a brief overview of the research that has been published on translation and interpreting directionality over the years, and provide you with some context. At first, a few scattered, pioneering studies appeared, whose focus was mostly directionality in translation (e.g., McAlester 1992, 2000; Beeby Lonsdale 1996, Marmaridou 1996, Campbell 1998, Stewart 1999, 2000a+b, 2008; Kocijančič Pokorn 2000a+b, Lorenzo 2002, 2003). These studies were followed by a growing number of publications dealing with directionality in interpreting (e.g., Tommola/Helevä 1998, Al-Salman/Al-Khanji 2002, Lim 2005, Monti et al. 2005, Bartłomiejczyk 2006, Chang/Schallert 2007, Bendazzoli 2010, Opdenhoff 2011). In the noughties, when directionality research seems to have enjoyed its heyday in TIS, directionality-dedicated conference proceedings and a thematic special issue were published (Grosman et al. 2000, Kelly et al. 2003, Godijns/Hinderdael 2005). A fair number of works produced during that period approached directionality from an emancipatory perspective, which is attested to by titles such as Challenging the Traditional Axioms: Translation Into a Non-Mother Tongue (Pokorn 2005) or Into Forbidden Territory: The Audacity to Translate into a Second Language (Feltrin-Morris 2008).

Most recent contributions to directionality research have been made in translation process research (e.g., Hirci 2007, Alves et al. 2009, Pavlović/Jensen 2009, Maier 2011, Chang 2011, Wimmer 2011, Ferreira Alves 2010, 2012; Rodríguez/Schnell 2012, Ferreira 2014, Barbosa de Lima Fonseca 2015, Hunziker Heeb 2016, Ferreira/Schwieter 2017). Because of the nature and research designs of process studies in TIS, this does not come as a surprise. In the often quantitative studies typical of this TIS research tradition, directionality or related concepts may appear as explicitly spelled-out, highly visible dependent, independent or control variables (Krings 2005 mentions translation direction as one of his “task factors”).

Directionality has now been established as an important issue in TIS, and received its own entry in major reference works (Shuttleworth/Cowie 1997, Delisle et al. 1999, Beeby 2009, Palumbo 2009, Pokorn 2011, Bartłomiejczyk 2015).

TIS researchers will keep exploring the topic of directionality, also from new angles; directionality has recently become an issue in (improving) statistical machine translation and sign language interpreting (van Dijk et al. 2011, Wang/Napier 2013, Nicodemus/Emmorey 2013, 2015; Wang/Napier 2015, Wang 2016) and third language interpreting (Crasborn/van Dijken 2009, Topolovec 2012), for example. Recent conference (conference 1, conference 2), meeting (meeting 1, meeting 2) and workshop contributions, including the occasional keynote, a special journal section, a special issue on English as a lingua franca and translator/interpreter education including reflections on directionality, observations on directionality in so-called non-Western ‘translation cultures’, a survey reporting on the perception and role of translation direction in the Spanish technical translation market, and ongoing (post-)doctoral research projects are signs that the patient is alive and kicking.

In the rest of the post I would like to turn to conceptual and methodological issues. Taking a cue from other disciplines that have language(s) and communication as their object of study (e.g., Paikeday 1985, Piller 2001, 2002; Davies 2003, 2013; Bonfiglio 2013, Hulstijn 2015), we might want to be more careful when defining and operationalizing key concepts in directionality research. For instance, the criteria for assigning study participants to the ‘native’ or ‘non-native’ group are not always transparent. The linguist Tove Skutnabb-Kangas mentions origin, function, competence and/or identification as criteria we could rely on to determine a person’s mother tongue(s). To avoid threats to our studies’ credibility, we should thus perhaps be more cautious about ‘I know one when I see one’.

In the second and final part of this two-part blog post, I am going to talk about methodology in directionality research and will suggest methodological improvements (such as blinding) to avoid biased results.

  • Feltrin-Morris, Marella. 2008. Into forbidden territory. The audacity to translate into a second language. PhD thesis, Binghamton University/SUNY.


by Matthias Apfelthaler

TREC: The Next Generation

junio 10th, 2013 Posted by blog, empirical research, methods, PhD project, TransWorld Airy Lines No Comment yet

The international research network “Translation Research Empiricism Cognition” (TREC) convened in Barcelona for a regular meeting on July 4-5 2013. This time we also held a previous seminar 

on empirical and experimental research in  translation, where PhD students presented their ongoing work. These are, in alphabetical order, some of the stars of TREC’s next generation:
José Jorge Amigo Extremera (PETRA, ULPGC) talked about “Fitting culture into Translation Process Research”, where he summarized his project to develop operationalizations of culture and knowledge for empirical and experimental research, drawing form social and situated cognition approaches.
Mariceli Aquino (LETRA, UFMG) presented “A relevance-theoretic study of processing effort in post-editing tasks: an analysis of German modal particles  “. She will be using Translog and Tobii T60 to study post-editions of the MT output of text excerpts from a corpus of articles from Deutsche Welle.
Claudine Borg (Aston University) is working on an in-depth case study of post-drafting self-revision of the translation a novel from French into Maltese through think-aloud, translator observation, interviews, analysis of drafts and ST-TT comparison drawing on corpus-based techniques.
Luis Miguel Castillo (PACTE, UAB) contributed with “Acceptability and the acquisition of translation competence: preliminary results”, where he described his goal of tracing the evolution of translation quality throughout the process of the acquisition of translation competence.
Norma Fonseca (LETRA, UFMG) draws from Krings (2001) to distinguish temporal, technical and cognitive aspects of effortful processing during task execution and builds on Alves & Gonçalves (2013) to study cognitive effort during monolingual post-editing processes using key logging, screen recordings, and guided written protocols.
Andrea Hunziker Heeb (ZHAW) struck a vital chord by focusing on ethical issues that may arise with professional translators as research participants. She used a general academic self-evaluation checklist and a code of good practice in research to frame her presentation, which fostered a lively discussion.

Andrea Hunziker Heeb and Annina Meyer (ZHAW) presented the design, the methods and the hypotheses of a research project focused on ergonomic issues associated with software settings, equipment, and/or physical conditions that might impede the  efficiency of translation by slowing down decision-making and other cognitive processes during translation.

Arlene Koglin (LETRA, UFMG)  presented her project “Processing effort and cognitive effects trade-off in metaphor post-editing.“ Arlene is using eye tracking, key logging and retrospective protocols to gather data, and Relevance Theory as a referential framework.
Minna Kumpulainen (University of Eastern Finland) presented an overview of the use of pauses as potential cognitive indicators in translation process research, where she centered on pause length and their correlation with process segment boundaries.
Gisela Massana Roselló (PACTE, UAB) presented the design and some methodological issues of her research project on the acquisition of translation competence in trainees who have Portuguese as their second foreign language. Language typological proximity between Portuguese and Spanish is a major concern in this project.
Christopher Mellinger (KETRA, Kent State University) is well advanced in his PhD research  project on how cognitive effort is distributed during the translation task, which he is analyzing through the pause contour of  applied cognitive effort when using a translation memory to translate. He presented some preliminary findings on how the use of TMs and specific fuzzy match features affect the translation process in Spanish-to-English translation professionals with 4–7 years of experience.
Ana Muñoz Miquel (GENTT, UJI) presented her ongoing work on medical translator’s profiles, where she combines the cognitive notion of translators’ competence with a sociological survey of medical translators self-image and a pedagogical perspective on the needs of medical translator trainees.
Christian Olalla Soler (PACTE, UAB)  will be using screen recording, translations and questionanaires to study the acquisition of translators’ cultural  competence by Spanish trainee students with German as their second foregin language, from the perspective of PACTE’s (2003) translation competence model.
Raphael Sannholm (Stockholm University) presented the results of his MA thesis, which checked whether  different text types give rise to different foci in the cognitive processes during translation within a fairly homogenous group of participants, and also outlined his future PhD project on on automaticity in the cognitive processes in translation.
Karina Szpak’s (LETRA, UFMG) research project applies the relevance-theoretical concepts of conceptually and procedurally encoded information to study eye fixations, time spent, and attention units to identify instances of processing effort in translation.

The times, they are a-changin’ (3/3)

enero 2nd, 2012 Posted by aside to camera, blog, empirical research, methods, reliability No Comment yet

In the first post of this threefold series, I praised e-journals and suggested that specialization might lead tooverall improvements in journal quality. Let us welcome TC3, a promising addition to the short list of specialized journals within Translation and Interpreting Studies. In the second one, I sketched some advantages of a few digital resources, and of using on-line tools to build a community of practice. Here I would like to address what I think amounts to a precondition for the success of any attempt of building a community of practice, namely setting high and common standards. 

I will go straight to the point: I would like to argue that combining reproducible research with open access may probably be the best option to bring about these standards.

Now you see it, now you don’t

You try to repeat a fascinating piece of research reported in a journal article. You follow it step by step, veeeery carefully, only soon to realize that a lot of missing bits make it unlikely that you are actually replicating the original test by any rigorous accounts, and settle for approximation instead of comparability. Your results are different, but you cannot really hypothesize why. —A colleague asks you for the way you carried out certain details in an old research project. It takes you long to find the appropriate materials, and even longer to reconstruct some of the rationals behind decisions that were obviously taken but perhaps never stated as explicitly as it seems to have become appropriate now. Yeap, sometimes some of us cannot even make sense of our own old materials.

Yet another typical case. You would like to find support for a given hypothesis and the setting you establish to put it to the test actually fits quite well in that of a previous piece of research, so that their data might really be of use to you. But you will never know because many of the conditions were implicit in the published paper, or had not been recorded. You may become unnecessarily stuck with a reduced number of subjects. Testing your new hypothesis in their old materials might have also been a sound previous step to carry out your planned research project, but you can’t because it seems very difficult to get ahold of these materials. You cannot even challenge the interpretation of data that was offered then by reanalyzing data now from a different perspective on the same issue, on the same materials, because they are not available.

These are only some of the problems we have to deal with in our daily research efforts to study translators’ and interpreters’ cognition. Process research methods have not yet been standardized enough, and the information structure and contents of research reports are still lacking field-wide guidelines (where field allows for several readings). In PETRA we are trying to contribute to method standardization by looking at ways to profile texts and subjects. But perhaps starting the other way round, i.e. setting standards for research reporting in future efforts, might simply be a more practical way to reach both goals. That is, setting report standards might prompt agreements on some methodological standards.

A simple strategy to draft tentative report guidelines might be letting information standards emerge from the demands of the readers of research reports. A fuzzy network of researchers who mutually recognize each other as pertaining to the field of translation and interpreting process research might, after a while, reach agreements on initial, tentative report and methodological standards by exchanging information of their works. They might simply do so by spotting new, additional information demands when trying to re-use the information to reach their own goals, until these information needs are streamlined and settled. In any case, it should be an additive and realistic process of study of past projects to be applied only to new ones.

Reproducible research

A publication is not the research project itself, but merely a report of that piece of research. That is why one single research project may lead to many publications. Current information and communication technologies allow for the exchange of all research-related information in digital formats of any kind (first post, again). Today’s digital scholarship comprises the publications plus the complete environment and the full set of instructions applied to carry out the project. Simply because we can? I don’t think so. Rather, we will benefit from accessing the whole lot. For instance, providing additional materials might enlarge the possibilities of other researchers to suggest more solid observations by re-using somebody else’s tools (e.g. questionnaires) and even some data, when possible.

Replicating research projects often yields different results in all fields, not just in ours. Statistics became a discipline to come to the rescue in such scenarios. However, often many differences in replicated results may be ascribed to lack of information in the research report. As I wrote above, digital media let us share a wealth of information which would make our job easier by letting us re-use each others’ data. The information should include—as many, when not in full—actual materials whenever possible, in order to make sure that future approaches will be able to analyze the real data, and not only filtered reports, which would become just one part of the information provided. In other words, publications would become points of access to further information on a research project, including raw materials, which altogether are seen as a compendium. A reproducible-research compendium for translation and interpreting process research might include

1. The research paper

  • Full text (e.g. a PDF)
  • Full bibliographic citation with current publication status (e.g. a BibTeX file)
  • Supporting bibliography (with abstracts, when possible)

2. The experimental setting

  • Explanatory documentations for each factor, if not standardized
  • A list of the parameters, settings, and platforms under which the project was carried out that lead to the published (also unpublished?) results
  • Original texts as presented to subjects
  • Subjects profiles [and those of other parties, such as evaluators, when applicable]

3. The Data

  • Raw output data (log files, video recordings, user activity data, questionnaires, tests, etc.)
  • Criteria used for data cleaning
  • Notes on data cleaning and preparation process
  • Prepared data (files after cleaning)
  • Explanatory documentations for each part of the process of manipulating data

4. Results

  • All the results including high resolution figures, complete tables, full statistical analyses when applicable, etc.
  • Explanatory documentations for each part of the analysis.
Such a compendium might raise concerns about data privacy and the release of potentially confidential information. This may simply be sorted out by choosing a combination of the toughest standards. For instance, we might request for proof that papers in compendiums have cleared their status regarding their publishers’ policies for self-archiving. At the same time, we definitely need to look for ways to record information that will guarantee anonymity while providing a maximum of data with the consent of the participants. At PETRA we are now exploring the possibilities of on-line data collection through a website (+ computer application) to let subjects participate from their usual working environments. Apart from our main goals, we are also interested in finding out whether there are any effects of going distant and on-line on the behavior of participants in general and on ecological validity, in particular. This includes the availability of participants, their willingness to take part in the project, and other factors such that might be affected by the research situation.

Establishing standards for a compendium such as the one sketched above is probably still far too ambitious a goal, but that’s what goals are for. Several items in the checklist above need further elaboration. Variability is not easy to reduce when studying such complex tasks as those carried out by translators and interpreters. Clarifying the contents and extent of such items is as good a starting point as any to advance to standardization. In any case, we need to set the tone and fine-tune our efforts to jointly perform the score. Staying committed to just the information provided by papers, books and book chapters is like trying to stick to your old vinyl records. We all love vinyl records, but just try to carve out a ringtone for your phone from your old 33 rpm’s or to stream them over from your stereo into your computer or your tv. Just find that song by what’s-his-face-again (musical categories have really fuzzy edges) or choose to listen music from twenty different records in a row without having to attend the device after each song. In translation and interpreting process research, we seem to be sticking to scholarly vinyls. However, project documentation initiatives such as TransComp are steps in the right direction.

Open access

Once an initial standard has been established, the next step is broader dissemination. Open access offers several advantages. From a technical scope, it allows for comparison, fosters rigor in both methods and reports, and may ease junior researchers into replicating projects. If the how-tos are completely available, repeating a research project is a suitable and manageable goal for MA candidates. A wider scope on the advantages of open access may also take into account the benefits of extending the community of practice to new members and promoting good practices at once. Gatekeeping should become less tricky when more supported by both evidence and general agreement. Also, compendiums might make science more transparent to the public, and accountability is one of the sources of legitimacy for scientific endeavors.

New research projects offering compendiums could be made available in one or several dedicated website repositories, only when abiding to standards or properly challenging them. New candidates to enlarge, reduce or modify the standards for research reports would need to prove the relevance of their proposal for new or established research goals and also their fit and interaction with the rest of variables already in the standard. Successive versions of this standard would accommodate new information demands. In other disciplines there are frequent splits in standards for competing frameworks, so we may expect standards to vary in different types of projects. That's ok. Probably, new communication venues might specifically target standard-abiding research reports.

There may be other strategies, but reproducible research and open access seem both to be taking roots in neighboring disciplines, including linguistics, reading research, writing research, cognitive science, and psychology, so perhaps it is worth to give them a try, ‘cause los tiempos están kambiando.
by R. Muñoz

Training on Research

junio 6th, 2011 Posted by blog, empirical research, methods, training, TransWorld Airy Lines No Comment yet

Ph.D Course in Translation Processes Research15 – 19 August 2011

Theoretical aspects of process research; experimental research design and methodology; data visualization and human translation process modeling; qualitative and quantitative data analysis; user interaction with language technological tools.

CBS Center for Research & Innovation in Translation & Translation Technology

Registration fee for PhD students: 190 €
Registration fee for university researchers: 250 €
Reduced registration fee for the immediately ensuing NLPCS workshop: 110 €.
Registration deadline: 15 July 2011 at noon.
Support requests deadline: 15 June 2011.
Writing Process Research 2011: Keystroke Logging and Eye Tracking

7-9 September 2011

Possibilities and limitations of keystroke logging and eye tracking; good practices for ethnographical and experimental writing process research; complementary nature of observation methods; writing process data exploration; data preparation for further analysis; statistical analysis of writing process data; networking.

University of Antwerp Training school
20 trainees max.
Registration fee 125€
Registration deadline:7 July 2011
Confirmation of participation & grants: 1 August 2011

Writing an exposé

abril 26th, 2011 Posted by aims, blog, exposé, hypotheses, methods, PhD project, state of the art, The sorcerer's apprentice No Comment yet

Once you have found your doctoral advisor, you can start writing the exposé, i.e., a summary of your first thoughts on the PhD project. Depending on the country or even the university you are in, you may need to present it and have it approved by a committee, or else you may want to send it to some important scholar to convince her to become your supervisor. In any case, it will

be very useful to you, to clarify your ideas. You should always bear in mind that what you are going to put down in your exposé are just preliminary notions. Also, before you get into this jungle, there is something you should be aware of:
There is no explicit model for an exposé
[sudden drum banging in the dark]
Of course, that does not mean there is no structure at all. There is simply considerable variation, depending on the norms, traditions, etc., of the place you are in. In Europe, you may ask your advisor or university department for detailed information, but do not expect there will be something like the great Writing Lab at Purdue. At least, not in every university (but see Toronto’s and Victoria’s, though). Do not give up hope if you are given no information or just a couple of vague hints on this topic. Take it as your first quest for the holy grail.
Although there are no black-on-white rules on the contents and structure of an exposé, at least there seems to be some kind of consensus regarding the need to include at least the following sections:
project topic
state of the art
Other points, such as a sketch of the dissertation structure, work plan, and so on, is usually optional, or a specific requirement of your institution. Let us have a closer look on these points.

Project topic
You need a concise working title for your project. If somewhat obscure, you may explain the meaning of the title in your introduction. To avoid extra work, it is always a good idea to draft the introduction at the end, so you already know which information you need to summarize.

State of the art
Here you should concentrate on the following questions:
Which related topics have been investigated until now?
Which points or aspects have not been researched (enough) and are therefore desiderata?

Aim(s) - The question is… Why?
In this section you will spell out your project aims, the reasons to embark on such a project, what you expect from it. Generally, scientific aims tend to focus on verifying or falsifying hypotheses or they attempt to find some answers to clearly stated problems. You could consider other possible aims regarding your project (e.g., practical or didactic ones).

Hypotheses/Problems – The question is… What?
Since you are aiming to find answers to your stated hypothesis or problems, you need to make them clear from the beginning by explaining what you have found out on the topic and what led you to your conclusions.

Method(s) – The question is… How?
Your aims and objects of study define the methods applied. There are mainly two scientific methods of which one would be applied to your project:

You conclude general theoretical rules on the basis of empirical observations. For example, translation experiment results show that there are differences in problem-solving between professional and non-professional translators; now you would have to establish some rules, theories or hypotheses on these observations.

You would propose a theory or hypothesis first and then investigate it empirically, e.g., you would state that there are differences in problem-solving between professional and non-professional translators and then verify or falsify this statement based on the results of an empirical experiment.
If you want to do empirical research, you should specify your materials. In cognitive translatology, this would concern, for example, subjects, source text(s), experimental environment, data collection tools and methods (e.g. key logging, questionnaires, eye-tracking, to name a few).

At the end of your exposé, you should give complete information on every source you cited in your proposal. More often than not there are strict norms and requisites for the bibliography. You may complain about it, but that is the way it is: hating bibliography norms seems to be part of being a scholar, so you are getting closer!

Further reading
Chesterman, A. (2001). Empirical research methods in Translation Studies. Erikoiskielet ja käännösteoria (VAKKI-symposiumi XX), 27, 9-22.
Jääskeläinen, R. & Tirkkonen-Condit, S. (1991). Automatised processes in professional vs. non-professional translation: A think-aloud protocol study. In S. Tirkkonen-Condit (Ed.): Empirical Research in Translation and Intercultural Studies. Tübingen: Narr, 89-109.
Neunzig, W. (2002). Estudios empíricos en traducción: apuntes metodológicos. In F. Alves (Ed.): O proceso de traducão. Cadernos de Traducão, 10, 75-96.
Nünning, A. & Sommer, R. (Eds.) (2007). Handbuch Promotion – Forschung – Förderung – Finanzen. Stuttgart: Metzler.
Nussbaum, M. A. (2010). How To Write a (Thesis/Dissertation) Proposal. URL:
Pries, L. (2007). Wie schreibe ich ein Exposé? URL:
Research Proposal Guide. URL:

Recent Comments