Posts in aside to camera

The times, they are a-changin’ (3/3)

enero 2nd, 2012 Posted by aside to camera, blog, empirical research, methods, reliability No Comment yet

In the first post of this threefold series, I praised e-journals and suggested that specialization might lead tooverall improvements in journal quality. Let us welcome TC3, a promising addition to the short list of specialized journals within Translation and Interpreting Studies. In the second one, I sketched some advantages of a few digital resources, and of using on-line tools to build a community of practice. Here I would like to address what I think amounts to a precondition for the success of any attempt of building a community of practice, namely setting high and common standards. 

I will go straight to the point: I would like to argue that combining reproducible research with open access may probably be the best option to bring about these standards.
 

Now you see it, now you don’t

You try to repeat a fascinating piece of research reported in a journal article. You follow it step by step, veeeery carefully, only soon to realize that a lot of missing bits make it unlikely that you are actually replicating the original test by any rigorous accounts, and settle for approximation instead of comparability. Your results are different, but you cannot really hypothesize why. —A colleague asks you for the way you carried out certain details in an old research project. It takes you long to find the appropriate materials, and even longer to reconstruct some of the rationals behind decisions that were obviously taken but perhaps never stated as explicitly as it seems to have become appropriate now. Yeap, sometimes some of us cannot even make sense of our own old materials.

Yet another typical case. You would like to find support for a given hypothesis and the setting you establish to put it to the test actually fits quite well in that of a previous piece of research, so that their data might really be of use to you. But you will never know because many of the conditions were implicit in the published paper, or had not been recorded. You may become unnecessarily stuck with a reduced number of subjects. Testing your new hypothesis in their old materials might have also been a sound previous step to carry out your planned research project, but you can’t because it seems very difficult to get ahold of these materials. You cannot even challenge the interpretation of data that was offered then by reanalyzing data now from a different perspective on the same issue, on the same materials, because they are not available.

These are only some of the problems we have to deal with in our daily research efforts to study translators’ and interpreters’ cognition. Process research methods have not yet been standardized enough, and the information structure and contents of research reports are still lacking field-wide guidelines (where field allows for several readings). In PETRA we are trying to contribute to method standardization by looking at ways to profile texts and subjects. But perhaps starting the other way round, i.e. setting standards for research reporting in future efforts, might simply be a more practical way to reach both goals. That is, setting report standards might prompt agreements on some methodological standards.

A simple strategy to draft tentative report guidelines might be letting information standards emerge from the demands of the readers of research reports. A fuzzy network of researchers who mutually recognize each other as pertaining to the field of translation and interpreting process research might, after a while, reach agreements on initial, tentative report and methodological standards by exchanging information of their works. They might simply do so by spotting new, additional information demands when trying to re-use the information to reach their own goals, until these information needs are streamlined and settled. In any case, it should be an additive and realistic process of study of past projects to be applied only to new ones.

Reproducible research

A publication is not the research project itself, but merely a report of that piece of research. That is why one single research project may lead to many publications. Current information and communication technologies allow for the exchange of all research-related information in digital formats of any kind (first post, again). Today’s digital scholarship comprises the publications plus the complete environment and the full set of instructions applied to carry out the project. Simply because we can? I don’t think so. Rather, we will benefit from accessing the whole lot. For instance, providing additional materials might enlarge the possibilities of other researchers to suggest more solid observations by re-using somebody else’s tools (e.g. questionnaires) and even some data, when possible.

Replicating research projects often yields different results in all fields, not just in ours. Statistics became a discipline to come to the rescue in such scenarios. However, often many differences in replicated results may be ascribed to lack of information in the research report. As I wrote above, digital media let us share a wealth of information which would make our job easier by letting us re-use each others’ data. The information should include—as many, when not in full—actual materials whenever possible, in order to make sure that future approaches will be able to analyze the real data, and not only filtered reports, which would become just one part of the information provided. In other words, publications would become points of access to further information on a research project, including raw materials, which altogether are seen as a compendium. A reproducible-research compendium for translation and interpreting process research might include

1. The research paper

  • Full text (e.g. a PDF)
  • Full bibliographic citation with current publication status (e.g. a BibTeX file)
  • Supporting bibliography (with abstracts, when possible)

2. The experimental setting

  • Explanatory documentations for each factor, if not standardized
  • A list of the parameters, settings, and platforms under which the project was carried out that lead to the published (also unpublished?) results
  • Original texts as presented to subjects
  • Subjects profiles [and those of other parties, such as evaluators, when applicable]

3. The Data

  • Raw output data (log files, video recordings, user activity data, questionnaires, tests, etc.)
  • Criteria used for data cleaning
  • Notes on data cleaning and preparation process
  • Prepared data (files after cleaning)
  • Explanatory documentations for each part of the process of manipulating data

4. Results

  • All the results including high resolution figures, complete tables, full statistical analyses when applicable, etc.
  • Explanatory documentations for each part of the analysis.
Such a compendium might raise concerns about data privacy and the release of potentially confidential information. This may simply be sorted out by choosing a combination of the toughest standards. For instance, we might request for proof that papers in compendiums have cleared their status regarding their publishers’ policies for self-archiving. At the same time, we definitely need to look for ways to record information that will guarantee anonymity while providing a maximum of data with the consent of the participants. At PETRA we are now exploring the possibilities of on-line data collection through a website (+ computer application) to let subjects participate from their usual working environments. Apart from our main goals, we are also interested in finding out whether there are any effects of going distant and on-line on the behavior of participants in general and on ecological validity, in particular. This includes the availability of participants, their willingness to take part in the project, and other factors such that might be affected by the research situation.

Establishing standards for a compendium such as the one sketched above is probably still far too ambitious a goal, but that’s what goals are for. Several items in the checklist above need further elaboration. Variability is not easy to reduce when studying such complex tasks as those carried out by translators and interpreters. Clarifying the contents and extent of such items is as good a starting point as any to advance to standardization. In any case, we need to set the tone and fine-tune our efforts to jointly perform the score. Staying committed to just the information provided by papers, books and book chapters is like trying to stick to your old vinyl records. We all love vinyl records, but just try to carve out a ringtone for your phone from your old 33 rpm’s or to stream them over from your stereo into your computer or your tv. Just find that song by what’s-his-face-again (musical categories have really fuzzy edges) or choose to listen music from twenty different records in a row without having to attend the device after each song. In translation and interpreting process research, we seem to be sticking to scholarly vinyls. However, project documentation initiatives such as TransComp are steps in the right direction.

Open access

Once an initial standard has been established, the next step is broader dissemination. Open access offers several advantages. From a technical scope, it allows for comparison, fosters rigor in both methods and reports, and may ease junior researchers into replicating projects. If the how-tos are completely available, repeating a research project is a suitable and manageable goal for MA candidates. A wider scope on the advantages of open access may also take into account the benefits of extending the community of practice to new members and promoting good practices at once. Gatekeeping should become less tricky when more supported by both evidence and general agreement. Also, compendiums might make science more transparent to the public, and accountability is one of the sources of legitimacy for scientific endeavors.

New research projects offering compendiums could be made available in one or several dedicated website repositories, only when abiding to standards or properly challenging them. New candidates to enlarge, reduce or modify the standards for research reports would need to prove the relevance of their proposal for new or established research goals and also their fit and interaction with the rest of variables already in the standard. Successive versions of this standard would accommodate new information demands. In other disciplines there are frequent splits in standards for competing frameworks, so we may expect standards to vary in different types of projects. That's ok. Probably, new communication venues might specifically target standard-abiding research reports.

There may be other strategies, but reproducible research and open access seem both to be taking roots in neighboring disciplines, including linguistics, reading research, writing research, cognitive science, and psychology, so perhaps it is worth to give them a try, ‘cause los tiempos están kambiando.
 
 
by R. Muñoz

The times, they are a-changin’ (2/3)

junio 26th, 2011 Posted by aside to camera, blog, community, internet No Comment yet

In the first post on this topic, I wrote that scholarly e-journals are definitely better than their printed counterparts in many respects. They are, however, just an improvement on an existing research resource. The same is often said about digital libraries such as ACM, ISI Web of Knowledge and Google Scholar, although they should not be seen as catalogues or 

bibliographical databases, such as BITRA, but as sources for documents. These new libraries are actually turning good old packed and dusty shelves into computer directories cluttered with large numbers of pdf documents scholars need to manage.
 
Enter Mendeley, a user-friendly proprietary reference-and-pdf manager that will make your work much easier. This all-in-one program has too many features to explain them here, just have a look! Now this feels like something really new. On the one hand, boring, mechanical tasks such as maintaining your own library records and tracking and typing your references on a paper have now become nearly automated. On the other one, new needs such as pdf tagging and grouping are finally easier to perform, and features such as one-click document download and full-text searches might in time even change the way you gather, use, and present information. Now we are getting closer to scholarship 2.0. but there is even more to Mendeley than that, through social networking features.

Social networking

Social networking is not new to scholars, thanks to websites such as Connotea, Academia.edu and MyExperiment. And there is also Zotero, which has many of Mendeley’s features. in fact, Zotero and CiteULike may be better suited for the humanitites and the social sciences than Mendeley is (which is probably why Mendeley can be synchronized with both Zotero and CiteULike accounts). You may just silently follow the work of your colleagues simply by adding them to your contact list, very much like Google Alerts do (Mac users should also check MyPeers). One more step: Collaborative filtering helps you reduce the information overload while it expands your sources of knowledge (e.g. Logvynovskiy & Dastbaz 2010). In brief, rather than a sort of Facebook for scholars, it is more like a last.fm for researchers (Henning & Reichert 2008).
 
Mendeley also lets users establish public, open research groups, such as Translation Studies and Cognitive Translatology, and also closed, smaller ones, such as PETRA’s. Public groups let users share reading lists and closed groups let them share the documents and also collaboratively tag and annotate them. Thus, Mendeley is an invaluable tool to help us build an online scientific community for cognitive translatology (and other areas of Translation Studies, of course). However, there are some differences worth taking into account between Mendeley and Academia.edu. In Academia.edu, linking to colleagues is just a one-way step, whereby users may silently follow somebody else's work. In Mendeley, linking attempts must be accepted by adressees. This subtle difference provides for some embarrasing situations in Mendeley which in Academia.edu are simply out of question. My recommendation is to use both networks. Mendeley is better suited for smaller, topic-centered networks and Academia.edu is great for larger, interdisciplinary networks.
A community of practice
Beyond the hype of crowdsourcing, collective intelligence (but see Williams et al. 2010) and similar concepts, the web 2.0 (the read and write web) really supports easier and more varied ways to communicate and cooperate with colleagues. Sure, the new possibilities also have a dark side, such as the danger of trading scientific thoroughness for the wisdom of the crowds (check my next posts on cognitive biases). But before we dismiss scholarly social networking as a bummer, we might want to try some possibilities.Perhaps online scientific community is a little pompous; I prefer to think of it as a community of practice instead (see also Gannon-Leary & Fontainha 2007). Whatever the name, I sure hope that looking at a screen will never substitute face-to-face interaction. But there are often situations in which you simply cannot join your peers (didn’t get it? Look here). A community of practice for cognitive translatology (or TPR) is already slowly but steadily growing. We may arbitrarily date its start in the now extinct European network EXPERTISE (expert probing through empirical research on translation processes). Hubs or poles for this community are at least the Translog and CRITT nets, the TransPro database, TransComp seminars on translation process research, PACTE’s network, and even this blog, which should soon welcome many researchers to a neighborhood with street names such as cognitive translatology, translation process research, psycholinguistic approaches to translation and interpreting, bilingualism, etc.
 
Cloud computing and really smart smartphones are going to foster the development of scholarly communities of practice a little bit further. We should not drag our feet and we should not rush into it. Not because most of us are digital immigrants, but simply because we are witnessing the dawn of a new era and it is not clear which changes are just transitory, which ones are here to stay, and which ones are just one more step from a longer way. We just need to keep our research standards as high as we can, while we interiorize change and welcome innovation ‘cause the times, they are a-changin'.
 
 
References
 
Gannon-leary, Pat & Elsa Fontainha. 2007. Communities of practice and virtual learning communities: Benefits, barriers and success factors. @ Elearning Papers 5. ISSN 1887-1542.
 
Henning, Victor & Jan Reichelt. 2008. Mendeley - A last.fm for research? @ Proceedings of IEEE 4th International Conference on Escience, pp. 327–328. DOI 10.1109/escience.2008.128.
 
Logvynovskiy, Alex & Mohammad Dastbaz. 2010. Bibliography mapping with semantic social bookmarks. @ Proceedings of IEEE 5th Conf. of Intelligent Systems, pp. 7–12.
 

Williams Woolley, Anita, Christopher F. Chabris, Alex Pentland, Nada Hashmi & Thomas W. Malone. 2010. Evidence for a collective intelligence factor in the performance of human groups. @ Science 330/6004: 686-688. DOI 10.1126/science.1193147

 

The times, they are a-changin’ (1/3)

abril 30th, 2011 Posted by aside to camera, blog, internet, journals, research No Comment yet

The printing press was one of the major inventions that took humankind out of the darkness of the Middle Ages and into a more egalitarian, learned modern era. Many people think that the Internet is but leading to a comparable or even greater revolution (e.g. Bawden & Robinson 2000, Dewar 2010). And so it seems, by the looks of the sweeping democratic revolutions taking place in many countries, which might well be heralding the beginning of deep changes at home too.

Many colleagues are at odds. Some think that the new scholarship 2.0 is just a fad (Wheeler 2008), while others think new media can and must change universities in the 21st century (Thompson 2009). What is sure is that, just as the printing press enlarged the number of readers and shrank physical and 

cultural distances, the Internet is widening and strengthening the ties amongst distant members of scientific communities in ways that were just unthinkable back in 1989, when I sent my first e-mail to a friend sitting in front of another computer across the hall. Let us start with scholarly e-journals.

Scholarly e-journals

Scholarly e-journals seem to be the digital counterparts of their print forebears: their focus on research, their author style guides, their double blind peer reviews—uh, yes, and their ISSNs—make us feel that they are pretty much the same. Not quite, though, you may object, for we need to read in front of a screen. Also, electronic sources are not easy to cite, and Internet addresses are often bulky and always ugly. And then we also get this gut feeling of the ephemeral: Sic transit gloria mundi.

On the other hand, they have some important pros. Remember the mixed feelings of finally seeing your article published when you already knew better or thought differently? Well, e-journals are much faster at delivering content. This is so because, in contrast to using snail-mail, turning in a paper to an e-journal is often just a breeze (e.g. IEEE). That is especially important for reviews, which are often published in print when most addressees have already made up their minds about the reviewed text. The peer review system has also become automated (e.g. Mutatis Mutandis) and there are plenty of tools for typesetting and online publishing in general.

Another reason for this faster pace is that e-journals do not need to fit content into a fixed number of pages every x months. For instance, the forum on shared ground, which spanned from Target 12:1 (2000) to Target 14:1 (2002), would probably have taken just an (evolving) issue in an online format. In fact, many e-journals have ceased to impose (strict) limits on the length of the articles. More importantly, scholarly contributions are not constrained to words or still images any longer, and they may include video (e.g. JOVE) and other materials, such as audio or eye tracking files (e.g. JETVCE)

New features

Some e-journals offer new interactive possibilities, such as appending public comments to articles, (e.g. PloS ONE), or entering such comments in a discussion list. E-publishing can in fact be too fast. Thus, interaction may also take place before submitting manuscripts for publication. There are many repositories for preprints or working papers, such as arXiv and PhilSci-Archive, which authors use to gather feedback from their peers before they submit their (improved) work somewhere else. Some e-journals even have this interaction built in their peer review system (e.g. JIME, Ocean Science, Economics; see Nwagwu 2006).

Last, but not least, e-journals offer many advantages at handling copy. Searching through the issues, copying quotes, compiling bibliographies, sorting and storing articles is simply much easier. C’mon, you even have customizable publisher alerts! (search through your library). In brief, e-journals are now a must (Williams, Nicholas & Rowlands 2010). That is why print journals, such as Meta and Translation & Interpreting Studies, are offering now online tables of contents and abstracts of all issues. Often, new print journals, such as Translation Spaces, are hybrids supported by their own websites and blogs. More on this topic here.

Translation Studies already has quite a list of peer-reviewed e-journals, but only a couple of them have reached stardom. The cause does not seem to be their e-format, nor an alleged crisis in the peer review system (Ware 2011). It is not because of the tension between open access and toll access either. The reasons might rather be related to the scarce specialization of the journals (but see, e.g. IJIE, Tradumàtica). That is, exactly the same problem of their print neighbors and predecessors, a problem that electronic formats may easily solve because distance between like-minded peers and publishing costs are not an issue any longer, 'cause the times, they are a-changin'.

References

  • Bawden, David & Lyn Robinson. 2000. A distant mirror? the Internet and the printing press. @ Aslib Proceedings 52/2: 51–57.
  • Dewar, James A. 1998. The Information Age and the printing press. Looking backward to see ahead. RAND corporation Report # P-8014. @ http://www.rand.org/pubs/papers/P8014.html
  • Nwagwu, Williams. 2006. Peer-review and the electronic journal: Opportunities for the participation of developing countries’ scientists in mainstream science. @ Africa Media Review 14/1&2: 73–93.
  • Thompson, Gary. 2009. How the Internet (our 21st century printing press) can and must transform universities. @ http://cloudinc.org/ecosystems/article/how-the-internet-our-21st-century-printing-press-can-and-must-transform-universities
  • Wheeler, Brad. 2008. E-Research Is a Fad: Scholarship 2.0, Cyberinfrastructure, and IT Governance. @ R N Katz, ed. The tower and the cloud: Higher education in the age of Cloud Computing. Boulder (CO): EDUCAUSE, pp 108–117.
  • Williams, Peter; David Nicholas & Ian Rowlands. 2010. E-Journal usage and impact in scholarly research: A review of the Literature . @ New Review of Academic Librarianship 16/2: 192–207.

Research & the market

abril 16th, 2011 Posted by aside to camera, blog, market, research No Comment yet

Now and then it feels like some of the topics we have been researching came as a freebie in the same package of the methods we imported. Take problem solving—a nice example, since so much effort has been devoted to it. Problem solving and decision-taking seem to have come wrapped with think-aloud techniques and protocols and all in a ready-made set which did not necessarily respond to established interests, testable hypotheses, construct

validation needs or theoretical development plans within cognitive translatology.

True, research methods condition what can be studied. Conversation analysis, for instance, may have focused on turn taking and sequence organization (adjacency pairs and the like) simply because videotaping quality in the 1960s and the 70s was pretty much the same we have in our cell phones today. Thus, researchers at the time seemed to prefer tape-recorded (mainly phone) dyadic conversations, for they provided much better audio quality, even though image was lost. The choice certainly colored the way CA developed. In our case, think-aloud techniques were very well suited to capture comments on conscious mental experience and that meant that interiorized, procedural knowledge was to be sacrificed. So that was it.

Problem solving is very interesting, that is out of question. What I doubt is whether directly isolating problematic text segments and classifying them or else qualitatively delving in their circumstances will yield any significant results, mainly because the psychological frameworks for problem-solving and creativity being applied seem in need of a thorough shake and update. Corpora techniques may today help us look into text segments which are unproblematic for experts but not so for non-experts, and new data gathering techniques might help us get closer to what is going on in the translators’ minds. New tempora, new mores.

Screen recording, keylogging and eye-tracking techniques for collecting translation process data have appeared in the relatively short time span of a decade, and their potential needs to be explored. Hence, methodology is all the buzz. Even on a conservative account, about half of the sixty-odd entries in Hansen’s select select TPR bibliography focus on research methods. Once books, meta-research analyses, guides and bibliography are also removed, the remaining 18 entries may be considered a still shot of what was going on in cognitive translatology in the last decade (the 00s?).

The picture is as varied as one could expect: revision, emotions, time pressure, writing, parallels between translation and interpreting, pedagogy, evaluation, some construct validational testing, you name it. Now, one of the things that stand out from this and other lists is that only very few articles in them seem likely to attract the attention of somebody else than a cognitive translatologist. One of the welcome exceptions is O’Brien (2006), which shows why cognitive translatology is relevant for CAT.

Comparisons between professionals or experts and trainees—four entries in Hansen’s biblio, big time topic—are useful to find out what makes experts different from laypeople. The ultimate goal may be how to get some people in the second group somehow become part of the first one. Here, the operative word is somehow. But translator employers seem to be interested only in the skills newcomers do have, not in the way they acquired them, so these pieces of work do not really count as profession-oriented TPR for them.

There does not seem to be a specific reason for cognitive translatology to stay away from professional concerns. Some topics such as mental fatigue due to task length have been devoted some efforts in simultaneous interpreting (Moser-Mercer, Künzli & Korac 1998; AIIC 2002, summarized in Mackintosh s.d.) and definitely deserve more attention. To my knowledge, Petra Klimant’s ongoing research may be an all-time first attempt to extend this thread to translators (corrections welcomed). So this seems to be a promising path, one of the several tracks we need to tread to get out of the ivory tower. There are other topics, such as the influence of computing on mental processing, cognitive implications of CAT tool design, sight-translating + transcribing vs. translating (compare to Gorm Hansen & Dragsted 2007) and so on. So let’s get the ball rolling. Let us show them why they should be interested in what we do.

References
Gorm Hansen, Inge & Barbara Dragsted. 2007. Speaking your translation: Exploiting synergies between translation and interpreting. F Pöchhacker, AL Jakobsen & IM Mees, eds. Interpreting studies and beyond: A tribute to Miriam Shlesinger. Copenhagen: Samfunslitteratur, pp. 251-274.
Moser-Mercer, Barbara, Alexander Künzli & Marina Korac. 1998. Prolonged turns in interpreting: Effects on quality, physiological and psychological stress (Pilot study). Interpreting 3/1: 47-64.
O’Brien, Sharon. 2006. Eye-tracking and translation memory matches. Perspectives: Studies in Translatology 14/3: 185-205.

Recent Comments