ĐÁNH GIÁ KHOA HỌC NHÂN VĂN - phục sinh những khoa học bị lãng quên Alesia Zuccala, March 2013 trên Research Trends Evaluating the Humanities: Vitalizing ‘the forgotten sciences’ researchtrends/issue-32-march-2013/evaluating-the-humanities-vitalizing-the-forgotten-sciences/ EVALUATING the Humanities: VITALIZING ‘the forgotten sciences’ Scholars and university administrators worldwide are concerned about the long-term sustainability of Humanities research, particularly in a time of increasing financial cutbacks and growing policies towards quantifying scholarly achievement (1). The key to sustainability is to develop relevant evaluation methods; however, standards for this are not yet as well established, at least not metrically, as they have been for research in Science and parts of the Social Sciences. At the Royal Netherlands Academy of Arts and Sciences (2), a committee designated for the National Plan for the Future of the Humanities suggests that new indicators are part of the solution, and that the Humanities demand a fairly wide range of quality indicators that will do justice to the diversity of products, target groups, and publishing cultures present within this field (p. 11). Some scholars believe that evaluators should focus more on the role and future of the monograph, including its possible survival in the digital age (3). Others are convinced that the open access movement will play a significant role, where universities can take responsibility for creating their own databases for Humanities outputs and maintaining them as part of their individual digital repository programs (4, 5). These are the most prevalent issues, and attempts to ease the evaluation ‘crisis’ will not likely succeed without considering how the Humanities have evolved, and how useful it is (or not) to label this field distinct from other research fields. What are the Humanities? Rens Bod (6, 7) at the University of Amsterdam has addressed this question in detail in De Vergeten Wetenschappen: Een Geschiedenis van de Humaniora (“The Forgotten Sciences: A History of the Humanities”). According to Bod, there was a long-standing assumption that the Humanities were not considered a separate field of study (i.e., separate from the Sciences) until the nineteenth century. In truth, it was the Italian political philosopher, Giambattista Vico, who first worked out a conceptual distinction between a science of the human and a science of the natural as early as the 1700s. Throughout the fourteenth century there was a branch of thriving disciplines known as the studia humanitatis from which the (early) modern humanistic disciplines emerged. Many changes have occurred since then, and now, if the following question is posed, What are the Humanities? Bod says it is like asking St. Augustine to explain the notion of “time”: “If you don’t ask, we know, but if you ask, we are left empty handed. Since the nineteenth century the humanities have generally been defined as the disciplines that investigate the expressions of the human mind. Such expressions can be language, music, art, literature, theatre, poetry, etc. Thus philology, linguistics, musicology and the study of the visual arts all belong to the realm of the humanities, unlike the study of nature, which belongs to the domain of science (such as physics, astronomy, chemistry and biology). Similarly the study of humans in their social context is one of the social sciences (such as sociology, psychology, anthropology and economics). But these definitions are unsatisfactory. Mathematics is to a large extent a product of the human mind, and yet it is not considered a humanistic discipline. A pragmatic stance may be more workable: the humanities are the disciplines that are taught and studied at humanities faculties. According to this pragmatic ‘definition’, the humanities currently include linguistics, musicology, philology, literary studies, historical disciplines (including art history and archaeology) as well as more recent fields such as film and television studies. In some countries theology and philosophy are also taught in humanities faculties, whereas in others they are faculties in their own right “(7). Bod’s historical overview also points to the fact that for many centuries there has been no distinction between the Humanities and Sciences. He even suggests that some of the distinctions that we seek are somewhat artificial. Instead of working towards establishing a distinction, perhaps the more important question to ask is the following: “To what extent can expressions of the human mind, such as language, literature, music and art, be called ‘empirical’ if they are created by people?” “Indeed products of the humanities have been created by people, but when the products manifest themselves in the form of (collections of) manuscripts, pieces of music, literary works, sculptures, grammar books, plays, poems and paintings, they are obviously just as open as other objects to empirical research and the development of hypotheses. [S]ince Antiquity humanistic material has indeed been exposed to hypotheses and evaluation relating to assumed patterns and interpretations” (7). There are many reasons for not separating Humanities scholarship from the Sciences. As Bod notes, the Humanities, like the Sciences, possess a memory function. In books, manuscripts, documents, and other forms of record keeping, scholars keep events alive from past to present. The Humanities also have an educational function, which can be and has been passed on from generation to generation. It is in light of their critical interpretive and research function that we need a proper definition, or at least a suitable framework to approach this field in terms of scholarly evaluation. The crux of the evaluation crisis is not our lack of understanding as to why the Humanities are distinct or special compared to the Sciences. It is that we have allowed ourselves to ‘forget’ that the societal, economic and even technological impact of the Humanities has already been very significant throughout history and vastly underestimated (6, 7). With this forgetfulness, we seem to have convinced ourselves that the products of Humanities research are not ‘empirical’ enough for objective forms of evaluation. They are, but stakeholders must be prepared to accept the challenge of amassing, standardizing and promoting access to different forms and levels of information, data, and metadata pertaining to these outputs (5). Bibliographic datasets, citation monitoring and publication trends Within the bibliometrics community, the Humanities and Sciences have traditionally been regarded as distinct, but here, this distinction arises purely from a ‘citationist’ perspective, where citations, specifically journal citations, reign supreme in evaluation procedures for the Sciences. The Humanities are different, because humanists often disseminate information using media other than journals (8, 9). Some parts of the Humanities function quite similar to the Sciences: for instance, the discipline of Linguistics where it is quite common for scholars to publish regularly in and cite articles in fairly high-impact journals (10, 11). Nevertheless, citation-based indicators pertaining to Humanities journals are not easily compared across all subjects, let alone normalized on the basis of field-specific citation practices. Elsevier’s Scopus and Thomson Reuters’ Web of Science, providers of the most prolific bibliographic citation indices, have been making significant efforts to increase their coverage of journals for a variety of Humanities subjects. Additional document types such as film reviews, art exhibit reviews, poetry, and prose are also gradually appearing in the Web of Science. Still, bibliometricians are reluctant to use these commercial tools for evaluation purposes, due to the type and quantity of materials covered (see Table 1). Web of Science Arts & Humanities Index Scopus Arts & Humanities Subject Area Document Type Record Count % Document Type Record Count % BOOK REVIEW 227,105 40.64 ARTICLE 178,791 56.05 ARTICLE 189,304 33.88 REVIEW 76,733 24.05 EDITORIAL MATERIAL 36,082 6.46 SHORT SURVEY 19,733 6.19 POETRY 25,057 4.48 NOTE 12,042 3.77 ART EXHIBIT REVIEW 11,870 2.12 EDITORIAL 11,989 3.76 LETTER 9,931 1.78 CONFERENCE PAPER 11,577 3.63 REVIEW 6,210 1.11 LETTER 6,724 2.11 OTHER* 53,264 9.53 OTHER** 1,407 0.44 TOTAL 558,823 100.00 TOTAL 318,996 100.00 Table 1 – Overview of Web of Science and Scopus Document Types indexed for the Humanities (2008 – 2012) *Other (WoS): includes Fiction Creative Prose, News items, Biographical items, Proceedings papers, Book chapters, Scripts, Music Scores and multiple types of Reviews (Record, Film, Theatre, TV Radio, Music Performance, Dance Performance, etc.) **Other (Scopus): includes Conference Reviews, Book Reports and Dissertations Note that book reviews top the rank of all document types processed for the Web of Science Arts & Humanities Index. Journal articles are ranked second, but book reviews clearly play an important role in the scholarly communication system, serving as a gateway to the value of a newly published book (12). Rhetorical notes or ‘cites’ to various parts of a book can be anywhere from bold or subtle at conveying how well it was written, including the reviewer’s judgment of the authors scholarly credibility (13). Book reviews also correspond with how we observe influences in scholarship when we trace patterns of citations. Nicolaisen’s research has shown that books receiving favorable reviews tend to be cited more often in journal articles than books receiving neutral or negative reviews (14). Today, the prolific nature of book reviews, particularly in History and Literary studies, suggests that the university press monograph is alive and well (15). In 2002, there was in fact some concern over the death of the scholarly monograph; thus Thompson carried out an analysis of 6,708 citations (isolated from British and American literary texts) to determine whether or not the truth was evident in current publishing patterns (24). Here, she was able to identify a significant core group of journals and publishers, where university presses were clearly dominant. According to Williams et al. (3) monographs published in the Humanities are like the main course of a meal; journal articles and other scholarly communication are like tapas” (p. 76). The book or monograph is still also considered a strong requirement for career promotion and tenure (16). For the Humanities scholar it is important that his or her book is taken seriously; that it is published by a prestigious university press; read as widely as possible, reviewed and cited, and purchased by libraries (17). Since monographs and their citations have not been included as source material in commercial bibliographic indices, researchers have begun to focus on Google as a bibliographic resource. Kousha and Thelwall (18) note that there are substantial numbers of citations to academic books from Google Books and Google Scholar to help evaluate research in book-oriented disciplines. Other scholars have explored the potential of library catalogues for analyses, where an analogy may be created between journal-based citations and library holdings (19, 20, 21). White and his colleagues (21) recently introduced the term ‘libcitation’, which may be seen as an “indictor of perceived cultural benefit” (p. 1087). Currently, Scopus and Web of Science are focusing on expanding their journal indices to include books and book citations, but there is an element of uncertainty as to how much value will be given to international and multi-language publishing houses. Many works of literary theory and criticism, including texts published in History, are highly regional in character (9). Bibliometric analyses have also shown that few books published in the Humanities will become so ‘canonical’ in status, that they are able to cross regional, linguistic or disciplinary boundaries (22). Will a number of texts be ignored or undervalued because they have more significance in a regional context than they do in a global one? With the global movement towards open access and digitalization, we can expect greater opportunities to address regional differences in publishing, perhaps by normalizing for these differences, as we do metrically with field-specific citation practices across the Sciences. Publishers such as Cambridge University Press have done well to embrace the digital movement with new products, like University Publishing Online, but e-publishing innovations alone are not enough to provide insight into the Humanities broader cultural, economic, or societal impacts. What we do know is that books are regularly used by scholars and cited. For instance, in review articles published for literary studies (i.e. Dutch, English, and Catalan), the majority of references are to monographs: citation percentages range from 60% to 90%, with citations from journal article to journal article normally less than 20% (23, 24, 25). What we do not know, or have not done yet, is to objectively measure this concept of regionalism and to determine the validity of publisher prestige. A new project at the University of Amsterdam, supported by the Elsevier Bibliometrics Research Program (EBRP) is currently exploring this topic in depth, by linking monograph titles cited in journals to their publishing houses, and to international library holdings confirmed by WorldCat®. Given what the open access movement is doing for the Sciences, and the increasing numbers of scientific journal articles now freely available to scientists, it is fair to say that monographs, particularly Humanities monographs, also need to become more accessible. Progress in this regard, including the promise of complementary book indices, like Scopus and the Thomson Reuters Book Citation Index, can only tell us that the Humanities do not necessarily have to be so different from the Sciences. Clearly, we have just taken too long to observe, collect, and manage most of the relevant outputs associated with this somewhat ‘forgotten’ field. References (1) Guillory, J. (2005) “Evaluating scholarship in the humanities: principles and procedures”, ADE Bulletin, Vol. 137, No. pp. 18–33. (2) Royal Netherlands Academy of Arts and Sciences (2011) “Quality indicators for research in the humanities”, Interim report by the Committee on Quality Indicators in the Humanities. Available at knaw.nl/Content/Internet_KNAW/publicaties/pdf/20111024.pdf [Accessed 12 December, 2012] (3) Williams, P., Stevenson, I., Nicholas, D., Watkinson, A., Rowlands, I. (2009) “The role and future of the monograph in Arts & Humanities research”, Aslib Proceedings: New Information Perspectives, Vol. 61, No. 1, pp. 67-82. (4) Hicks, D. & Wang, J. (2009) “Towards a bibliometric database for the Social sciences and Humanities”. Available at dfg.de/download/pdf/foerderung/grundlagen_dfg_foerderung/informationen_ fachwissenschaften/geisteswissenschaften/esf_report_final_100309.pdf [Accessed 12 December, 2012] (5) Moed, H.F., Linmans, J., Nederhof, A., Zuccala, A., López Illescas, C., de Moya Anegón, F. (2009) “Options for a comprehensive database of research outputs in Social Sciences & Humanities (Version 6)”. Available at 83.143.5.70/download/pdf/foerderung/grundlagen_dfg_foerderung/informationen_fachwissenschaften/geisteswissenschaften/annex_2_en.pdf. [Accessed 12 December, 2012] (6) Bod, R. (2010) De vergeten wetenschappen: Een geschiedenis van de Humaniora. Amsterdam: Prometheus. (7) Bod, R. (2013, in press) A new history of the humanities: The search for principles and patterns from antiquity to the present. Oxford: Oxford University Press. (8) Huang, M-h., Chang, Y-w. (2008) “Characteristics of research output in social sciences and Humanities: from a research evaluation perspective”, Journal of the American Society for Information Science and Technology, Vol. 59, No. 11, pp. 1819-1828. (9) Nederhof, A.J. (2006) “Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A review”, Scientometrics,Vol. 66, No. 1, pp. 81-100. (10) Nederhof, A.J., Luwel, M., Moed, H.F. (2001) “Assessing the quality of scholarly journals in Linguistics: An alternative to citation-based journal impact factors”, Scientometrics, Vol. 51, No. 1, pp. 241–265. (11) Zwaan, R. A., Nederhof, A. J. (1990) “Some aspects of scholarly communication in linguistics: An empirical study”, Language, Vol. 66, pp. 523–527. (12) Lindholm-Romantschuk, Y. (1998) Scholarly book reviewing in the Social Sciences and Humanities. The flow of ideas within and among disciplines. Westport, Connecticut: Greenwood Press. (13) Zuccala, A. (2012) “Quality and influence in literary work: evaluating the ‘educated imagination’”, Research Evaluation, Vol. 21, No. 3, pp. 229-241. (14) Nicolaisen, J. (2002) “The scholarliness of published peer reviews: a bibliometric study of book reviews in selected social science fields”, Research Evaluation, Vol. 11, No. 3, pp. 129-140. (15) Zuccala, A., van Leeuwen, T. (2011) “Book reviews in humanities research evaluations”, Journal of the American Society for Information Science and Technology, Vol. 62, No. 10, pp. 1979-1991. (16) Cronin, B., La Barre, K. (2004) “Mickey Mouse and Milton: book publishing in the Humanities”, Learned Publishing, Vol. 17, pp. 85-98. (17) Gump, S.E. (2006) “Prestige and the university press”, Journal of Scholarly Publishing, Vol. 37, No. 2, pp. 69-85. (18) Kousha, K., Thelwall, M. (2009) “Google book citation for assessing invisible impact?”, Journal of the American Society for Information Science and Technology, Vol. 60, No. 8, pp. 1537-1549. (19) Linmans, A. J. H. (2010) “Why with bibliometrics the Humanities does not need to be the weakest link. Indicators for research evaluation based on citations, library holdings, and productivity measures”, Scientometrics, Vol. 83, No. 2, pp. 337-354. (20) Torres-Salinas, D., Moed, H.F. (2009) “Library catalog analysis as a tool in studies of social sciences and humanities: An exploratory study of published book titles in economics”, Journal of Informetrics, Vol. 3, pp. 9–26. (21) White, H., Boell, S.K, Yu, H., Davis, M., Wilson, C.S., Cole, F.T.H. (2009) “Libcitations: A measure for comparative assessment of book publications in the humanities and social sciences”, Journal of the American Society for Information Science and Technology, Vol. 60, No. 6, pp. 1083-1096. (22) Hammarfelt, B. (2011) “Interdisciplinarity and the intellectual base of literature studies: Citation analysis of highly cited monographs”, Scientometrics, Vol. 86, No. 3, pp. 705–725. (23) Nederhof, A. J. (1995) “A Bibliometric Study of Literature Projects Funded by the NWO Stichting Literatuurwetenschap”, Report CWTS-95-05, Leiden. (24) Thompson, J. W. (2002) “The death of the scholarly monograph in the humanities? Citation patterns in literary scholarship”, Libri, Vol. 52, No. 3, 121–136. (25) Ardanuy, J., Urbano, C., Quintana, L. (2009) “A citation analysis of Catalan literary studies (1974–2003): Towards a bibliometrics of humanistic studies in minority languages”, Scientometrics, Vol. 81, No. 2, 347–366.
Posted on: Mon, 28 Oct 2013 16:26:21 +0000
Recently Viewed Topics
© 2015