Impact of Funding on Scientific Output and Collaboration

Transcription

1 Impact of Funding on Scientific Output and Collaboration Ashkan Ebadi Hadi Shahidi Nejad Andrea Schiffauerova 2 November 2013 Outline q Funding q Collaboration q Impact of funding on collaboration q Impact of funding on scientific output q Research gaps and conclusion 1 / 16 1

2 Funding One of the most crucial factors for improving the research performance (Martin, 2003) Huge annual investment on R&D Importance of the link between funding and scientific development Evaluation is needed! (King, 1987) 2 / 16 Role of Funding Affects size, efficiency and productivity of R&D sector (Jacob & Lefgren, 2007) Benefits of the funded research: Ø Increasing the level of available knowledge Ø Motivating collaboration networks Ø Training skillful graduates Ø Creating new jobs and companies Ø Augmenting the problem-solving ability of the researchers (Martin et al., 1996) 3 / 16 2

3 Why researchers collaborate? Collaboration! How to measure it (Lee & Bozeman, 2005) Higher produc-vity Advantages Disadvantages (Beaver, 2001) Be2er and faster access to Prac9cal exper-se (Katz, et al., 1997) Collabora9on not necessarily results Invariant Idea exchange among scien-fic in a joint ar9cle disciplines (Heinze, et al., 2008) Verifiable (He, Geng, & Campbell-Hunt, 2009) Inexpensive Be2er access to funding resources Par9cipa9on (Beaver, share 2001) Quan9fiable Ø Co-authorship Ø Sub-authorship Coordina9on cost Finding right partners Delays (Subramanyam, 1983; Katz & Martin, 1997) 4 / 16 Funding Impact on Collaboration Author(s) Year Methodology Data Target area Result(s) Beaver and 1979 Indicators (no of Rosen authors per article) Heffner 1981 Statistics 395 articles, 28 journals Lewison and Dawson Bozeman and Corley 1998 Statistical descriptive analysis 2004 Questionnaire Regression analysis Adams et al Regression analysis Gulbrandsen and Smeby 2005 Questionnaires Logistic regression analysis Research Outputs Database (ROD) 12,925 UK papers scientists and engineers in the US 2.4 million papers of 110 US universities 24 scientific fields Positive relation 4 scientific fields Positive relation found in 2 (out of 4) disciplines Biomedical (gastroenterology) Scientists collaboration choices Top US universities Collaboration and number of funding bodies increase paper quality Significantly positive Positive effect of funding on team size ,967 records Tenured university A positive relation observed professors in Norway Lundberg et al Indicators Industrial funding to a medical university Co-authorship between university and industry Incomplete results Some signs of positive collaboration 5 / 16 3

4 Funding Impact on Collaboration Author(s) Year Methodology Data Target area Result(s) Thune 2007 Qualitative analysis Interviews with 29 researchers and R&D managers Rosenzweig et al Logistic regression 5,728 articles published in 4 American journals Collaborative R&D projects in two academic fields US Emergency Medicine (EM) Important, but other factors are also involved No significant relation Defazio et al Regression analysis Panel of 294 scientists 39 EU research networks Researchers in the EU funding program framework Funding may affect the formation of more effective collaboration networks 6 / 16 Funding Impact on Collaboration Article Ctrl. Grp Fund Gender Prestige/career Independent Variables Scientific fields Regional share Past prod. Type Various sources of funding Different funding periods? Have network structure variables? Heffner (1981) ü ü Desc. analysis No No No Bozeman and Corley (2004) Adams et al. (2005) Gulbrandsen and Smeby ü ü ü ü ü ü ü (2005) ü ü ü ü ü ü OLS Linear regression Linear regression Logistic regression No No No No No No Yes No No Rosenzweig et al. (2008) ü ü Logistic regression Yes No No Defazio et al. (2009) ü ü ü Linear regression No Yes No Literature Review 7 / 16 4

5 Funding Impact on Collaboration Highlights: Relatively new From simple indicators to statistical analysis No study done in Canada! Relatively limited data sets Lack of using a control group Vague net impact of funding on collaboration 8 / 16 Scientific Output Number of publica-ons as a measure of output quan-ty Very reliable measure for large- scale data Quan9fiable (Gingras, 1996) Number of cita-ons as an indicator of quality A good index of the mean impact at the aggregate level Easy to calculate Drawbacks: Papers of famous scien9sts are more likely to be cited Self cita9on (Okubo, 1997) Etc. 9 / 16 5

6 Funding Impact on Scientific Output Author(s) Year Methodology Data Target area Result(s) McAllister and Narin 1983 Bibliometric indicators Peritz 1990 Statistical analysis Articles published in two British journals in 1978 and 1979 Gingras 1996 Bibliometric indicators NSERC funded research NIH funded researchers Economics Feasibility study of using bibliometrics Strong positive relationship High quality schools are more productive Funded researchers are more being cited Feasible to apply at disciplines or specialties level Arora et al Econometrics Lewison and Dawson OLS regression Research funded by CNR Statistical analysis Research Outputs Database (ROD) 12,925 UK papers Boyack and Borner 2002 VxInsight 3D map 33,448 grants 4,549 outputs Biotechnology, bioinstrumentation Italy Biomedical (gastroenterology) BSR program output NIA grants More unequal distribution of funds may increase the output in the short term Positive relation between the number of funding bodies and research output impact Positive relation between funding and output in most of their cases Godin 2003 Bibliometrics Science Citation Index (SCI) database Payne and Siow 2003 Regression analysis Federal funding NSERC funded research 74 research universities Positive relation between funding and productivity, but no impact on quality Positive effect on output No impact on research quality 10 / 16 Funding Impact on Scientific Output Author(s) Year Methodology Data Target area Result(s) Huffman 2005 Econometrics USDA and Evenson Agricultural research productivity Negative impact Carayol and Matt 2006 Statistical analysis OLS regression Louis Pasteur University (ULP) Faculty members No significant effect of contractual funding Jacob and Lefgren Crespi and Geuna 2007 OLS regression NIH Positive impact on the output 2008 Econometric Thomson ISI database Higher education in OECD countries Albrecht 2009 Bibliometric indicators PubMed database, CANSA No conclusion Leydesdorff 2009 Main S&T indicators of Thomson s Web of Science and Wagner OECD (2008) Campbell et al. Shapira and Wang 2010 Bibliometrics Thomson Reuters Web of Science (WoS) database 2010 Bibliometric indicators Thomson ISI database Beaudry 2010 Regression analysis and Clerklamalice 3,678 articles Beaudry and Allaoui 2012 Regression analysis Articles and patents ,724 articles, 566 patents Macro-level comparisons NCIC C r o s s - c o u n t r y evaluation A significant impact on the time lag structure of the output Different patterns of the link between investment and world-share of publication Positive relation between funds and output quality Positive impact of China s investment on output quantity, but no major effect on the quality C a n a d i a n Positive effect of strong network b i o t e c h n o l o g y position and individual funding on academics scientific output C a n a d i a n Positive effect of public funding, no n a n o t e c h n o l o g y impact of private funding researchers 11 / 16 6

7 Funding Impact on Output - Canada Few case studies done by Campbell et al. in Beaudry and Allaoui (2012) studied the impact of funding on the Godin (2003) studied the impact of NSERC funding on the Gingras productivity productivity (1996) of Canadian and discussed nanotechnology papers the quality feasibility researchers: Ø Evaluated specifically the selection of the supported procedure of bibliometric researchers: of Genome evaluation Canada of funded Ø research: Ø Negative Ø Evaluated binomial Bibliometric the evaluation impact regression Ø Thomson Reuters Web of funded Science research (WoS) database Ø Ø Ø Evaluated Ø National Cancer the impact Institute of funded of Canada research Ø Bibliometrics (NCIC) Ø Ø Two Ø 3,724 Canadian Ø Thomson grant articles Science selection Forest Reuters and Service Citation committees 566 Index Web patents of (CFS) Ø Peer-review process was successful (SCI) Science of database NSERC in (WoS) researchers (ME database and EE) selection Ø Ø Ø Positive Bibliometrics Ø relation Positive indicators between relation between are public good funding funding measures and and productivity for productivity Ø Higher scientific impact of the funded researchers investigating papers the Ø relation Ø No Assessed Ø Positive impact No among impact relation CFS of funds private internal, of funding between and funding productivity national on NCIC papers funds and quality but international and not scientific individual position performance level Gingras Godin Campbell et al. Beaudry and Allaoui 1996 Feasibility study 2003 Positive impact on productivity No impact on quality Case studies Positive impact on performance No impact on quality 2012 Network variables Positive impact of public funding No impact of private funding Literature Review 12 / 16 Funding Impact on Scientific Output Article Ctrl. Grp. Fund Other productivity shocks Prestige/ career/ Age Independent Variables Content/ scientific fields Regional share Group size Paper quality Type Various sources of funding Different funding periods? Network structure variables? Peritz (1990) ü ü ü Descriptive analysis No No No Arora et al. (1998) ü ü ü ü ü OLS regression No No No Lewison and Dawson (1998) ü ü ü ü Statistical analysis No No No Godin (2003) ü ü ü ü Descriptive analysis No No No Payne and Siow (2003) ü ü ü Linear regression No No No Huffman and Evenson (2005) ü ü Econometrics Yes No No Carayol and Matt (2006) ü ü ü ü OLS regression No No No Jacob and Lefgren (2007) ü ü ü ü OLS regression No Yes No Crespi and Geuna (2008) ü ü ü Econometrics No Yes No Campbell et al. (2010) ü ü ü ü Descriptive analysis No No No Beaudry and Clerk-lamalice (2010) ü Beaudry and Allaoui (2012) ü ü ü ü Negative binomial regression Negative binomial regression Yes No Yes Yes No Yes 13 / 16 7

8 Funding Impact on Scientific Output Highlights: Recently attracted more attention Simple indicators and statistical analysis Several studies in Canada Relatively limited data sets Lack of using a control group Vague net impact of funding on output Less studies done at the individual level 14 / 16 Research Gaps Ø Vague relation among funding, collaboration and scientific production Ø Lots of contradictory results Ø Lack of a comprehensive study Ø Very limited knowledge about the effects of funding on collaboration Ø The importance of the impact of network structure Ø Limited scope Ø Bibliometrics of the as studies the main (collaboration methodology among the university Ø Consider different periods of funding professors or the cooperation between universities and industry) Ø Relatively old results Ø No test group Ø Adjust funding! Ø Simplified linear regressions Ø Score collaborations! (Linking to the same institute gets lower score) 15/ 16 8

9 Research Summary Ø The need for developing a clear link between the funding, multidisciplinary collaboration and knowledge production Ø Most of the studies have used bibliometrics or statistical methods for performing the analysis Ø The need for a comprehensive study that covers several scientific fields and aspects 17 16/16 Suggested Solution Bibliometrics Survey Sta9s9cal Analysis Data & Text Mining Visualiza9on Techniques Social Network Analysis 18 / 22 9

10 Methodologies Bibliometric Analysis Ø Scientific output and research quality Ø Funding trend and funding efficiency Ø Universities (regional) performance Ø Collaboration trend and research group structure Ø Measuring the complexity of research Data Mining Ø Dimensionality and size reduction Ø Clusters of researchers Ø Pattern and rules detection Text Mining Ø Pre-processing and data cleaning Ø Keyword extraction Ø Co-authorship trend (probabilities of a connection) 19 / 22 Methodologies Collaboration Network Analysis Ø Network structure variables Ø Trend of collaboration Why Ø several Collaboration methods patterns Ø A multi-dimensional phenomenon! Ø Triangulation (cross examination), using multiple methods are used in a study to empower the validation of results Statistical Ø Enabling Analysis comparisons of different explanations (Fulop et al., 2001) Ø Statistical validation of the results Survey Ø Qualitative investigation to check the quantitative results 20 / 22 10

11 Evaluation Methods Peer review Data mining Econometric Peer review Ø Fast and low cost Ø High Bibliometrics quality works are most Visualiza9on Sciento- of the time detected Surveys metrics Ø Good Ø tool Modern for Ø testing Visualiza9on technique Highly the depends findings on the experts and criteria Social Case network Ø Simplified studies Ø Relatively analysis assumptions Ø More expensive fame for will creating result the in gezng model higher funds Ø techniques High Scientometrics administra9ve (Salter (Ruegg, & cost Martin, 2007) 2001) Surveys (King, Case studies 1987) Ø Simple and easy to use Ø Not integrated and simplified (Ruegg, 2007) Ø Give a big picture of the subject Ø Useful Ø Narrative, tool Ø for Limited studying flexibility Data and easy text to collaboration understand Ø Useful Ø Less tool Econometric for convincing studying impact of a program Ø The Ø generated Could Cost- benefit mining network inconsistent can be time limited analysis (Ruegg, 2007) (Ruegg, 2007) 22 / 22 11

12 Funding Impact on Collaboration Ø Co-authorship as a measure of scientific collaboration Ø Studying the Impact of governmental funding on scientific collaboration and formation of scientific networks is relatively new (Katz & Martin, 1997; Lee & Bozeman, 2005) Ø Higher financial investment can change the structure of research Ø The efficient collaboration network will not be stable due to huge coordination cost Ø Funding enables researchers to cover the collaboration costs (Ubfal & Maffioli, 2011) Ø Funding may help the central actor(s) to make a balance between the new knowledge creation and the management of the existing collaborative relationships in the network (Porac et al., 2004) Literature Review 23 Funding Impact on Collaboration Heffner (1981) analyzed the relation between funding and multiple authorship: Ø 500 articles published in 28 journals in four scientific fields Ø Ø Statistical analysis Ø Positive relation between financial support and size of the research teams Ø Statistically significant just in chemistry and biology Researchers of top US universities with larger amounts of funding tend to work in larger scientific groups (Adams et al., 2005) Defazio et al. (2009) studied the impacts of funding on collaborative behavior and productivity of the researchers: Ø Used a panel of 294 scientists in 39 EU research networks Ø 15-year time period Ø Funding may play an important role in forming more effective collaboration networks Literature Review 24 12

13 Funding Impact on Collaboration Gulbrandsen and Smeby (2005) studied the effect of industrial funding on the performance of university professors: Ø Norway Ø Used questionnaires to collect data from all tenured professors Ø Statistical analysis Ø A Positive relation observed Ø Funded professors tend to collaborate more with other researchers from both universities and industries A significant positive effect of funding on the collaboration of the US university researchers (Bozeman & Corley, 2004; Lee & Bozeman, 2005) Literature Review 25 Funding Impact on Output Ø Number of publications as a measure of output quantity Ø Number of citations as an indicator of quality McAllister and Narin (1983) found a positive relation between NIH s funding and number of publications of the U.S. medical schools. Peritz (1990) found that even if both funded and unfunded researches are published in a high-impact journal, the funded research will be more cited. Lewison and Dawson (1998) used journal impact as a quality measure. They concluded that the number of authors per article and the number of funding bodies have a great effect on the impact of research output. Literature Review 26 13

14 Funding Impact on Output Cross Country Leydesdorff and Wagner (2009) analyzed the relation between research macro-level investment and world share of publication: Ø Different efficiencies observed among examined countries Ø Different schemes of funding Crespi and Geuna (2008) analyzed impact of investment on scientific productivity of 14 OECD countries: Ø Used Thomson ISI database Ø Ø Focused on the time lag Ø Found a significant impact of investment Shapira and Wang (2010) investigated impact of nanotechnology funding: Ø Used Thomson Reuters database Ø August 2008 to July 2009 Ø Very basic bibliometric indicators Ø China is getting closer to the U.S. in terms of the number of publications but still lower quality Literature Review 27 Publications Conferences: 1. Ebadi, A. & Schiffauerova, A., Investigating Climate Change Patterns in Montreal: A Data Mining Approach, Third International Conference on Climate Change, Brezil. 2. Ebadi, A., Moazami, A. & Schiffauerova, A., Analyzing the Collaboration Trend of Canadian Researchers in Natural Sciences and Engineering: The Impact of Geographical Proximity and Research Funding, Proximity Days Conference - 7 th Edition, Montreal, Canada. 3. Ebadi, A. & Schiffauerova, A., Impact of Funding on Scientific Output and Collaboration Patterns of Canadian Researchers: A Bibliometric and Text Mining Approach, 13 th Collnet Conference, Seoul, South Korea. Journal: 1. Eslami, H., Ebadi, A. & Schiffauerova, A., Effect of collaboration network structure on knowledge and innovation productivity: The case of biotechnology in Canada

15 Data Gathering Procedure ,156 articles 184,770 authors 381,197 grantees 29 15