An updated/revised/improved version of this analysis has been published in the Canadian Journal of Experimental Psychology. An open-access preprint is available on PsyArXiv.
What follows is the original blog post (I suggest reading the actual paper).
For the busy or lazy, here are the highlights:
- The average number of published papers for Assistant Professor hires in Canadian cognitive psychology divisions (or related) between 2006-2016 was ~16 (median = 12). [Note: This only includes hires for universities with Ph.D. programs in cognitive psychology or related. These are people who were hired as PI’s in research institutions.]
- For context: The average number of publications for Psychonomic Society Early Career Award winners (from 2012-2017) at the time of their first faculty position was 12 (median = 8).
- The average number of published papers has almost doubled since the start of the last decade: The average went from 11 for 2006-2011 to ~20 for 2012-2016. The average for 2016 (N = 11) was 28.
- Since 2012, 70% of the new hires in cognitive psychology divisions have been, by my estimation, neuroscience hires.
- Only ~55% of the hires (2006-2016) graduated from Canadian Ph.D. programs. [Note: The students of the faculty members who decided to hire non-Canadian Ph.D.s have (or will have) a Canadian Ph.D.] Worth noting that Canadian Ph.D. and International Ph.D. hires were identical in terms of how much work they published.
- Since 2012, there have only been 6 non-neuro (i.e., experimental/behavioural) researchers with Canadian Ph.D.’s hired across the entire country.
I’ll begin with a bit of backstory.
I finished my Ph.D. at the University of Waterloo (Ontario, Canada) in 2016. I was fortunate enough to get a position as a postdoc at Yale. I had (have) a two-year contract, which means I had to hit the job market immediately. Also, for both family and political reasons, I had a strong desire to find a position in Canada.
I decided to look at the people who have been hired as Assistant Professors (tenure-track) in cognitive psychology at Canadian universities from 2006-2016. The idea was to get a sense of how I matched up. I was also interested in whether people who don’t do neuroscience work are being hired. (Note: I am not a neuroscientist.)
I did this for 11 institutions and the results were… surprising. I decided that this is something that people ought to know about, so I contacted Valerie Thompson (my honours thesis supervisor and former Department Head at the University of Saskatchewan). We emailed the Department Heads/Chairs across Canada asking for a list of who was hired and when. I then went through (manually) and found the hires for any institution that did not email us back (to reiterate: focusing entirely on institutions with Ph.D. programs in cognitive/experimental/behavioural psychology). In total, 61.4% of the sample came from Department Head/Chair emails. Further details of the methodology can be found at the bottom of this post.
The analysis was simple: I counted how many publications each new hire had up to and including the year of their hire. The idea was to get a sense of the body of work required to get a tenure-track faculty position as a cognitive psychologist in Canada. My logic for including the year of hire was that the new work that someone might present in a job talk would be published around the time that they begin their position (and, in some cases, would have been “in press” on their CV when the hiring committee decided to hire them). By “publication” I mean published journal articles (or, potentially but probably not, relatively highly cited chapters, conference proceedings, or books – see the bottom of this post for details).
I also recorded a few other factors:
- How many Google Scholar citations that the person had in the year that they were hired (I didn’t use cumulative citations because it would have been too much work). This is intended to get some sense of scholarly impact.
- Whether the person is a credible neuro hire. I noticed that a lot of job ads were directed toward neuroscience candidates, so I looked at everyone’s publication record and counted anyone with at least 1 first author neuroscience publication as a credible neuro hire. The vast majority of neuro hires have published only neuro work, to be clear.
- Whether the individual obtained a Ph.D. from a Canadian institution.
- Gender. Inferred by me based on name/profile pictures.
Further details of the methodology can be found at the bottom of this post. Please email me with any questions or inquiries: email@example.com. Or simply leave a comment at the bottom of the post.
In total, 70 positions have been filled in Canada since 2006.
The good news here is that the number of positions has been increasing since the recession.
How many publications?
The mean number of publications across the entire time span is 15.5 [95% CI: 13.1, 18.9] (median = 12). The positive correlation between year and number of publications (r = .51) is noteworthy given the fact that the number of positions has been increasing since 2010.
The following is a comparison of the first and second half of the time span.
The mean number of publications for Assistant Professor hires in Canadian cognitive psychology has almost doubled since 2011. New faculty since 2012 have, on average, around 20 published journal articles by the time they finish their inaugural semester at their new institution [M = 19.5, 95% CI: 15.8, 23.1]. The increase (comparing 2006-11 to 2012-16) is significant, t(68) = 3.78, SE = 2.22, p < .001, d = .92.
A natural question is whether this increase is driven by a small number of top institutions. For this, I used Maclean’s 2017 ranking of the top 10 institutions in Canada based on the reputation of their medical/doctoral programs.
Top 10: Toronto (3 campuses), UBC, McGill, Alberta, Queen’s, McMaster, Western, Montreal, Calgary, Dalhousie.
Rebuffed: Brock, Carleton, Memorial, Ryerson, Manitoba, UNB (Fredericton), Ottawa, Regina, Saskatchewan, Waterloo, Winnipeg, Laurier, York.
As is evident from Figure 4, the increase in the number of publications from 2006-11 to 2012-16 has been larger for institutions outside of the top 10 (Note: There is very little statistical power here and the interaction is not actually significant, F(1,66) = 1.28, p = .261). It is noteworthy that, since 2012, the bottom 14 institutions have essentially caught up to the top 10 in terms of the # of pubs for new hires.
Putting these publication numbers into context.
The mean number of publications for Assistant Professor hires in Canadian cognitive psychology divisions since 2012 was precisely 19.459 [95% CI: 15.8, 23.1]. So, the range is somewhere between 16 and 23 publications.
I put up a very scientific Twitter poll to get a sense of what people’s priors were in this case. Note that this is a particularly unrepresentative sample and the very act of posing the question implies (to me) that the answer is surprising. At any rate, here are the results:
So, 24% guessed correctly. This is higher than I expected, but obviously still low.
I will offer two points of contrast.
For the first, I created a list of the most eminent cognitive psychologists (based on other people’s lists and personal opinion). I then figured out how many publications they had up to and including the year that they obtained their first Assistant Professor (or Lecturer, for non-NA scientists) position.
Only one of these eminent scientists is within the range that we are currently seeing for new hires in Canadian cognitive psychology. Obviously, publication practices have changed substantially since these individuals were hired. Technology has made things easier. Postdoctoral fellowships are far more common now. There is also a more dense pool of applicants nowadays (i.e., more people are getting Ph.D.’s). It should be noted that there is no correlation between year and number of publications here, r(14) = .02 [excluding Kanwisher, r(13) = .14].
For a more contemporary comparison group, I also looked at every winner of the Psychonomic Society Early Career Award (from 2012-2017 – which is the entire range of the award). Again, I’ve recorded how many papers they published up to and including the year that they started their first Assistant Professor position (along with where they were hired). Note that the listed date is when the scientists were hired, not when they won the Early Career Award.
The average number of publications for Psychonomic Society Early Career Award winners (from 2012-2017) at the time of their first faculty position (M = 12) is lower than for Assistant Professor hires in Canadian cognitive psychology divisions (since 2006: M = 15.5). It is also noteworthy that there is a strong positive correlation between year of hire and publication number here as well, r(24) = .56.
The following joke went “viral” (in academic circle terms) following a post by Penny Pexman about my presentation of these results at CSBBCS2017. The author clarified that his supervisor was very accomplished. Anyway, it’s a good joke:
But what about quality?
The foregoing suggests that Canadian Assistant Professor hires have published a lot (particularly lately), but what about their impact? Google Scholar conveniently allows you to see how many citations an individual had in a particular year. Thus, I looked at the number of citations in the year that the person was hired (i.e., not cumulatively, which would have taken too much time).
Figure 5. Citations in the year of hire (among those who have a Google Scholar profile)
This is an overestimate given that people with Google Scholar profiles (N = 51, M = 17.5) have more publications than those without profiles (N = 17, M = 10). Nonetheless, the number of citations in the year of hire is increasing with time. New hires are publishing more and, naturally, having more impact on the field. The number of citations per publication has essentially held constant during this time period.
Looking just at the 11 hires from 2016, here are the top journals (based on impact factor and relevance for cognitive psych) for each of the hires.
Among the 11 hires from 2016, the majority (7/11) had at least one paper in TiCS, Nature Neuroscience, or PNAS.
What percentage of positions are being filled by “neuro” hires?
I also assessed whether the person was a credible “neuro” hire. That is, do they have at least one first-author neuroscience publication? Of course, the vast majority of neuro people publish, essentially, only neuro papers (and experimental/behavioural people publish only non-neuro papers). However, someone who publishes primarily experimental/behavioural work but who has published at least one neuro paper is being counted as a credible “neuro” hire in this analysis (I can’t recall if there are any actual examples of this in this data set).
Since 2012, 70% of the Assistant Professor positions in cognitive psychology divisions were filled with “neuro” hires. This went up to 79% for 2014-2016.
It should be noted that the increase in publications over the years is not driven by the increase in neuro hires. For cognitive hires, the mean increased from 9 (2006-11) to 17 (2012-16). For neuro hires, the mean increased from 13 (2006-11) to 20 (2012-16).
Also, secondarily, one reason why neuro researchers may be getting hired more frequently than cognitive researchers is that the former have published more papers (Mean for neuro = 17; Mean for cognitive = 13). However, for a subset of the hires, I also recorded the number of first-author publications. I found that 147 out of 241 papers from cognitive researchers were first-author (61%) whereas 437 out of 1041 papers from neuro researchers were first-author (42%). Thus, the (inferred) mean number of first-author papers is similar for cognitive (13*.61 = 8) and neuro hires (17*.42 = 7). In other words, the fact that neuro researchers publish more papers may be because neuroscience labs are more likely to share authorship. It is nonetheless unclear what is driving the higher proportion of neuro hires.
What percentage of positions are being filled by people with PhDs from Canadian universities?
From my understanding, Canadian universities prioritize hiring Canadian applicants. However, as far as I’m aware, they do not prioritize hiring people who have obtained Ph.D.s from Canadian universities. This emerges in the analysis.
Only ~55% of the people hired at Canadian universities have PhDs from Canadian Universities. Reminder: Canadians sometimes graduate from non-Canadian universities.
Remarkably, there have only been 6 experimental/behavioural researchers hired as Assistant Professors in the entire country of Canada since 2012. This, from my understanding, is the modal category for Ph.D. students in cognitive psychology divisions in Canada. There have been 14 “neuro” researchers hired since 2012 (although this could be 16 as I couldn’t determine where two neuro researchers completed their Ph.D.).
Interestingly, Canadian Ph.D. and International Ph.D. hires were identical in terms of how much work they published. International PhDs were also no more likely to do neuroscience research (in this sample).
It is important to first note that I have no information about the base-rate of man v. woman applicants. That is, a gender imbalance could reflect the possibility that more men are applying for jobs, or it could reflect a bias toward hiring men. My guess is that it’s the former, but I’m a man so of course I would say that.
So, 40% of the hires were women. Notably, it looks like the imbalance is stronger among neuro hires.
The job market for Canadian cognitive psychology is exceptionally competitive, particularly in recent years.
Naturally, the present analysis doesn’t tell us anything about how the Canadian job market matches up to other countries. However, it is noteworthy that the correlation between year of hire and number of publications was evident (and similar) among both Canadian hires (r = .51) and Psychonomic Society Early Career Award Winners (r = .54).
I would be surprised if the monumental increase in the amount of published work required to get a faculty position in cognitive psychology is unique to Canada. I would also be surprised if it is specific to cognitive psychology.
On a personal note: I should mention that I ultimately found a position in Canada… but not in cognitive psychology. I am starting at the Hill/Levene Schools of Business at the University of Regina in July 2018. I feel very lucky to have found a tenure-track faculty position. Particularly one that is relatively close to my family (I have two little girls who I want to be near grandparents).
Musings and recommendations
- First and foremost, students need to know about this. Students far too often enter Ph.D. programs under the assumption that everyone who graduates automatically gets a faculty position. Even among more enlightened students, I highly doubt that most (or even many) have an accurate representation of what is required to become a PI in a cognitive psychology program in Canada (or elsewhere, for that matter).
- The neuroscience “trend” is evident even from a casual look at job ads. People who do experimental/behavioural work cannot apply for positions in neuroscience divisions, but neuroscience researchers can apply for positions in cognitive divisions. In fact, many universities do not have separate “cognitive” and “neuroscience” divisions. McGill, for example, has a “cognition and cognitive neuroscience” division and “behavioural neuroscience” division (the latter being closer to neurophysiology, the former is included as a “cognitive division” in my analysis). I am not advocating for separate divisions (and, in fact, it would be perfectly sensible if all “cognitive” divisions were actually “cognitive neuroscience” divisions). I also think neuroscience work is important. However, I am also of the position that neuroscience needs behaviour and that it would be a shame if future generations of cognitive psychologists focus on a particular methodology at the (potential) expense of cognitive theory. More pragmatically, neuroscience work is also far more expensive, so there are funding issues that need to be considered (Canadian cognitive psychologists as a community depends largely on tri-council funding, which only has so much money to pass around). Finally, many cognitive psychology Ph.D. students don’t have the opportunity to do neuroscience work.
- To me, it’s obvious that Canadian universities should both favour Canadian applicants and applicants who graduated from Canadian universities. Canadian faculty members have students who have (or will have) Canadian Ph.D.s. I assume that everyone wants their students to be competitive for positions; why hire from elsewhere when equally competent individuals are graduating from Canadian institutions (see above)? Naturally, all of this is assuming all else is equal.
- Finally, I think this sort of analysis is really interesting and revealing. I would love to know what things look like elsewhere. My goal is to complete the analysis for other divisions in Canadian psychology as well. I will leave it up to the non-Canadian readers to take up the challenge for other countries.
Additional notes on the methodology
- As mentioned, 61.4% of the sample came via Department Head/Chair emails. The individuals who I tracked down manually did not differ from the individuals who were included in the Dept Head/Chair response: Mean publication (p = .838), Mean Google Scholar citation (p = .996), proportion “neuro” (p = .837).
- In the cases where I had to manually go through departmental websites, faculty members often did not indicate on their websites or CVs when they were hired. In those cases, I went through their publication record and inferred the date of hire based on when their affiliation changed.
- Some universities do not have separate cognitive and neuroscience divisions (although most do). These were nonetheless included in the analysis. This may partially explain why the proportion of neuro hires is so high. However, it’s worth noting that, for example, 13 people have been hired at U of T, McGill, and UBC since 2006 (all institutions that have both cognitive and neuro divisions)… and only 3 of them have been non-neuro hires.
- I noted about that highly cited conference proceedings/chapters/books may have been counted as publications. This is because Google Scholar collects a lot of junk, which is most easily filtered by looking at citations. Thus, for the 73% of the hires who had Google Scholar profiles, highly cited conference proceedings/chapters may have been confused for journal articles. I think this is suitable, however (i.e., I wouldn’t normally consider a conference proceeding or chapter a publication, but the impactful ones perhaps should be). It’s worth noting that this favours faculty who were hired earlier since their conference proceedings/chapters have more opportunity to be cited. In other words, the notable increase in the number of publications runs counter to this factor. But, at any rate, it’s rare for conference proceedings/chapters to be highly cited (it might not have happened at all in this data set, though I would have to go through it all again to know for sure… and I’m not going to do that).