Wednesday, June 30, 2021

Fast grants and the economics of subsidizing science

One of the great insights of modern growth theory -- Paul Romer's Nobel Prize -- is that ideas are the foundation of economic growth. Ideas are also "nonrivial." If you use my car, I can't use it, but if you use our family recipe for road-oil chocolate cake (yum), we can still enjoy it as much as ever. Once an idea has been had, economics says it should be used as widely as possible as soon as possible  

But coming up with ideas is expensive. And aside patent protections, I can't charge for the benefit to you of my new ideas.  So, economists naturally notice the mother of all public goods. Research -- finding new ideas --  has enormous benefits, and people will not naturally devote enough resources to finding, refining, implementing new ideas. So, economists conclude,  the government should subsidize idea-production. 

But which ideas?  Now we face the conundrum. It's just as easy to subsidize bad idea production as good idea production, and it's even easier to waste money and produce no new ideas at all. How to subsidize actual productive ideas is a hard question of bureaucratic structure. The economics of science is, I think, vastly understudied. How can government agencies or philanthropies give away money and actually do good? This topic is especially relevant as we contemplate a big ramp-up in federal spending. 

Enter today's topic, a fascinating review of Fast Grants by Patrick Collison, Tyler Cowen, and Patrick Hsu.  Read first the Marginal Revolution summary, then the full article

I found it as interesting for its insights into the pathologies of our current system for subsidizing research as for its summary of how well fast grants worked. 

They survey fast grant recipients. Despite being in an evident crisis, and $5 trillion being shoveled out the door... 

64% of respondents told us that the work in question wouldn’t have happened without receiving a Fast Grant.

For example, SalivaDirect, the highly successful spit test from Yale University, was not able to get timely funding from its own School of Public Health, even though Yale has an endowment of over $30 billion. Fast Grants also made numerous grants to UC Berkeley researchers, and the UC Berkeley press office itself reported in May 2020: “One notably absent funder, however, is the federal government. While federal agencies have announced that researchers can apply to repurpose existing funds toward Covid-19 research and have promised new emergency funds to projects focused on the pandemic, disbursement has been painfully slow. …Despite many UC Berkeley proposals submitted to the National Institutes of Health since the pandemic began, none have been granted.” [Emphasis ours.]

 We know that applying for grants is drudgery, but 

57% of respondents told us that they spend more than one quarter of their time on grant applications. This seems crazy. We spend enormous effort training scientists who are then forced to spend a significant fraction of their time seeking alms instead of focusing on the research they’ve been hired to pursue.

I have heard many anecdotes how the research funding mechanism skews research to incremental, or currently fashionable topics. (MRNA research struggled for years!) 

In our survey of the scientists who received Fast Grants, 78% said that they would change their research program “a lot” if their existing funding could be spent in an unconstrained fashion. ...

81% percent of those who responded said their research programs would become more ambitious if they had such flexible funding. 62% said that they would pursue work outside of their standard field (which the NIH explicitly discourages), and 44% said that they would pursue more hypotheses that others see as unlikely (which, as a result of its consensus-oriented ranking mechanisms, the NIH also selects against).

Many people complain that modern science is too frequently focused on incremental discoveries. To us, this survey makes clear that such conservatism is not the preference of the scientists themselves. Instead, we’ve inadvertently built a system that clips the wings of the world’s smartest researchers, and this is a long-term mistake. 

 ...the current grant funding apparatus does not allow some of the best scientists in the world to pursue the research agendas that they themselves think are best.

...The entities involved in science funding, most notably the NIH, demand long applications and subject those applications to multiple stages of administrative review, written and in-person peer review, program officer review, advisory council review, and even council of councils review. Consensus plays a heavy role. Scientists are discouraged from pursuing research outside of their regular fields and there is a strong preference for funding late-career rather than younger individuals. 

...According to the NIH, a grant application will typically result in a decision after something between 200 and 600 days.

That's a long time! Ideas build on each other, so the speed of economic growth depends on the speed of new idea production and implementation. The grant application process is starting to look like the construction permitting process, with the same results. 

But we are economists. Not everything is screwed up because of stupidity.  When giving away money, especially taxpayer's money, there is always a danger of fraud, nepotism, corruption, favoritism, waste. The fast grants structure (below) is interesting. But do government and private science subsidy bureaucracies not adopt it because they just hadn't thought of it? That's possible -- institutional innovation happens too. But it surely is a question worth asking. Are the institutional incentives of the NIH and NSF the problem? (See for example the astonishingly screwed up incentives for regulating Nuclear Power.) I don't see signs that they are rewarded for producing useful and speedy research. CYA behavior from bureaucracies does result from CYA incentives. 

They link to a good article by José Luis Ricón, "We don't know how to fix science," which I recommend as documentation for how little is known about the science and economics of science, how many simple cocktail party ideas -- fund people, not proposals, allocate randomly -- remain on the table. 

Collison, Cowen and Hsu acknowledge a few things that did work if not perfectly. In the end we want to build on positive models not just disparage failures. 

We would not have had vaccine candidates as quickly as we did without a great deal of basic science work painstakingly pursued (sometimes for decades) by people like Katalin Karikó, Kizzmekia Corbett, Jason McLellan, and many hundreds of others.... This research ultimately was supported and that is no small feat. The funders involved (the NIH included) deserve our great gratitude. On the translation front, Moderna, BioNTech, and Novavax, creators of the vaccine candidates that performed best in clinical trials, all started out as privately backed biotech startups that relied on risk-tolerant funders, underscoring the importance of a vibrant private ecosystem.

Operation Warp Speed, an interagency project that the NIH and FDA both participated in, was an excellent and successful example of bold action and also institutional courage. We were able to produce vaccines as quickly as we did in part because the NIH funded three grants at UT Austin before 2020; thanks to these, we had a stabilized spike protein ready to go...It’s also important to note a lot of the unglamorous infrastructure that enabled science to progress quickly during the pandemic, such as databases maintained by the National Center for Biotechnology Information (NCBI, part of the U.S. National Library of Medicine), are enabled and funded by the NIH and other public funding bodies. 

Fast Grants

The original vision was simple: an application form that would take scientists less than 30 minutes to complete and that would deliver funding decisions within 48 hours, with money following a few days later.

...To help identify the most immediately deserving recipients, our criteria were quite stringent: eligibility was restricted to “principal investigators” (that is, scientists running their own labs or research programs) who were already working on COVID-19-related research (rather than those who merely had ideas as to how they could be).

...Within a few days, we started to distribute millions of dollars of grants, and, over the course of 2020, we raised over $50 million and made over 260 grants. All of this was done at a cost of less than 3% Mercatus overhead...

The grant applications were refereed by a team of 20 mostly early-career individuals drawn from top universities and labs, who worked hard to vet and review the more than 6,000 applications received over the course of the program. Every funded application was reviewed by at least three reviewers, but unanimity was not required: the goal was to identify projects that at least one or two reviewers thought were very much worth funding. (Successful NIH grant applications, on the other hand, are typically reviewed by 10-20 scientists and program officers across three phases of review.) 

“Let’s do it” was then the basic attitude and we were much more worried about missing out on supporting important work than looking silly.

...We allowed research teams to repurpose funds in any plausible manner, as long as they were used for research related to COVID-19. Besides the 20 reviewers, from whom perhaps 20-40 hours each was required, the total Fast Grants staff consisted of four part-time individuals, each of whom spent a few hours per week on the project after the initial setup.

We give examples of actions Fast Grants took not to indicate some kind of supposed brilliance but rather to emphasize the opposite: Fast Grants pursued low-hanging fruit and picked the most obvious bets. What was unusual about it was not any cleverness in coming up with smart things to fund, but just finding a mechanism for actually doing so. 

This blog usually sympathetic to private solutions. And billions sitting in philanthropic organizations such as the Gates foundation, or university endowments seem like a natural source of financing for basic research. The Medici funded Galileo, and though science is bigger, so are pots of wealth. But

We found it interesting that relatively few organizations contributed to Fast Grants. The project seemed a bit weird and individuals seemed much more willing to take the “risk”. (That said, a few institutions did contribute substantial amounts, and we’re very grateful to those that did.) Beyond Fast Grants, we suspect that a lot of valuable projects in the world are blocked on something like this: the willingness of funders (especially institutional funders) to support something unusual simply on the basis of belief in the individuals involved. Too often, the de facto goal of funders is to find established things that look like everything else — it is typically easier to defend supporting a long-established institution. But, of course, the most valuable opportunities will often be those that look quite different, and a structural bias towards familiarity can easily militate against innovation.

This comment also suggests a way that Fast Grants needs an underlying ecosystem. They fund people with reputations, but where did those people get those reputations? 

Regular vs emergency mode?

Collison, Cowen and Hsu were motivated by an expectation that the US would Shift from "regular mode" to "pandemic mode," and astonishment that this did not happen. 

As the first U.S. lockdowns commenced in March last year, we reached out to various top scientists, and were surprised to learn that funding for COVID-19 related science was not readily available. We expected the U.S.’s immense government funding systems to be unleashed, with decisions made in days if not in hours. This is what happened during World War II, which killed fewer Americans.

Instead, we found that scientists — among them the world’s leading virologists and coronavirus researchers — were stuck on hold, waiting for decisions about whether they could repurpose their existing funding for this exponentially growing catastrophe. It’s worth visiting the National Institutes of Health (NIH)’s application overview for this, launched in March 2020, to get a tangible sense for what those seeking emergency funding were facing.

But why are we not always in pandemic mode? What purpose does the bureaucracy and delay serve in normal times? 

 The failure of science in the pandemic

Where is the science? The story of this pandemic must include big failures of the science establishment. 

the U.S. did virtually no surveillance sequencing in January or February, even as outbreaks took hold in China and Italy. The CDC botched the initial diagnostic test. Even once working tests existed, they remained invariably slow and hard to obtain for members of the public. We didn’t get off on a good foot.

And to this day,  

as numerous variants of concern — such as B.117 and B.1.351 — took hold late in 2020 and in early 2021, we were surprised that the U.S. was engaging in very little surveillance sequencing with sufficiently fast turnaround to detect their real-time spread. 

A year after Pearl Harbor, the US was cranking out ships and airplanes. A year later and the basic infrastructure of monitoring the spread of the disease seems to be lacking.  I use the word "infrastructure" deliberately. 

Update

Coincidentally, Heather MacDonald writes in the Wall Street Journal of the NIH's new racial  and social policies in deciding what research to fund:

On June 10, NIH director Francis Collins announced a new requirement for participating in the brain initiative. Neurologists, molecular biologists and nanophysicists seeking NIH funding must now submit a plan showing how they will “enhance diverse perspectives” throughout their research. Scores on the “plan for enhancing diverse perspectives” will inform funding decisions.

Each “plan for enhancing diverse perspectives” must show how the principal investigator will “empower” individuals from groups “traditionally underrepresented” in biomedical research, such as blacks, the disabled, women and the poor. Institutions are also covered by the diversity mandate. Researchers working on an NIH neuroscience grant should be drawn from institutions that are traditionally underrepresented in biomedical research, including “community-based” organizations.

... Dr. Collins also announced on June 10 an additional $30 million in grants for addressing the “impact of structural racism and discrimination on minority health” and another $60 million for projects “aimed at reducing health disparities.”  

How much this move will heal the nation's racial, gender, disability, and poverty (how many poor neuroscientists are there anyway?) and other "disparities" I will leave to you to ponder. Whether the Federal Government could heal those wounds more effectively by spending $100 million dollars somewhere else seems a good topic for discussion. How much this will advance scientific knowledge of how the brain works seems pretty obvious. 

27 comments:

  1. "nonrivial" I think that should be nonrivalrous which is the adjectival form.

    ReplyDelete
  2. Something I hadn't understood until conversation with friends in the biological sciences is that grants are closer to payment for successful research than finance of risky projects.

    As the grant landscape has grown more competitive, it is difficult to win a grant without good "preliminary" data. It's like a grant is written for two papers, and the first paper is already completed at the time the grant is written.

    This raises a financing question of who funds the research necessary to win a grant? If the existing grant establishment moves even further toward funding high reputation scientists and labs, the research analogue of big, highly profitable companies, where do startup funds come from?

    ReplyDelete
    Replies
    1. Many economists have suggested that we just formalize this. Forget about pretending to write grants for future research, just give prizes for good past research.

      Delete
  3. In the aftermath of Pearl Harbor, political partisanship reached a nadir and essentially disappeared. During 2020, an election year, political partisanship reached its zenith and political gamesmanship thrived.

    It is not difficult to see how, in a politically charged adversarial environment where the FBI is investigating the president and the major media outlets are contra the administration at every point, and the president himself was inconstant and inconsistent, whole departments would become dysfunctional in the face of unknown unknowns. The president had basically shredded the administrative apparatus in the first three years of his term in office and had not put in place the adaptable bureaucracy necessary to meet the emergency that arrived in January of 2020.

    The Democrats were no better prepared or more capable of rising to the challenge, as was seen from the blue states' responses.

    Even today, with the experience of 2020 in hindsight, there is a basic level of dysfunction in the administrative state.

    The wonder is that it wasn't worse than it was. If it was asymmetric warfare that the nation had had to face, it would be quite impossible for the country to mount a rapid response under the present conditions.

    History is replete with examples of disunited political units succumbing to a determined well organized and led foreign aggressor. The U.S. today is akin to those earlier examples of disunited political states; it fails at its peril.

    ReplyDelete
    Replies
    1. I would say that the end of political partisanship began about 1933 when the Democrats gained overwhelming majorities in the House and Senate.

      https://en.wikipedia.org/wiki/1932_United_States_House_of_Representatives_elections

      "The inability of Herbert Hoover to deal with the Great Depression was the main issue surrounding this election, with his overwhelming unpopularity causing his Republican Party to lose 101 seats to Roosevelt's Democratic Party and the small Farmer–Labor Party, as the Democrats expanded the majority they had gained through special elections to a commanding level."

      https://en.wikipedia.org/wiki/1932_United_States_Senate_elections

      "With the Hoover administration widely blamed for the Great Depression, Republicans lost twelve seats and control of the chamber to the Democrats, who won 28 of the 34 contested races (two Democratic incumbents, Duncan U. Fletcher of Florida and John H. Overton of Louisiana, were re-elected unopposed)."

      I would also say that our two party political system is broken in that it incentivizes winner take all tactics and fosters a lack of cooperation.

      Delete
    2. This comment has been removed by the author.

      Delete
    3. This comment has been removed by the author.

      Delete
    4. Old,

      "In the aftermath of Pearl Harbor, political partisanship reached a nadir and essentially disappeared."

      "1933 is too way-back to be representative today."

      Really, 1933 is too way back, but the attack on Pearl Harbor (Dec. 7, 1941), a mere 8 years later, is somehow light years ahead?

      Delete
    5. This comment has been removed by the author.

      Delete
    6. Old,

      "Even today, with the experience of 2020 in hindsight, there is a basic level of dysfunction in the administrative state."

      "If you consider the multiparty systems in place in the U.K., Germany, Canada, the EU, etc., there is little there to commend itself to the world hegemon, the U.S.A. at this point."

      Does the same level of dysfunction in the administrative state exist in places like the UK, Germany, Canada, the EU, etc.?

      Delete
    7. FRestly,
      The aftermath of Pearl Harbor, incl. the declaration of war on the USA by Germany, created a unified body politic -- an emergency caused politics as usual to be put aside. The reference is in connection with John Cocrane's and Collison, Cowen and Hsu's observation and remarks wondering why in the face of the pandemic the bureaucratic organizations of government failed so abysmally. On that basis only is 1933 too way back (a play on words: the "Wayback Machine" is a website that offers access to technical and economics papers that are now out of print.)

      Delete
    8. Old,

      "The aftermath of Pearl Harbor, incl. the declaration of war on the USA by Germany, created a unified body politic -- an emergency caused politics as usual to be put aside. The reference is in connection with John Cocrane's and Collison, Cowen and Hsu's observation and remarks wondering why in the face of the pandemic the bureaucratic organizations of government failed so abysmally."

      https://en.wikipedia.org/wiki/World_War_II_casualties
      US Deaths - 407,300
      Total Worldwide Deaths - 70 to 85 million (including 21 to 25 million military)

      https://en.wikipedia.org/wiki/COVID-19_pandemic_deaths
      US Deaths - 588,756
      Total Deaths - 3.66 million

      So the bureaucratic institutions in the US failed during the Covid 19 pandemic because of political disunity, but succeeded following the attack on Pearl Harbor because of a unified body politic?

      I don't see the difference in the numbers above.

      And you didn't answer my question - Does the same level of dysfunction in the administrative state exist in places like the UK, Germany, Canada, the EU, etc. (places where more than two parties share power)?

      https://en.wikipedia.org/wiki/Multi-party_system

      "Argentina, Armenia, Belgium, Brazil, Denmark, Finland, France, Germany, Iceland, India, Indonesia, the Netherlands, New Zealand, Norway, the Philippines, Poland, Sweden, Tunisia, and Ukraine are examples of nations."

      "On the other hand, if there are multiple major parties, each with less than a majority of the vote, the parties are strongly motivated to work together to form working governments. This also promotes centrism, as well as promoting coalition-building skills while discouraging polarization."

      https://en.wikipedia.org/wiki/COVID-19_pandemic_death_rates_by_country

      US Deaths per 100,000 - 184.5
      Canada Deaths per 100,000 - 69.98
      Germany Deaths per 100,000 - 109.5
      Netherlands Deaths per 100,000 - 104.07
      Ukraine Deaths per 100,000 - 123.2
      Sweden Deaths per 100,000 - 142.2

      Delete
    9. The figures you cite do not support your contentions.

      Delete
    10. Your statement (not mine):

      "Even today, with the experience of 2020 in hindsight, there is a basic level of dysfunction in the administrative state. The wonder is that it wasn't worse than it was."

      Do you have any figures that back up your contention?

      Delete
    11. Such as what? The number of times that the Biden administration has lost a court case because it erred in its basic assumptions. Two and counting... There will always be a level of dysfunction. The withdrawal from Afghanistan based on an arbitrary choice of a calendar date because it coincides with an anniversary of some event. An explanation used to justify a withdrawal because x number of casualties had occurred in the past and a decision maker couldn't ask others to make the sacrifice on the basis of past losses, notwithstanding more lives will be lost because an incomplete job was done. Sidestepping the law to impose regulatory burdens because of ideological beliefs despite the prospect of those regulations being successfully challenged in court, and knowing it all the while. Vilification of 3/4s of your constituents because their skin color isn't the correct hue, and then expecting to be re-elected because the other guy isn't ``woke``. Cutting your nose off to spite your face comes pretty close to the definition of dysfunctional. Consider yourself lucky that the PRC is not yet in a position of absolute strength. When it is, this year's shortage of IC chips is going to look like Nirvana by comparison. The political situation is rife for a challenger to exploit. There's no depth in the US to carry on a two front war, and too few resources to win a short war in the western Pacific. As the Japanese have made clear, they are prepared to take on the PRC in the event of an invasion of Taiwan; they expect the US to come in with them. The trigger is the prospect of the loss of Okinawa. It will surprise and disappoint the Japanese when they find the US unprepared and unable to uphold its end of the mutual defense pact. This is administrative dysfunction--mutual promises made and exchanged by one department of government that another department cannot or will not fulfill. The risk, of course, is that you lose both Japan and Taiwan to China. All the while, politicians squabble and the administration wages cultural war against its own citizens. As I stated, it is dysfunctional; fortunately the antagonists overseas are not yet ready to act.

      Delete
    12. And then there is the 'good word' from California--how the new reality is feeding into the administration in Washington, D.C.

      "Californiarchy" by Spencer Klavan (July 7, 2021), Law & Liberty, describes the descent of California, and the rise of ''woke'' cohort to positions of power in D.C. within the Biden administration. One would be hard pressed to make this up. But there it is, in all its grandeur, in plain view.

      https://lawliberty.org/forum/californiarchy/

      Delete
    13. Does the same level of dysfunction in the administrative state exist in places like the UK, Germany, Canada, the EU, etc. (places where more than two parties share power)?

      Delete
    14. Does the same level of dysfunction in the administrative state exist in places like the UK, Germany, Canada, the EU, etc. (places where more than two parties share power)?

      Delete
    15. So,

      "The president (Republican - Donald Trump) had basically shredded the administrative apparatus in the first three years of his term in office and had not put in place the adaptable bureaucracy necessary to meet the emergency that arrived in January of 2020."

      "Californiarchy by Spencer Klavan (July 7, 2021), Law & Liberty, describes the descent of California, and the rise of ''woke'' cohort to positions of power in D.C. within the Biden administration."

      You are making my point in that in a closely divided two party electorate, fringe elements in either party have an outsized influence on policy decisions - Trumpism on the right and Wokeism on the left.

      Does the same level of dysfunction in the administrative state exist in places like the UK, Germany, Canada, the EU, etc. (places where more than two parties share power)?

      Delete
    16. FRestly, the answer is 'yes'.

      You might find the following WSJ article of interest
      https://www.wsj.com/articles/if-war-comes-will-the-u-s-navy-be-prepared-11626041901?mod=MorningEditorialReport&mod=djemMER_h

      An excerpt:
      The report surveyed active and recently retired service members of various ranks, conducting 77 candid hourlong interviews. A key finding: “Many sailors found their leadership distracted, captive to bureaucratic excess, and rewarded for the successful execution of administrative functions” rather than core competencies of war.

      “I guarantee you every unit in the Navy is up to speed on their diversity training,” said one recently retired senior enlisted leader. “I’m sorry that I can’t say the same of their ship-handling training.”

      FRestly, you can continue to look for exceptions, but it won't serve any practical purpose. The only nation that matters is the U.S.A. But, I can assure you that Canada is not the exception that you are looking for.

      Delete
    17. Old Eagle Eye,

      "FRestly, the answer is yes".

      Based upon what evidence? So far, the only thing you have been able to do is complain about US policies and actions.

      "FRestly, you can continue to look for exceptions, but it won't serve any practical purpose."

      I am looking for what works and what doesn't work. And it has become apparent to me that two party governance doesn't work.

      You seem to agree in your complaints about Trumpism (on the right) and Wokeism (on the left), but offer no substantial analysis on the cause of those administrative failings and no remedies other than to reminisce about a conflict that killed nearly 100 million people world wide.

      "The only nation that matters is the U.S.A."

      If that were the case the US would not have a Navy to begin with. The US is quite capable of defending it's own borders with ground and air based military installations.

      Delete
  4. What - no recipe for road-oil chocolate cake? It's hard to do good economics without snacks!

    ReplyDelete
  5. The idea that citizens should be forced to fund university workers (that call themselves "scientists") via taxation is preposterous.

    The fact that economists feed the "externality" narrative to justify this atrocity is just another sign of the corruption that permeates all public servants, really.

    ReplyDelete
  6. Ideas come for free but are a severely constrained resource. As in "why didn't I think of that?".
    I believe Thomas Edison said "invention is 1% inspiration and 99% perspiration".
    The usefulness of funding is to let the perspiration happen.
    --E5

    ReplyDelete
  7. Bell Labs, where have you gone? ;) There's a virtue in "trying things out" - intuition is rewarded, you get a surprise or you get disappointed. The introduction to "Order Out of Chaos" by Prigogine touches on the subject on how society/the economy directs what science should be doing. A half-lamentation.

    Icrementalism can make life easier when it comes to diffusion - but in some instances you get a leapfrog effect. Think cell phones in Africa and the reliance on mobile payments through phones. Prior to that no infrastructure for it. Tech had been refined so well it almost became drag n drop.

    ReplyDelete
  8. Anyone interested in the economics/political economy of science and public health should read The Death of Cancer (https://www.amazon.com/Death-Cancer-Pioneering-Oncologist-Winnable/dp/0374536481) authored by a physician at the forefront of the development of chemotherapies in the 1960s and beyond. This history makes clear that scientists/physicians and the institutions that they inhabit (the NIH in this case), suffer the same kinds of pathologies seen in other individuals and institutions ranging from myopia, turf protection, laziness, narrowness, ego, ... It also shows how it takes "political entrepreneurs" inside and outside of these organizations to really change things.

    ReplyDelete

Comments are welcome. Keep it short, polite, and on topic.

Thanks to a few abusers I am now moderating comments. I welcome thoughtful disagreement. I will block comments with insulting or abusive language. I'm also blocking totally inane comments. Try to make some sense. I am much more likely to allow critical comments if you have the honesty and courage to use your real name.