Embracing Bayesian methods in clinical trials
jamanetwork.comI found it surprising that this article persistently did not capitalize the word "Bayesian." Is this a new trend or something?
> Bayesian inference assumes the observed data are fixed and aims to quantify the evidentiary support for all possible levels of treatment effectiveness based on the data at hand.
The problem with this approach is that we can only observe ONE level of treatment effectiveness, i.e., the level of treatment effectiveness that the treatment actually possesses. All other possible levels of effectiveness are entirely hypothetical. There's no data about all these other possible levels of effectiveness because they don't occur in reality. So the data cannot possibly tell you anything about how likely is the observed outcome, because the observed outcome is the only outcome that you observe. I
This criticism was made over 100 years ago, and Bayesians still don't have an answer. They just keep going as if nothing happened, but the reality is their methodology is utterly and fatally flawed.
> So the data cannot possibly tell you anything about how likely is the observed outcome, because the observed outcome is the only outcome that you observe.
This could also be viewed as supporting the Bayesian perspective, where the observed data are not viewed as random variables - they are fixed. This is because, as you say, the observed outcome is the only outcome that you observe. It is the classical setting, in comparison, where we instead do our analysis by treating the sample as a random variable, placing the counterfactual on other non-observed values ("what if I had drawn a different sample?"), even though we didn't. Bayesian methods treat the data as gospel truth, and place the counterfactual on the different parameters ("what if the population were different?"), even though it isn't.
The other criticism you have is
> The problem with this approach is that we can only observe ONE level of treatment effectiveness, i.e., the level of treatment effectiveness that the treatment actually possesses. All other possible levels of effectiveness are entirely hypothetical.
This is true of both Bayesian and classical methods. We build models that would explain how different hypothetical levels of effectiveness would affect what data we should expect to see - that is the whole point. Classical methods also involve exploring scenarios in which purely hypothetical values of the parameter may be potentially true, and characterizing counterfactual samples that could have been drawn from them, even though in real life they couldn't have been.
> Although an adaptive design guidance finalized in 2019 left the door open for bayesian trials, their use in drug development has to date been limited,4 such as in Ebola and SARS-CoV-2 epidemics and in pediatric and rare disease trials.
the reason it has been limited to those cases is
drug development, today, is constrained by commercialization.
all four categories listed - ebola, sars-cov-2, pediatric and rare disease drugs - each for their own reason, have low commercialization risk, so if they are scientifically robust solutions, there is ROI.
most drugs in development today are new indications for existing molecules, generally in oncology, also because these are the most favorable conditions for commercialization.
commercialization, commercialization, commercialization. people don't want innovations in drug approval, nobody is clamoring for that. they want innovations in commercialization.
i don't know if bayesian methods will make trials a lot cheaper. and that's their problem! there are already a lot of very smart people working on this issue, and they have litigated to death all the objective facts.
There's a weird thing that happens with cancer drugs that I just experienced.
When they are working with a pharmacy and your insurance, the price they'll charge you is $10,000 for enough pills to last a month. But when your insurance says "We won't cover that" all the sudden you find out the company has a backdoor subsidization program which will fully cover the cost of the drug for reasons I can't really fathom (good will?)
What's even more uncomfortable is my insurance (aetna) also mandates that I get my medicine from their subsidiary, CVS.
I really don't like this sort of thing. The price of everything in medicine feels distorted in unbelievable ways. Like famously a $0.25 acetaminophen pill that somehow magically costs $10 in the hospital. I guess there's some nice individual packaging.
I'm less and less convinced that it's even a cost sort of thing with these pharmaceutical companies. Like, sure they'll love to reduce that as much as possible. But the price itself seems entirely fictitious and based on what they can commonly get insurers to pay, and not anything related to the actual R&D of the drug.
> Like famously a $0.25 acetaminophen pill that somehow magically costs $10 in the hospital. I guess there's some nice individual packaging.
When my first child was born the nurses had a terrible time drawing blood, so they gave him sugar water to calm him down billed at $40/5ml.
The mutability of the price emphasis the cost/price disjoin.
Probably you are a soft recruit to "they got better' numbers for post trial marketing but even then, it's like a Ford is $30,000 or $3 depending how easily it can be sold.
Isn’t it just regular price discrimination? Profit maximizing firms with market power charge different prices based on willingness to pay in order to sell to people who won’t pay a higher single value monopolist price.
It's a bit like software pricing, the marginal cost of production is low. You often see massively different prices charged to different types of customer.
I used to support an application used by about half a dozen businesses. They all knew each other, that's how I got their business.
Two of them were paying significantly more for their support than the others. That's because those two had their phone numbers set to ring even when my phone is in quiet mode. Just for the privilege of being able to wake me whenever you want, and get my attention even when I'm in the middle of a hike, you're paying substantially more.
> I really don't like this sort of thing. The price of everything in medicine feels distorted in unbelievable ways. Like famously a $0.25 acetaminophen pill that somehow magically costs $10 in the hospital. I guess there's some nice individual packaging.
okay well, let's say you were to model "what explains the price of something" and you had two factors in your model, "markets" and "politics." most good models will have a residual which means "everything else."
when i say "low risk" to commercialization, i could mean a lot of things. it depends on what the drug category is.
if you develop the only cure for an otherwise fatal, pediatric, congenital disease, the parents will be willing to pay an unlimited amount of money for it. so there's no market price for it. so we know that, no matter what, a correct model for that is, 0% market. this is really, extremely difficult for most people who look at this issue, because it renders their opinions about what should you pay for this drug basically moot. they'll all have different answers. so the numbers of our independent variable, "price people are willing to pay" will be uncorrelated with the price people actually pay. and this is one of those instances where the absence of correlation means no causation. instead of looking cogently at this situation and saying, "well, okay, maybe i should temper my outrage about drug pricing in this actually very important scenario" they just get more outraged.
so what factors are left: political and everything else? well, the drug is effective, it's a cure, this describes a lot of drugs. that's what we are talking about, effective drugs. the FDA process has ALSO attacked the importance of market pricing from this angle too, the clinical trials process here and how it has been standardized around the world basically works. so the residual, if it's about the efficacy of the drug, actually does NOT explain the price very much at all. this is counterintuitive!
this is why a CURE for a disease is not 10x more expensive than something that already exists and manages a disease. so...
what's left? my guess would be, the price would be explained by, okay, a lot of it will be political. like 90% political. this is what i mean by "low risk of commercialization." nobody is going to win an election being like, your kid should die.
are you getting it now? you are getting hung up on prices. you don't know the first thing about prices. you think the journey to understanding prices has to do with "$0.25 acetaminophen pill that somehow magically costs $10 in the hospital" and hospital charge lists or whatever, which is 200% wrong, that's a huge red herring.
prices in categories that are low commercialization risk are almost always explained by a political process. which is to say, all the innovations today in commercialization are political. that's bad. but that should also illuminate why, if I were running the FDA, that's where my focus would be - besides, at the rest of the HHS, 10x the basic science R&D budget.
Isn't the usual answer here to try and segment the market and even if your initial guess is horribly wrong price discovery will follow?
> the reason it has been limited to those cases is drug development, today, is constrained by commercialization.
That's a good observation, but I think it's an incomplete picture. Another important constraint is often regulatory inertia and historical baggage.
The UK pioneered small classical and adaptive trials using Bayesian methods, and there were some promising results. A lot of modern Bayesian methodology was, in fact, developed at the MRC BSU Cambridge with this goal in mind. For example, the probabilistic programming language BUGS (1989).
Given that most drugs fail, the industry is highly incentivized to use Bayesian methods to fail faster. These models allow for more rapid dose-finding and the ability to distinguish promising leads using interim data, which is vital given the massive cost of any trial, especially late-stage failures.
But for Bayesian methods to make a dent, they'd need to be applied to a large number of trials, and change doesn't happen overnight. Lots of big pharma players, e.g. GSK, are becoming interested in moving to Bayesian methods in order to leverage prior information and work better within small-data regimes.