L-Theanine Experiment (1 Month)
doubleblinded.comThis website almost certainly violates FDA rules on human subject experimentation because it collects data on the effect of a drug on human subjects. I would stop working on this project until you have advisors (and lawyers) who have experience with regulatory oversight from the FDA. At the bare minimum you are going to need (a) procedures to gain informed consent from trial enrollees and (b) an Institutional Review Board (IRB) to sign off on your trial. Depending on the drug, you might also need an FDA waiver to conduct the trial.
The PhD student (Rob Wagner) listed in the about section should have had at least one course on the responsible conduct of research. This course is required for most (all?) scientists working under a training or research grant from the NIH. Design and implementation of clinical trials--along with the ethical and legal issues involves--is standard fare for these courses for precisely this reason!
Thanks for the thoughts. We've spoken with a lawyer who specializes in FDA issues. We will have informed consent waivers for participants.
Rob does have formal training in scientific ethical issues and is more than competent on the subject.
Also, L-Theanine is something that you can purchase at the grocery store as a food supplement and it has an excellent safety record and no known toxic dosage.
If it makes you feel better, we are a supplement distributor that happens to help you record whether or not the supplements actually work.
I don't think your platform is exempt from the "common rule" requirement to obtained oversight from an IRB.
The fact that L-theanine is obtainable over the counter is not material. If you study the effects of aspirin on cancer outcomes, you must still obtain consent and IRB review.
A supplements distributor that records whether supplements work is still engaging in human subjects research.
>If it makes you feel better, we are a supplement distributor that happens to help you record whether or not the supplements actually work.
From your website:
>we want to make this clear: we are not a supplement company. We are a personal experimentation platform.
That being said, I actually like your idea. You just need to be more careful about choosing a message (and staying on message).
> If it makes you feel better, we are a supplement distributor that happens to help you record whether or not the supplements actually work.
You are then, by definition, conducting human research.
Cool idea, but as many have pointed out there are some potential issues with the execution.
I'd suggest you get an additional formal legal opinion on applicability of HIPPA as well, it would be financial painful (up to something like $250k/record if I recall correctly) for you to get caught on the wrong side of that one.
You should also (already) have a proper formal ERB.
The other comments are on point. From my perspective as someone in the digital health / clinical research industry, the site (as is) raises a bunch of red flags:
- There's no indication this study has received IRB (Institutional Review Board) approval, which is a baseline. Here's a handy reference when asking the question, "Does my research need IRB approval?": http://www.irb.umn.edu/research.html
Based on several factors, the study/experiment qualifies as research (it meets the requirements for being a systematic investigation, contributing generalizable knowledge, and involving human subjects). IRB review/approval can be fairly straightforward, but does cost a non-trivial amount of money.
Chris, if you need to work with an IRB, I recommend getting in touch with Copernicus Group (http://www.cgirb.com/). I've worked on 20+ projects where they've been the IRB and they've always been responsive and pretty open to innovative ideas.
Echoing rgejman, if you don't have IRB approval, stop everything. Now.
- There's no mention on the site of participants completing an informed consent, which should really be done prior to collecting their money for the kit.
- Has any thought been given to how the team will handle adverse event reporting? This deserves serious consideration and is sometimes overlooked (there are a slew of FDA guidelines/regulations around this). Even for something that seems benign or where "nothing could ever go wrong" - you'd be surprised.
- The site doesn't instill a ton of confidence that data security is a high priority. Using SquareSpace to host the public facing informational portion of the site isn't necessarily a problem, but it leads me to wonder how the team is handling surveys and how the survey data will be stored. It seems like participants will likely provide personally identifiable information (PII) and personal health information (PHI) within surveys, and that's when things start to get a bit more interesting.
It's certainly possible to use an off the shelf survey tool (like SurveyGizmo), but there are a few more hoops you'll have to jump through. Take a look at the following link: https://www.surveygizmo.com/survey-blog/online-survey-hipaa-...
- There doesn't seem to be a privacy policy. The About Us page mentions, "We will have a complete policy online before we start taking orders for the first experiment." The SquareSpace checkout appears to be up and running, but I'm unable to find the privacy policy.
There's a more or less standard process for designing, developing, and getting a study out the door (highly simplified, and skewed heavily towards direct to patient studies): 1. Develop your protocol. 2. Get your protocol approved by an IRB 3. Develop/configure the tech to implement the mechanics outlined in the protocol (this can be done in parallel to IRB review and/or protocol development) 4. Recruit patients 5. Collect informed consent, enroll patients 6. Collect data 7. Everything else...
One of the tricks here is that all of the content a patient might see has to be IRB approved.
Want to run ads soliciting your study? They have to be IRB approved.
Want to optimize your ads? Make sure your content is submitted modularly for review (ie, list all your headlines, body content, and images separately - and throw the kitchen sink at any variations you can think of to save yourself another round of review).
Want patients to pay for their kit via a SquareSpace checkout? Probably requires IRB approval, and likely needs to occur after a participant has completed an informed consent doc.
Need to send password reset emails to patients in the event they can't remember their login to access the study survey tool? Has to be IRB approved.
If you wanted to use efficacy tracking as a marketing tool for your supplement business, you might have a bit more leeway if you were to reposition what you're offering. You could (and there's a good chance I'm wrong as this is off the top of my head) simply offer your customers a tool to track the effects of their supplements. This would also help you with reorders by providing an adherence mechanism.
Unfortunately, you'd likely have to scrub using the Double Blinded site, referring to what you're offering as an experiment, or mentioning that the results will be aggregated and published. Data protection issues would still be a serious consideration though.
All of that said, the concept of citizen science and open/transparent studies is very cool. It's just a (minor) hassle to dot all the i's and cross all the t's.
Also, brilliant idea to offer customers a way to track efficacy. I've been toying with similar ideas, so it's cool to see someone else implementing something.
This is an incredibly informative and useful post. Thank you.
Thanks! Digital health can be incredibly fun and rewarding, but there are plenty of opportunities to run head first into walls you may not have realized even existed.
Aside from the ethics considerations, privacy regulations (and the desire to tread conservatively) can lead to some odd implementation decisions...especially if you're coming into the industry from a general or consumer oriented tech background.
I once worked on a medication adherence project for patients in Italy where patients needed to complete bi-weekly assessments and generally track their medication adherence. We set up a secure portal/website for patients, but were prohibited from emailing them reminders when their next assessment was due.
The sponsor (aka pharma co) was extremely conservative re: patient privacy laws and more or less left it at, "well, the patient will just have to remember to return to the website, log in, and complete their survey at the appointed time." After a couple of months of sub-optimal survey completion rates, they re-evaluated their position and let us send out survey reminder emails.....which is a baseline thing to do when you need to retain participants over the life of a study.
Oddly enough, this was for a commercial project (medication was available on the market, physicians were referring their patients to this website as a value added sort of service) and the commercial side is a bit more wild west than phase 2/3 research. Well, as wild west as the pharma/clinical research industry gets :)
There are parallels between customer acquisition/retention in SaaS and patient recruitment/retention in clinical trials that are pretty interesting...but I think most folks on the Saas side would think the healthcare folks are trying to solve problems with one (if not two) hands tied behind their backs - and in some cases, wouldn't be wrong in thinking so.
> If it makes you feel better, we are a supplement distributor that happens to help you record whether or not the supplements actually work.
I liked that statement a lot, at least I think it is excellent marketing!
As an aside: it's a cool idea and worth trying--but you are going to need to make sure that you are compliant with this country's laws regarding the responsible conduct of human subjects research!
I love the idea, but the execution seems less than ideal. Besides the research ethics question posed in another comment, what is the dosage of L-Theanine in the capsules? What kind of responsibility does the company take for possible side effects? I understand the side effects should be mild or non-existant in the experiment for L-Theanine, but just a discussion around that issue would be nice.
Again, I want to stress that I love the idea, but the web site just feels incomplete.
What kind of responsibility does the company take for possible side effects?
Probably the same kind of responsibility taken by green tea manufacturers.
Green tea sellers don't make medical claims for their products; green tea sellers don't enroll people in "research".
We make no medical claims about our product either - we only reference existing science.
"[...] and is known to promote relaxation without sedation. It appears to be effective at this as well as reducing stress at standard dosages."
I would read that as a medical claim, it doesn't matter if you quote another site or not.
The claim about stress seems to be based on experiments in rats and a single study with 12 participants.
This made me curious so I checked the FDA site. In case anyone else is interested:
From a very cursory review, it appears that health claims on the packaging are quite heavily regulated by the FDA[1], but advertising for dietary supplements and foods seems to fall outside their mandate and regulatory responsibility lies instead with the FTC:
"The Federal Trade Commission (FTC) regulates advertising, including infomercials, for dietary supplements and most other products sold to consumers. FDA works closely with FTC in this area, but FTC's work is directed by different laws."[2]
I'm not sure what's on the bottle, but presumably the websites falls under the jurisdiction of the FTC. A brief review of their site seems to indicate a far more relaxed approach is taken. For example:
"The FTC's standard for evaluating substantiation is sufficiently flexible to ensure that consumers have access to information about emerging areas of science"
"These factors are often weighed together to ensure that valuable product information is not withheld from consumers because the cost of developing substantiation is prohibitive"
"...the FTC gives great weight to accepted norms in the relevant fields of research and consults with experts from a wide variety of disciplines, including those with experience in botanicals and traditional medicines" [3]
[1] http://www.fda.gov/Food/IngredientsPackagingLabeling/Labelin...
[2] http://www.fda.gov/Food/DietarySupplements/QADietarySuppleme...
[3] https://www.ftc.gov/tips-advice/business-center/guidance/die...
Fair enough. My impression was that this was the first of many similar experiments, but with different substances making my question perhaps more valid somewhere down the line.
This is the idea - we'll have to address the safety and ethical concerns of each substance individually.
And, if you're repackaging the supplement in capsules, do you need to be operating under whatever equivalent the supplement industry has to GMP conditions?
I tried taking L-Theanine for about 5 days. 30 minutes after taking it each day, I would get very nauseous and usually throw up. I tried varying the dosage from 200mg down to 100mg and then 75mg, I would still get the nausea, no matter if I had an empty stomach or not. I also get the same symptom when drinking more than a cup or so of green tea, and has for years (though I do remember being able to stomach the stuff in the past). It would be interesting to hear if this happens to anyone else.
I get the same effect from drinking Starbuck's "Awake" or black tea, whatever they call it now. An empty stomach usually makes it worse.
That's interesting. Now that I think of it, the nausea feels exactly like what you get from ingesting the tannins in overbrewed black tea. I just assumed my nausea from drinking green tea and from L-Theanine (which I know that green tea contains in a non-negligible quantity). Maybe levels of tannins in green tea are higher than in black tea? In that case the nausea from L-Theanine may be unrelated. Maybe it's a combination of both.
I think you might be on the right track, but I drink their green tea now and I don't get the same effect. I also drink overbrewed Twinings english breakfast tea with no negative effect.
I got side effects from some medication I took until one day I was about to take them and then forgot, still got the side effects, and finally found I hadn't taken the meds after all.
At that time I concluded it was all nocebo and the side effects were almost completely gone from that day on.
Interesting. I've never heard of someone reacting to L-Theanine this way. Perhaps it could be an allergy?
Yikes! I'm a bit disappointed and slightly surprised that there's no discussion nor consideration of research ethics appearing anywhere on that website.
You bring up a good point! We will be adding some outlines about ethics soon. Thanks for reminding us of the importance!
I'm sorry, you have a website called doubleblind, you are asking people to pay to participate in the study, and you need to be reminded of the importance of thinking about ethics?
There's a reason why science and medicine don't move the same way Web 2.0 does (ie move quickly and break things). It's because people have died from poor implementations, and people trialing drugs are prone to exploitation from unscrupulous merchants.
Now I'm not saying there's anything wrong with your study, or your proposed aims. But if you want to be taken seriously and not shut down by whatever regulatory authority governs your territory, you need to think long and hard about this on both this study and any future studies/products you intend to sell.
Do not screw this up, and do not make the mistake of approaching human health and clinical trials as though something critical like ethics review is something that can be 'patched' further down the line after the release date
We chose L-Theanine because it has an excellent safety record, is well studied, and has no known toxic dose. We are very aware of safety and ethics. There are many nootropics and supplement companies safely distributing the same L-Theanine.
While we work through our protocol we will continue to dive into ethics and safety.
What you have stated here you have stated in your previous thread announcing doubleblind and presumably elsewhere. I don't have an issue with what you are choosing, or why. In fact, it doesn't really matter.
What matters is that you have a clear and transparent ethics approval process, that you clearly state the steps your protocol had been through, and preferably that you submit your protocol to a human ethics review board, and identify that process openly and transparently.
It sounds like you have already developed your protocol, since you are offering your product for sale shipping from the start of this month. So have you written an ethics proposal? Have you considered submitting it to a HERC? Or are you 'continuing to dive into ethics and safety' now that you have been 'reminded of the importance' of it?
Don't get me wrong, I think your idea and platform is potentially great. But it really seems that you haven't thought about this, and now you're trying to scramble?
If you don't have a science background and aren't partnering with anyone who is, I'd be more than happy to lend my 2c outside of this thread (see profile). Otherwise, I wish you every success with a full and detailed examination of your ethics approach. Medical research (which you are conducting by calling your product an experiment) has a poor history when it comes to blurring the lines between researcher and pharmaceutical company/monetisation, of which you could be said to be doing both at the moment.
You keep saying this like it means something. Even _water_ has a toxic dose:[L-Theanine] has no known toxic dose.http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1770067/
It really makes me wonder whether you have adequate human subject protections in place or if you just hand-waved those away by assuming L-Theanine was completely safe.
This part, for what it's worth, is ass backwards.While we work through our protocol we will continue to dive into ethics and safety.
Research ethics must be a central concern, not something hand-waved away with an outline at a later date.
Maybe I'm getting a bad impression from your current website and comments, but appearances matter. The stereotypical "move fast and break things" attitude is wholly inappropriate here; human and animal research must be conducted with careful consideration and respect.
Interesting concept, shame it's limited to the US (understandable due to dubious legality of shipping unclassified chemicals between jurisdictions).
Been interested by the apparent 'attention enhancing' qualities of L-Theanine, although not gone as far as purchasing any. By coincidence someone got me into drinking green tea a couple of weeks ago, I haven't noticed any visible change. Although I expect my problem is concentrating on the right thing, rather than simply failing to concentrate.
From my personal experience, L-Theanine, coupled with caffeine, produces very serious results in terms of negating bad coffee effects, increased focus and productivity, and better sleep
Same here. A couple of pots of strong sencha (Japanese green tea) give me a huge boost in energy and mood, without the negative effects of derealization, tunnel vision and restlessness I get from too much coffee.
Maybe this actually part of a study to research whether having your research subjects pay you (rather than the other way around) has an effect the results of a study.
Could you elaborate?
Maybe they are taking the money that people pay him on this side and paying some other research subjects to do the same thing and is going to compare the results.
It's weird to ask people to pay to be part of a study. So maybe that is actually part of what this person is studying.
Maybe all the capsules are placebos and they are studying something else.
Can you tell I like conspiracy theories? :)
Oh I just noticed you are the person who posted this. You should do that, if this isn't what you're doing.
Hey! I've been taking L-Theanine for the past month. It feels to me like I've been waking up with a clearer mind, but it's super hard to know if that's form the l-theanine or just in general the change in the weather. So many variables.
Indeed it's difficult to tell - exactly the reason why a double-blinded trial on yourself can help you determine if it's making a significant difference or not :)
Show HN when the idea was presented: https://news.ycombinator.com/item?id=10401698
Looks promising. While it may seem obvious, I would spell out what double-blind means, the problem it solves, and how it is used elsewhere. That then sets you up to point out how doctors use these study types, adding credibility.
I absolutely love this idea. Are you gonna do a trial every month? Any schedule for which trials you're gonna run?
We're going to keep adding more trials - but there's no timeframe for when you have to start and end the trial. You can order it and start it whenever you receive it in the mail.
I also love the idea, would really like to try it! Any chance to get it shipped outside the US (I live in Sweden)?
Not yet :( Like Steer said, you can still order Theanine independently of our company but I can let you know when we are ready for international orders if you send me your email!
You can order L-Theanine (L-Teanin) in Sweden without any issues. You will just have to figure out the double-blind part.
How do you propose to analyse the data you collect?
I'm guessing that they're more interested in the collected cash. It's all about the chee$e!
cool concept. interested in seeing the results.