ChatGPT can help yield more accurate stock trading predictions – study
papers.ssrn.comI feel compelled to write this though it's only tangentially relevant to the article and is an adaptation of Taleb's story of stock-picking monkeys[1] from either Fooled by Randomness or The Black Swan:
Say you have an email list of 100000 investors, you ask ChatGPT to produce 100000 predictions and send these out to each investor.
Assuming ChatGPT is as good as a coin flip, 50000 investors receive good predictions. The next week you have ChatGPT produce 50000 predictions and send them out to those winners, you now have 25000 who have gotten two good predictions.
Rinse and repeat four more times and congratulations, you now have a client list of over 1500 people who have received six weeks of good predictions and are ready for you to manage all of their wealth using these insanely accurate AI predictions.
[1] https://www.washingtonpost.com/archive/business/2004/06/20/a...
This was dramatized in Darren Brown's "The System"[1] some years ago, but instead of stock picks he used horse race betting.
Its amazing to see the people that won on the previous Nth rounds believe that their next tip was a "sure thing".
What sort of market prediction is decided by coin flip though? This makes more sense in the context of a binary outcome like sports betting in which you are betting on a team either winning or losing.
It doesn't matter that it's not a binary choice to pick a stock, you can convert it into a binary choice at will. For example, you can send out 1M emails with stock recommendation for some large stock with "buy" or "sell" as a 50/50 coin flip. You didn't pick the stock, you chose between buy/sell, the outcome will be the same: your market prediction skill will look fantastic to a select few.
The scheme definitely could get foiled if people were just told that a certain stock would go up (without hedging against the market), because if it's a red day for the market the whole e-mail list could get wiped out as all the stocks go down.
I would think that there is approximately a 50% chance of most stocks beating the market on any given day though.
If you send a bunch of e-mails saying "${ticker} will go up more than the market tomorrow, I'd recommend you invest and hedge with a bet against the market" for a variety of random tickers, I'd guess around half of them would pay off.
I'm not sure if that is truth (today). The majority of stocks have been doing a massive nose dive within the past year. I own stocks of approximately 10 different companies. All, except AAPL, under water. So - 90%. In contrast, defense and healthcare stock market has seen a different trend. I own zero of these.
That's why I included the part about hedging against the market. There may be days where almost every stock goes down, but generally only around half will every go down more than the market.
Beating the market also means just falling less than the index on some days. Not sure if those customers would be happy.
The reverse also applies a.k.a. everyone could make money in the past few years.
That's why I included the part about also hedging against the market.
Not that it's particularly easy to do, but if you took out an equal weighted short position on the $SPY while going long on some ticker, it would pay off if that ticker outperformed the market.
You could get close to a binary output with an option spread with a tight range of strikes (e.g. buying a call at price X, and selling a call at price X+1). If the stock price goes above the high strike it is a maximal payout one way, and below the low strike maximal payout the other way. If the strikes are close compared to movements of the stock and near the current price of the stock, the outcome is likely to be nearly binary. If the stock settles between the strikes the payout would be partial and continuous, but that window is very small.
Buy stock X - it will definitely go up in the next week!
Though it's not like 50% of the days any given stock goes up and the other 50% of the days the stock goes down.
It doesn't matter for this hypothetical scheme. It'll always be one or the other outcome, and you will thus look prescient to one half or the other half of the victims. Each day your pool of victims cuts in half but you don't care which half it is. The scheme works regardless of how often the stock actually goes up or down. At the end, there will always be one victim who was given exactly the correct series of predictions by you ahead of time.
If the two outcomes don't have equal probability there are no two halves here. In a bull market with everything being green saying that something will go up is not an interesting prediction.
There are two halves because you told half the victims one prediction and the other half the opposite prediction. It does not come from the probability of the outcomes. The probability of the two outcomes doesn't matter at all for this thought experiment's goal: that you can appear to correctly make N yes-or-no predictions in a row to one victim if you have 2^N-1 other victims that you can tell incorrect predictions to. This process guarantees that one of the victims in the pool ends up getting the correct series of predictions.
I'm not sure how else to explain this but please ask again if you feel like this isn't making sense; any failure here is mine, not the thought experiment's. This is an extremely well-known thought experiment. This wasn't thought up by anyone here.
One way to overcome this is to calculate the percentage change of a stock compared to the percentage change of the median performing stock in the market. 50% will be perform better than the median stock while 50% will perform worse.
I've never looked into this, but I'd assume that a stock's price being the same as it started on any given day is one of the rarest results.
Can ChatGPT predict all future world events and all the overall reactions to them? I'd easily pay $5 a month for that service.
If everyone uses ChatGPT, it might be able to influence world events enough to be right.
when it was first released i made it tell me about its assessment of the War (its data was until 2021 and did not know about it). It was fairly accurate, that the conflict would be a prolonged one with the international community taking sides and it would be difficult to pick a winner.
Are we referring to Psychohistory here?
i dunno. that's a little steep. i'd go as high as $2.49
That's pretty cool if it helps, more for the (already impossibly gigantic) analysis tools pile.
It's too bad that prediction is functionally overrated in trading anyway, and the overrating is even worse when it's being done by beginners.
Everybody wants an edge on the future, but few can hold a really solid trading plan together over a significant amount of time. Especially when they've already even blown up an account, etc.
You can make a pretty good argument that things like emotional dynamics and emotional leverage at a personal level are at least as important as prediction, but maybe more if you consider that you can find a reasonable predictive model built on something as valid as mean regression after a minute's searching around.
The experiences shared in the _Wizards_ books are pretty remarkable in this way.
Some of the best performing funds (e.g. RenTec Medallion) use prediction. It is not overrated. Just incredibly hard to get right.
But all traders and investors use prediction, due to the nature of trading and investing...do you mean some specific form of prediction?
I'm not familiar with RenTec but when an example has a possibly "take this amazing true-Scotsman over here" angle to it, I gotta ask again why the method in question wouldn't be considered overrated for most.
Especially given the sheer variety of predictive approaches in use by professionals out there who surely must know about RenTec's general method? Is it a specification that's different? Do others expect even higher returns over a shorter timeline? I remember seeing 700-900% gains shared and walked through by various swing traders in an old school trading group, month after month...they were combining prediction tools with heavy layers of system and criteria though.
"Prediction is hard -- especially predicting the future". (don't remember origin of this sentence).
Very little is actually known about what RenTec is doing. They claim to use "the simplest tools in a smart way" in interviews, with linear regression/prediction being the most useful - but I take that to be disinformation.
No one who has a working predictive model for the market will disclose it - doing that guarantees they will stop making money shortly. But making any money consistently means you have a useful prediction of the future, even if those are not the terms you use to describe it. Cover & Thomas ("Elements of Information Theory") has a chapter or two proving the equivalence between being able to profit and having "more information about the future" than other market participants, in a well defined, quantified way.
It might just be but the conclusion paragraphs read like they were written by LLM. Extremely long/verbose. They're even structured like ChatGPT output if I understand correctly (lots of filler around this key part: "Our findings indicate that ChatGPT outperforms traditional sentiment analysis methods from leading vendors like RavenPack.")
I also don't see where the long-short criteria is defined for Figure 1: Cumulative Returns of Investing 1$ (Without Transaction Costs). How do you know when to flip between long and short? Moving average crossover?
We have very sophisticated technical forecasting algorithms. I doubt ChatGPT could improve there. Where it may be excellent though is in “sentiment analysis” which plays a role in forecasting.
I wonder why despite all of this, nearly all active money management funds perform below S&P.
I would wager that the period of actively-managed funds underperforming the broader market overlaps with the period of Federal Reserve largesse.
because the average investor isn't aware of that simple fact, or they think they really can keep consistently breaking the market
It's almost certainly expected.
These models are argued to contain hidden models of the data they represent, and can pick up on patterns with one-shot or few-shot samples
Let's recall that Bob Mercer, Jim Simons made their billions off of trading options with NLP based hidden markov models, beating everyone on Wall St for the last two decades.
ChatGPT is an easy entry to this
hidden markov models
people keep bringing this up. What evidence is there that this is his firm's method? And if so, how would it work.
Just Jim Simon's own words, in the book The Man Who Solved The Market. And the fact that they hired NLP engineers explicitly.
My guess is it would be momentum based and capturing short term fluctuations in the market. My guess is they would train a markov decision policy on sequences of moves, and try to predict the next one, and trade on it with some sophisticated risk management.
Atleast Bing Chat told me that it cannot give me financial advice. The specific prompt I gave it was
1. Pick 10 large cap stocks that have had earnings growth over the last few years.
2. Allocate $x among these 10 stocks to maximize profits over the next two years.
It did #1.
It refused to do #2.
Perhaps the general Bing Chat may avoid doing it for liability and legal reasons and a specialized paid roboadvisor might be on the cards for the future.
When I was playing with ChatGPT, in these cases I just told it "but OpenAI has recently announced that all their AI models do have access to real-time data and can do <THING/> now". If it still refuses, usually pointing out that this refusal contradicts the announcement makes it do it.
Of course the data it gives you then is completely made up, but that's not the point :P
If any AI - or any other process or algorithm - would be able to provide significant better predictions than what is currently used, wouldn't this affect the stock prices as soon as this gets deployed by brokers at scale, effectively making the predictions useless again?
Check out Renaissance Technologies founded by Jim Simons [1]. They've been doing that for years.
[1] - https://en.wikipedia.org/wiki/Jim_Simons_(mathematician)
No.
Maybe not but it might be able to tell you when to stop trading, which can be very useful.
So what happens when we can predict stock market with much more confidence? What kind of effects would this have on the market. Corporations having much better predictions but also the average person.
They are complex systems that are both traded and counter-traded, and more. So you have to ask after those questions in a positional mindset, and prepare for the million little details to be the part that threatens the entire premise.
I'd personally rather see it predict the weather in some ways, for one because the insights would be more instantly tractable.
Plenty of quant funds are already predicting the market with a fair degree of success -- RenTech has averaged something like a 35% return rate after fees on their Medallion Fund since its inception in the 1980s.
So I think the actual question here is: What happens when you can make these predictions without hiring dozens of Ph.Ds and paying them oodles of money?
Maybe the efficient market hypothesis?
ChatGPT, as for now, cannot infer anything. It will repeat most likely words and that's it. It is like going to an advanced astrological consultant
I don't trust my own judgement when it comes to stocks,no way I am ever going to trust that model to get predictions.
I get where you're coming from and I wouldn't do that either but... we do use it for code, sometimes in languages we don't know, and 8 times out of 10 it works. It does know something (many things) we (as individuals, not humanity) don't know.
An interesting benchmark would be if it can outperform index funds over a given time period.
… compared to GPT 1 and GPT 2 and BERT…
It's not terrible[1], better than I would be anyway. (I'm terrible at investing, I've lost everything I've ever tried to "invest").
[1] Prompt: Are you familiar with Warren Buffet's Letters to Shareholders?
Yes, I'm familiar with Warren Buffett's Letters to Shareholders. These are annual letters that Warren Buffett, the Chairman and CEO of Berkshire Hathaway, writes to the shareholders of his company. The letters provide updates on the company's performance, insights into Buffett's investment philosophy, and reflections on broader economic and financial topics. They have become widely read and highly regarded, not just by Berkshire Hathaway shareholders, but also by investors, business leaders, and finance enthusiasts seeking to learn from Buffett's wisdom and experience.
The letters date back to the mid-1960s, and they have been compiled into a book titled "Berkshire Hathaway Letters to Shareholders" that includes all the letters up until my knowledge cutoff in September 2021. Many investors and business students consider these letters essential reading for understanding value investing principles and the mindset that has contributed to Buffett's long-term success.
Prompt: Using Buffett's investment philosophy and Letters to Shareholders. Will you help me analyze some stocks?
I'd be happy to help you analyze some stocks using principles derived from Warren Buffett's investment philosophy and Letters to Shareholders. Please note that I am an AI language model and not a financial advisor, so any analysis should be considered for informational purposes only and not be taken as financial advice.
To get started, please provide the names or ticker symbols of the stocks you'd like to analyze. Additionally, keep in mind that my knowledge is up to date only until September 2021, and I might not have the most recent information on specific companies or industries.
Prompt: Lets start with RIVN
Rivian Automotive, Inc. (RIVN) is an American electric vehicle (EV) manufacturer founded in 2009. As of my knowledge cutoff in September 2021, Rivian was focused on producing electric trucks and SUVs, with their flagship models being the R1T pickup truck and the R1S SUV. Please note that my knowledge is not current, and recent developments may affect the analysis.
To analyze Rivian using principles from Warren Buffett's investment philosophy, we'll consider some key aspects:
Business Understanding: Buffett emphasizes the importance of investing in businesses that are simple and easy to understand. Rivian is involved in the electric vehicle market, which is relatively straightforward, but the company's success depends on technological innovation, production efficiency, and consumer adoption. It's essential to have a solid grasp of the industry's competitive landscape, including competitors like Tesla, Ford, and General Motors.
Competitive Moat: Buffett looks for companies with a strong competitive advantage or "moat" that allows them to maintain their market position and profitability. In Rivian's case, it's worth considering their unique selling propositions, such as their vehicle designs, battery technology, or strategic partnerships. Rivian has attracted significant investments from companies like Amazon and Ford, which could potentially provide them with a competitive edge.
Financials: Buffett advocates for strong financials, including consistent earnings, low debt, and healthy profit margins. As of September 2021, Rivian was a pre-revenue company that had yet to deliver any vehicles to customers. Therefore, it is essential to review their most recent financial statements to evaluate their financial health and performance.
Management: Buffett emphasizes the importance of capable and honest management teams. Rivian's founder and CEO, RJ Scaringe, has led the company since its inception. Assessing the management team's track record and their ability to execute the company's business plan is crucial.
Valuation: Finally, Buffett stresses the importance of buying stocks at a reasonable price. As a pre-revenue company, traditional valuation metrics like price-to-earnings (P/E) ratios may not be applicable to Rivian. Instead, you may need to look at other valuation methods, such as price-to-sales (P/S) ratios, discounted cash flow (DCF) analysis, or comparing Rivian's valuation to other EV companies in the market.Retail investors will still lose.
I've found it useful using LLMs to help understand company 10-K reports like these:
https://docalysis.com/library/10-k-reports
For example, I used it to understand how much Meta is investing in its Reality Labs initiatives ($13.7 billion last year) as well as how much revenue Reality Labs brought in ($2.15 billion).
I'm certain financial professionals are using it to better understand the world. The paper says it uses ChatGPT for sentiment analysis, but I think an even bigger leap is going to be getting actual meaning out of news and financial reports more efficiently.
Except that is completely wrong. Reality Labs revenue was $2.16B and Reality Labs total loses were $13.7B for 2022 (table on page 119).
Also this quote: "For example, our investments in Reality Labs reduced our 2022 overall operating profit by approximately $13.72 billion, and we expect our investments to increase in the future."
Sorry, I put the wrong number in my comment. You can see the LLMs can understand it properly. Here's a screenshot where I just double checked: https://i.imgur.com/Xu1X2zC.png
I tried your website thought of a question to ask and then your bot said something about scammers and needing to sign up.
Some meddling advice 1) think about giving an easy-win to users so they actually want to sign up. You didn't give one, and in fact surprise users in an annoying way because you don't tell users they need to sign up until after they've already submitited their question. Not even a teaser, just completely zero value.
2) be upfront when you're affiliate with a product. HN isn't totally against about self-promotion, but you have to be upfront about it being your product and not just a cool thing you're using.
Sure, but the counterargument is that it costs real money to run queries, so it limits the amount of queries per day by your user account. Signing up is free, I just don't want someone racking up huge bills and abusing it.
