Show HN: StonksGPT – A Natural Language search tool for Stocks and Finance data
stonks.newsQ: "Tell me about Stonks GPT?"
A: "Stonks GPT is a trading name for Stocks & Breads, a UK-based provider of bread and breakfast products. The company was founded in 1894 and is headquartered in London. It is a leading provider of bread products in the UK, with a portfolio of over 50 brands. Stonks GPT is a subsidiary of Unilever PLC, which owns the brands including Pepperidge Farm, Lipton, and Marmite. The company's core business is the production of bread and breakfast products, with a focus on quality and freshness."
There are a lot of hallucinations here. Recommend really specifying in the prompt that if it doesn't know or is unsure, say that and don't make up facts. Understand it's alpha rn and this is all free UAT but for trading related products, accuracy and trust in the product is vital
Is it it even technically for the model to know it’s hallucinating?
You won't catch everything but I've found if you specify that it's okay to respond that it doesn't know the answer and essentially accuracy on a smaller subset of answers > responding to all, it often will say it doesn't know rather than making something up.
Another option could be - assuming this is an LLMChain that adds the most similar (n) embedding(s) to the prompt before passing to LLM - basic entity recognition to find the asset name from the query initially and dynamic prompt that if the most similar results don't contain the asset name, don't provide examples and respond saying you don't know.
So no then
I feel like "hallucinating" is the wrong word. It is essentially predicting the next word based upon its neural net and training. If this training does not have the right data, it will predict things that are wrong.
While knowing might be impossible, it seems like the model could provide a confidence level and only provide answers that exceed some threshold. It'd be a bit like asking a human "are you sure about that?"
And in practice, I really don't think it is that different. We humans effectively make things up all the time. Sometimes we are well aware of our educated guesses and sometimes we are less aware.
It isn't realistic to expect an artificial intelligence to be vastly better than human intelligence in this regard.
The catch is that the models don't have confidence. They cannot distinguish between knowing something for sure, guessing something and hallucinating false knowledge.
Perhaps researchers will find some clever solution to mitigate this, but for now hallucinating is a pretty good word precisely because the model doesn't give confidence.
The Internal State of an LLM Knows When its Lying
It's not surprising that this information is somehow saved in the internal state, but it is surprising that they were able to read it to some level.
I don't think the model can access this information, but an external "lie detector" would be interesting. Thanks for the paper.
that seems wrong.. don't llms runs on probability distributions? Won't they have confidence built in?
I don't think probability of tokens can be mistaken for confidence of knowledge being correct. I'd take it more as a fitness whether the tokens fit naturally in the sentence.
Makes sense, but i wonder if there's some way to retrofit that functionality onto it
How is this #2 on the front page? It doesn't do anything! I typed in a question, no answer. I typed in a public company name, no answer.
Hey! Thanks for trying and Sorry it didn't work for you. I had to either build in vaccum with a lot of guessed training data or release a reasonably working tool, so i opted for later.
Things will definitely improve over time, names and tickers should at least give you a chart though! would love to know your prompts either here or email me at venk at stonks dot news
I entered a very reasonable query, "what stock should I buy (to get rich as hell)", and there were no useful results. Hopefully this will be fixed when it is officially released!
All the stock and finance data one needs is in earnings statement PDFs stored on the regulators website.
Why isn't there an AI product that can pull every PDF data table and provide the key ratio calculations? That's by far the most time consuming thing about analyzing earnings reports.There are computerized services that have done this for years but that costs money and is not AI-powered.
Shouldn't this be the first thing to be automated in the world of investment research?
> Shouldn't this be the first thing to be automated in the world of investment research?
People generally don't like to dig their own information graves so to speak.
Reducing information asymmetries is not the objective of investment research or finance more generally. The only objective is to profitably intermediate financial activity. It generally helps if the clients are not particularly information technology savvy.
Having said that, the obscurity-by-PDF moat looks like its drying up fast.
Shouldn't it be able to get the XBRL instead of the PDFs?
This is under development, We already parse form 4's to generate insider transactions and are going to do the same for 10Q's and 10K's very soon. If you are interested in sharing more feedback on what you would want, please reach out to me at venk at stonks dot news
I hear about so many AI powered stockpicking in the investing space that I am even more firmly convinced that just investing in broad indexes like S&P 500 is the only move left for retail investor. pretty much all the alpha is going to squeezed out even if a small number of these efforts are successful.
I would be shocked if these things outperformed index funds.
AI high frequency traders often outperform index funds. Different kind of AI though. (Although AFAIK some do measure public sentiment through language analysis and use it as one of the inputs.)
> Different kind of AI though.
Yeah exactly lol. You can't just slap GPT into a bot and expect it to outperform the purpose built stuff with a ton of domain knowledge built into it. Those bots exist and are successful because they work. This StonksGPT thing is a crappy product built around an overhyped api call.
With a name like Stonks, the audience isn’t supposed to take this seriously, right?
Yeah but it’s got AI. So you know it’s good.
I love how people continue to use the GPT acronym despite the trademark attempt by OAI, trademark are worthless if not enforced and if the word becomes generic in its usage, so continuing to use GPT to talk about transformer-based generative model is actually harming the trademark grab.
All my requests fail beyond the proposed prompts, and the "random". I asked "show me the chart of SP500 over the last 3 months" and it didn't work. Could you give a rough idea of the scope as of now ?
Thanks for trying and Sorry! It's super early, what works great:
* Screener type queries * Charting(prices, volumes), single company or comparing multiple * Insider transactions from any company * heatmap or top movers today * Earnings data for a certain year and quarter.
There's a bunch of things that also work, but overall we have a long way to go! Please do sign up so we could update you on the next set of features
Thanks for the answer. I signed up and will check it once in a while ! Good luck.
So how does this work?
“Top 10 technology stocks by marketcap” for example.
I can see how it can take technology stocks when looking at the whole sentence and use that for a vector search. How do you pull things like “top 10”and turn that into a query?
SELECT TOP(10) STOCK_NAME FROM STOCKS ORDER BY DESC
Haha, that’s not even SQL.
But I am interested in how it turns the request into SQL (if that’s what it does).
> Haha, that’s not even SQL.
It is in the MSSQL[1] world. I haven't touched it in ~20 years and I can still recognise it.
[1]: https://learn.microsoft.com/en-us/sql/t-sql/queries/top-tran...
Interesting! I haven’t felt with MSSql since the early 1990s. And I didn’t know the TOP(10) even back then I would guess.
Does it also let you not name what you are sorting on?
Typically you feed the schema in as context and literally just ask the model to turn it into SQL. So a prompt of the format "Given this schema {schema}, turn this query into SQL. {query}. Return the SQL and nothing else."
Or perhaps create a lot of “known” queries as templates with good paragraph explanations and use the question to do a search for the known query. I’m new to AI so curious how it’s best utilized.
I would guess a codex llm would do that, but hopefully the question by the user doesn’t end up being a delete statement, lol.
You can either have it show you the statement to confirm before executing it, or if you only want to do queries you can use a read-only database connection.
Asked for a PE ratio graph but didn't work, gave me a price only graph
Hi, congrats on your project. I've noticed some things:
- Since this doesn't seem to be made to deal with long complex prompts, I think enter should submit the query instead of adding a new line
- If I ask for something it doesn't know (e.g. spacex marketcap) I get a message explaining that the AI doesn't know that. So far so good. However, when I type into the box after that, that message changes on each keyDown.
edit: Now it seems to be down (error 429 in the console), but there is no error message in the frontend. There should be some explanation to the user :)
Thank you! I initially had a single line, but on mobile it was very awkward to edit text and considering there's no app, figured its easier to have elongated text. Also, adding support for longer queries, essentially like if you have a hypothesis, you could validate it a bit using this tool!
> However, when I type into the box after that, that message changes on each keyDown.
thanks for flagging, will fix it soon!
I would personally prefer to submit the query with ENTER and add new lines with SHIFT + ENTER, like in e.g. Slack. (Not on mobile though, of course.)
https://biztoc.com also has an approved ChatGPT plugin for finance news.
I tried typing in a ticker symbol "ozk" and it had nothing. I tried typing the full name, "Bank of the Ozarks" and it got nothing. I understand it's looking for natural language, but I would expect business names to pull up something?
Sorry, sometimes its just dumb. it works like this - https://stonks.news/gpt-share/4735/ozk
> show me nancy pelosi's options positions right now
Dang. I wanted to be a billionaire too.
don't have politicians data yet haha! but I plan to add it in the next week:)
Why? Their data is like three months behind. Is that still actionable?
Somewhat. If the options are long-dated, I don't see why they wouldn't be? If Pelosi has calls or puts that are dated a year from now, might make sense to follow the trade if it's a well known company that does business with the government.
Tried your service. Once I sent my question and got the answer it's not possible to enter another request in the search field. Possible a bug ?
Are you on mobile or web? I just checked, once you enter a prompt, you can clear it and then enter a new one. Happy to help look in more detail if you continue to get this error. feel free me to also email at venk at stonks dot news
Hey look, more ways to confirm bias and lose money.
"which stocks listed on the TSX are most closely correlated with real estate prices"
First result was "Plantir (PLTR)"
¯\_(ツ)_/¯
Looks really nice. Also, excellent name.
"compare the earnings of AAP today and ten years ago" didn't really return much.
Interesting idea though.
Currently only have 5 years data :/ , but plan to expand the data set a bit later!
If I was you I would talk to a lawyer to discuss the legal implications of this.
It'd be great if it worked outside of the provided prompts
Q: PE ratio of NVDA A: stock price chart of NVDA...
needs some work folks.
Hugged to death? Console shows: 429 (Too Many Requests)
0 for 5 questions. I gave up.
App doesn't work at all
crypto? dude from mcdonalds said OrangutanCoin is the future!