Show HN: Trivai.app – AI powered trivia questions, with references
trivai.appHi HN!
As many of you, I've had a lot of fun playing around with LLMs the past few months and I wanted to show you what I've built.
I made a trivia website using GPT3 a while back just to have something to play with. My initial interest was to see if I could get structured responses to build a UI around, and if I could get the LLM to refer back to what piece of text it used to create questions and answers with.
The initial results were pretty good, but not good enough. Fast forward a few months, GPT 3.5 was released to the public and I was released from my work.
I decided to pick this up again and have been making some changes.
- I've generated almost 3000 questions.
- I initially only let signed up users generate questions due to cost concerns. Since 3.5 is much cheaper, I've opened up the question generation for everybody.
- I built a prompt comparison tool so I could tweak the prompt to get better responses.
- I've added some more data to the questions. This includes a more free form explanation to the correct answer (separate from the references), and categories for all questions.
- I've been working on a question improvement process. This means I collect issues, generate new variations of the question and let people vote on them. Voting and question improvement is currently only available to signed in users.
I have many more ideas I'd like to explore, but I would appreciate your feedback and would be happy to answer any questions about the site or its development.
You can access the references by pressing the explanation text when
you've answered a question correctly. Something is wrong with this question[0]. It looks like the topic is Larry Sanger, one of the choices is Gerry Sanger, the "correct" answer is Jimmy Wales, and the explanation says Lawrence Sanger is the correct answer. Something screwy here. Seems pretty good otherwise Thank you for reporting it! I've actually seen this particular question before.
It's hard to figure out why some questions get bad or wrong answers.
This question also doesn't have a proper reference. I think it's difficult to get the initial result perfect 100% of the time.
So far, my solution to this is an upvote/downvote system.
This lets me either hide downvoted questions, or improve them.
Right now a user who downvoted can report an issue, and I will generate
new variations of the question. If either of the variations receive enough
upvotes, the question will be updated. Have you considered using a database and static questions and answers? This technique gets 100% perfect results each time and needs less energy. It would actually be interesting to know what the energy required by a human is to
research a topic, think of plausible options, find references and type it out. I'm not sure how much energy Trivai requires, or if it's even remotely comparable, but you might be happy to know that all generated questions are stored in a database. I would guess that humans produce "perfect" results 95+% of the time.
In my own tests, Trivai gets "good" results, ~90% of the time. The question is not human vs software, it’s software vs software. The software you use is unreliable. Why would i play trivia against an opponent that is likely as dumb as i am? Check out the pregenerated questions if you just want to see what the questions look like. And press the text that shows up after a correct answer to see
what references the question is based on! Did "20th century British art" and got "Which group of artists in Germany strove for emotional Expressionism?". Something wrong there. Thank you for trying it out! I'm matching the search queries with a Wikipedia page right now, and in this case
it found "20th century art". If you press the explanation "Die Brücke was a group of German artists who strove for emotional Expressionism." You'll see the reference highlighted in the Wikipedia page "Die Brücke strove for emotional Expressionism" I agree that it might be a bit confusing, I'll see if I can do anything to make these situations clearer. Thanks! I entered „John Mayer“ and got three questions. That was a quick game night. Yes I currently only generate 3 questions per article.
I tend to get worse results if I generate more questions per article. I instead try to encourage exploration by showing
related articles (clickable) to try to get a wider ranger of topics instead. I have a lot of work to do to make that whole experience smoother though. Since I posted this, you've generated over 2000 questions, thank you! Interesting, will have a try. Seems like a fun project Thank you! Doesn't work Sorry, what did you try to do and what happened?
