Introduction
This blog post (notebook) demonstrates the usage of the Python package “LLMTextualAnswer”, [AAp1], which provides function(s) for finding sub-strings in texts that appear to be answers to given questions according to Large Language Models (LLMs). The package implementation closely follows the implementations of the Raku package “ML::FindTextualAnswer”, [AAp1], and Wolfram Language function LLMTextualAnswer, [AAf1]. Both, in turn, were inspired by the Wolfram Language function FindTextualAnswer, [WRIf1, JL1].
Remark: Currently, LLMs are utilized via the Python “LangChain” packages.
Remark: One of the primary motivations for implementing this package is to provide the fundamental functionality of extracting parameter values from (domain specific) texts needed for the implementation for the Python version of the Raku package “ML::NLPTemplateEngine”, [AAp3].
Setup
Load packages and define LLM access objects:
Here is an example of finding textual answers:
By default llm_textual_answer tries to give short answers. If the option “request” is None then depending on the number of questions the request is one those phrases:
- “give the shortest answer of the question:”
- “list the shortest answers of the questions:”
In the example above the full query given to LLM is:
Given the text “Lake Titicaca is a large, deep lake in the Andes on the border of Bolivia and Peru. By volume of water and by surface area, it is the largest lake in South America” give the shortest answer of the question:
Where is Titicaca?
Here we get a longer answer by changing the value of “request”:
Remark: The function find-textual-answer is inspired by the Mathematica function FindTextualAnswer, [WRIf1]; see [JL1] for details. Unfortunately, at this time implementing the full signature of FindTextualAnswer with LLM-provider APIs is not easy.
Multiple answers
Consider the text:
Here we ask a single question and request 3 answers:
Here is a rerun without number of answers argument:
Multiple questions
If several questions are given to the function llm_textual_answer then all questions are spliced with the given text into one query (that is sent to LLM.)
For example, consider the following text and questions:
Then the query send to the LLM is:
Given the text: “Make a classifier with the method RandomForest over the data dfTitanic; show precision and accuracy.”
list the shortest answers of the questions:
- What is the dataset?
- What is the method?
- Which metrics to show?
The answers are assumed to be given in the same order as the questions, each answer in a separated line. Hence, by splitting the LLM result into lines we get the answers corresponding to the questions.
Remark: For some LLMs, if the questions are missing question marks, it is likely that the result may have a completion as a first line followed by the answers. In that situation the answers are not parsed and a warning message is given.
Here is an example of requesting answers of multiple questions and specifying that the result should be a dictionary:
Mermaid diagram
The following flowchart corresponds to the conceptual steps in the package function llm_textual_answer:
References
Articles
[JL1] Jérôme Louradour, “New in the Wolfram Language: FindTextualAnswer”, (2018), blog.wolfram.com.
Functions
[AAf1] Anton Antonov, LLMTextualAnswer, (2024), Wolfram Function Repository.
[WRIf1] Wolfram Research, Inc., FindTextualAnswer, Wolfram Language function, (2018), (updated 2020).
Packages
[AAp1] Anton Antonov, LLMTextualAnswer, Python package, (2026), GitHub/antononcube.
[AAp2] Anton Antonov, ML::FindTextualAnswer, Raku package, (2023-2025), GitHub/antononcube.
[AAp3] Anton Antonov, ML::NLPTemplateEngine, Raku package, (2023-2025), GitHub/antononcube.