LLM function calling workflows (Part 4, Universal specs)

Introduction

This blog post (notebook) shows how to utilize Large Language Model (LLM) Function Calling with the Raku package “LLM::Functions”, [AAp1].

“LLM::Functions” supports high level LLM function calling via llm-synthesize and llm-synthesize-with-tools. (The latter provides more options for the tool invocation process like max-iterations or overriding tool specs.)

At this point “LLM::Functions” supports function calling in the styles of OpenAI’s ChatGPT and Google’s Gemini. If the LLM configuration is not set with the names “ChatGPT” or “Gemini”, then the function calling style used is that of ChatGPT. (Many LLM providers — other than OpenAI and Gemini — tend to adhere to OpenAI’s API.)

Remark: LLM “function calling” is also known as LLM “tools” or “LLM tool invocation.”

In this document, non-trivial Stoichiometry computations are done with the Raku package “Chemistry::Stoichiometry”, [AAp4]. Related plots are done with the Raku package “JavaScript::D3”, [AAp6].

Big picture

Inversion of control is a way to characterize LLM function calling. This means the LLM invokes functions or subroutines that operate on an external system, such as a local computer, rather than within the LLM provider’s environment. See the section “Outline of the overall process” of “LLM function calling workflows (Part 1, OpenAI)”, [AA1].

Remark: The following Software Framework building principles (or mnemonic slogans) apply to LLM function calling:

• “Don’t call us, we’ll call you.” (The Hollywood Principle)
• “Leave the driving to us.” (Greyhound Lines, Inc.)

The whole series

This document is the fourth of the LLM function calling series, [AA1 ÷ AA4]. The other three show lower-level LLM function calling workflows.

Here are all blog posts of the series:

1. “LLM function calling workflows (Part 1, OpenAI)”
2. “LLM function calling workflows (Part 2, Google’s Gemini)”
3. “LLM function calling workflows (Part 3, Facilitation)”
4. “LLM function calling workflows (Part 4, Universal specs)”

Overall comments and observations

• Raku’s constellation of LLM packages was behind with the LLM tools.
  • There are two main reasons for this:
    • For a long period of time (say, 2023 & 2024) LLM tool invocation was unreliable.
      • Meaning, tools were invoked (or not) in an unexpected manner.
    • Different LLM providers use similar but different protocols for LLM tooling.
      • And that poses “interesting” development choices. (Architecture and high-level signatures.)
• At this point, LLM providers have more reliable LLM tool invocation.
  • And API parameters that postulate (or force) tool invocation behavior.
  • Still, not 100% reliable or expected.
• In principle, LLM function calling can be replaced by using LLM graphs, [AA5].
  • Though, at this point llm-graph provides computation over acyclic graphs only.
  • On the other hand, llm-synthesize and llm-synthesize-with-tools use loops for multiple iterations over the tool invocation.
    • Again, the tool is external to the LLM. Tools are (most likely) running on “local” computers.
• In Raku, LLM tooling specs can be (nicely) derived by introspection.
  • So, package developers are encouraged to use declarator blocks as much as possible.
  • Very often, though, it is easier to write an adapter function with specific (or simplified) input parameters.
    • See the last section “Adding plot tools”.
• The package “LLM::Functions” provides a system of classes and subs that facilitate LLM function calling, [AA3].
  • See the namespace LLM::Tooling:
    • Classes: LLM::Tool, LLM::ToolRequest, LLM::ToolResponse.
    • Subs: sub-info, llm-tool-definition, generate-llm-tool-response, llm-tool-request.
  • A new LLM tool for the sub &f can be easily created with LLM::Tool.new(&f).
    • LLM::Tool uses llm-tool-definition which, in turn, uses sub-info.

Outline

Here is an outline of the exposition below:

• Setup
  Computation environment setup
• Chemistry computations examples
  Stoichiometry computations demonstrations
• Define package functions as tools
  Show how to define LLM-tools
• Stoichiometry by LLM
  Invoking LLM requests with LLM tools
• “Thoughtful” response
  Elaborated LLM answer based in LLM tools results
• Adding plot tools
  Enhancing the LLM answers with D3.js plots

Setup

Load packages:

use JSON::Fast;
use LLM::Functions;
use LLM::Tooling;
use Chemistry::Stoichiometry;
use JavaScript::D3;

Define LLM access configurations:

sink my $conf41-mini = llm-configuration('ChatGPT', model => 'gpt-4.1-mini', :8192max-tokens, temperature => 0.4);
sink my $conf-gemini-flash = llm-configuration('Gemini', model => 'gemini-2.0-flash', :8192max-tokens, temperature => 0.4);

JavaScript::D3

#%javascript
require.config({
     paths: {
     d3: 'https://d3js.org/d3.v7.min'
}});

require(['d3'], function(d3) {
     console.log(d3);
});

Chemistry computations examples

The package “Chemistry::Stoichiometry”, [AAp4], provides element data, a grammar (or parser) for chemical formulas, and subs for computing molecular masses and balancing equations. Here is an example of calling molecular-mass:

Balance chemical equation:

'Al + O2 -> Al2O3'
==> balance-chemical-equation

# [4*Al + 3*O2 -> 2*Al2O3]

Define package functions as tools

Define a few tools based in chemistry computations subs:

sink my @tools =
        LLM::Tool.new(&molecular-mass),
        LLM::Tool.new(&balance-chemical-equation)
        ;

Undefined type of parameter ⎡$spec⎦; continue assuming it is a string.

Make an LLM configuration with the LLM-tools:

sink my $conf = llm-configuration($conf41-mini, :@tools);

Remark: When llm-synthesize is given LLM configurations with LLM tools, it hands over the process to llm-synthesize-with-tools. This function then begins the LLM-tool interaction loop.

Stoichiometry by LLM

Here is a prompt requesting to compute molecular masses and to balance a certain chemical equation:

sink my $input = "What are the masses of SO2, O3, and C2H5OH? Also balance: C2H5OH + O2 = H2O + CO2."

The LLM invocation and result:

llm-synthesize(
        [$input, llm-prompt('NothingElse')('JSON')],
        e => $conf, 
        form => sub-parser('JSON'):drop)

# {balanced_equation => 1*C2H5OH + 3*O2 -> 2*CO2 + 3*H2O, masses => {C2H5OH => 46.069, O3 => 47.997, SO2 => 64.058}}

Remark: It order to see the LLM-tool interaction use the Boolean option (adverb) :echo of llm-synthesize.

“Thoughtful” response

Here is a very informative, “thoughtful” response for a quantitative Chemistry question:

#% markdown
my $input = "How many molecules a kilogram of water has? Use LaTeX for the formulas. (If any.)";

llm-synthesize($input, e => $conf)
==> { .subst(/'\[' | '\]'/, '$$', :g).subst(/'\(' | '\)'/, '$', :g) }() # Make sure LaTeX code has proper fences

Adding plot tools

It would be interesting (or fancy) to add a plotting tool. We can use text-list-plot of “Text::Plot”, [AAp5], or js-d3-list-plot of “JavaScript::D3”, [AAp6]. For both, the automatically derived tool specs — via the sub llm-tool-definition used by LLM::Tool — are somewhat incomplete. Here is the auto-result for js-d3-list-plot:

#llm-tool-definition(&text-list-plot)
llm-tool-definition(&js-d3-list-plot)

{
  "function": {
    "strict": true,
    "parameters": {
      "additionalProperties": false,
      "required": [
        "$data",
        ""
      ],
      "type": "object",
      "properties": {
        "$data": {
          "description": "",
          "type": "string"
        },
        "": {
          "description": "",
          "type": "string"
        }
      }
    },
    "type": "function",
    "name": "js-d3-list-plot",
    "description": "Makes a list plot (scatter plot) for a list of numbers or a list of x-y coordinates."
  },
  "type": "function"
}

The automatic tool-spec for js-d3-list-plot can be replaced with this spec:

my $spec = q:to/END/;
{
  "type": "function",
  "function": {
    "name": "jd-d3-list-plot",
    "description": "Creates D3.js code for a list-plot of the given arguments.",
    "parameters": {
      "type": "object",
      "properties": {
        "$x": {
          "type": "array",
          "description": "A list of a list of x-coordinates or x-labels",
          "items": {
            "anyOf": [
              { "type": "string" },
              { "type": "number" }
            ]
          }
        }
        "$y": {
          "type": "array",
          "description": "A list of y-coordinates",
          "items": {
            "type": "number"
          }
        }
      },
      "required": ["$x", "$y"]
    }
  }
}
END

my $t = LLM::Tool.new(&text-list-plot);
$t.json-spec = $spec;

Though, it is easier and more robust to define a new function that delegates to js-d3-list-plot — or other plotting function — and does some additional input processing that anticipates LLM derived argument values:

#| Make a string that represents a list-plot of the given arguments.
my sub data-plot(
    Str:D $x,             #= A list of comma separated x-coordinates or x-labels
    Str:D $y,             #= A list of comma separated y-coordinates
    Str:D :$x-label = '', #= Label of the x-axis
    Str:D :$y-label = '', #= Label of the y-axis
    Str:D :$title = '',   #= Plot title
    ) {
  
    my @x = $x.split(/<[\[\],"]>/, :skip-empty)».trim.grep(*.chars);
    my @y = $y.split(/<[\[\],"]>/, :skip-empty)».trim».Num;
      
    my @points = (@x Z @y).map({ %( variable => $_.head, value => $_.tail ) });
    js-d3-bar-chart(@points, :$x-label, :$y-label, title-color => 'Gray', background => '#1F1F1F', :grid-lines)
}

Here we add the new tool to the tool list above:

sink my @tool-objects =
        LLM::Tool.new(&molecular-mass),
        LLM::Tool.new(&balance-chemical-equation),
        LLM::Tool.new(&data-plot);

Here we make an LLM request for chemical molecules masses calculation and corresponding plotting — note that require to obtain a dictionary of the masses and plot:

my $input = q:to/END/;
What are the masses of SO2, O3, Mg2, and C2H5OH? 
Make a plot the obtained quantities: x-axes for the molecules, y-axis for the masses.
The plot has to have appropriate title and axes labels.
Return a JSON dictionary with keys "masses" and "plot".
END

# LLM configuration with tools
my $conf = llm-configuration($conf41-mini, tools => @tool-objects);

# LLM invocation
my $res = llm-synthesize([
        $input, 
        llm-prompt('NothingElse')('JSON')
    ], 
    e => $conf,
    form => sub-parser('JSON'):drop
);

# Type/structure of the result
deduce-type($res)

# Struct([masses, plot], [Hash, Str])

Here are result’s molecule masses:

# {C2H5OH => 46.069, Mg2 => 48.61, O3 => 47.997, SO2 => 64.058}

Here is the corresponding plot:

References

Articles, blog posts

[AA1] Anton Antonov, “LLM function calling workflows (Part 1, OpenAI)”, (2025), RakuForPrediction at WordPress.

[AA2] Anton Antonov, “LLM function calling workflows (Part 2, Google’s Gemini)”, (2025), RakuForPrediction at WordPress.

[AA3] Anton Antonov, “LLM function calling workflows (Part 3, Facilitation)”, (2025), RakuForPrediction at WordPress.

[AA4] Anton Antonov, “LLM function calling workflows (Part 4, Universal specs)”, (2025), RakuForPrediction at WordPress.

[AA5] Anton Antonov, “LLM::Graph”, (2025), RakuForPrediction at WordPress.

[Gem1] Google Gemini, “Gemini Developer API”.

[OAI1] Open AI, “Function calling guide”.

[WRI1] Wolfram Research, Inc., “LLM-Related Functionality” guide.

Packages

[AAp1] Anton Antonov, LLM::Functions, Raku package, (2023-2025), GitHub/antononcube.

[AAp2] Anton Antonov, WWW::OpenAI, Raku package, (2023-2025), GitHub/antononcube.

[AAp3] Anton Antonov, WWW::Gemini, Raku package, (2023-2025), GitHub/antononcube.

[AAp4] Anton Antonov, Chemistry::Stoichiometry, Raku package, (2021-2025), GitHub/antononcube.

[AAp5] Anton Antonov, Text::Plot, Raku package, (2022-2025), GitHub/antononcube.

[AAp6] Anton Antonov, JavaScript::D3, Raku package, (2022-2025), GitHub/antononcube.