dale lane

In this post, I want to share a new feature in the generative AI support in Machine Learning for Kids: tool calling.

I wrote last year about how I introduced Generative AI in Machine Learning for Kids by adding support for projects using small language models. And earlier this year, I walked through my six favourite projects for explaining different aspects of Gen AI.

This week I’ve been working on extending the language model support in the site, by adding a new model that is capable of tool calling.

demo video at youtu.be/HdcTseNvjhU

Why is tool calling useful?

Try asking a model what the weather is like in New York right now.

If the temperature and top-p is high enough, many of the models will likely hallucinate an answer.

If the temperature and top-p is low enough, the models can just respond that they cannot answer that.

(If you don’t know why “temperature” makes that difference, I’ve written about that before and have a student worksheet that focuses on this.)

This second answer is more accurate at least, but it is still not helpful. The point is, if you ask the model for something that can’t possibly be represented in the knowledge used to create it, the model cannot give a helpful answer by itself.

Tool calling helps in these situations by making tools available to the model. It can call these tools to help respond to prompts that can’t be answered by the model alone.

Read the rest of this entry »

My commute to work is one of the favourite parts of my day. I’ve mentioned it before, more often than I probably should, but in this post I try and explain why I enjoy it so much.

The journey

Otterbourne village and crossing the motorway

I start off walking through Otterbourne: past the pub, across Otterbourne Hill Green and past the weird mini-castle at Otterbourne Hill Reservoir (I still don’t really understand what that’s all about).

Read the rest of this entry »

In April, I wrote an outline of my career. That post is already out of date, because I have a new job to add to the list.

I’m joining Confluent as a Principal Software Engineer working on Confluent Platform for Apache Flink.

I won’t try and describe what “Confluent Platform for Flink” is here. Partly because I’m sure I’ll have plenty to say about it in coming months. Mostly because I’ve got a lot to learn and anything I say today will almost certainly be incomplete or inaccurate in ways I’ll find embarrassing once I know more. In the meantime, the product page or the docs site are the best places to go if you’re curious.

But that won’t stop me speculating about what this all means for me.

Read the rest of this entry »

In this post, I want to share a new Scratch extension I made today.

It uses the iTunes Search API to let you search for songs, and play 30 second previews of them, in your Scratch projects.

The blocks are simple, and hopefully self-explanatory, but here is a short demo of them in action if it’s not obvious.

I’ve added this extension to the version of Scratch I host for Machine Learning for Kids, so you’ll need to go there to try it out. (To access the extensions library, click on the extensions button in the bottom left.)

Read the rest of this entry »

I gave a talk at Current yesterday about how to embed a tiny language model inside your Flink SQL pipeline.

I used a fun mix of demos to show what I think are the main approaches available for using generative AI with Kafka events from a Flink SQL job. Some demos were definitely more sensible than others!

These are the slides I used, and what I’d planned to say.

#

In this session, I’ll be talking about your options for running language models for Flink SQL jobs.

I’ll cover:

• your options for where you run them, in relation to Flink
• what sorts of choices you have for the models you run
• how to use them – the sorts of prompts and settings we’d want for Flink
• how to keep an eye on it that it’s working well
• and finally, some thoughts on when it’s a good idea to do any of this

Read the rest of this entry »

Metrics can help provide operational insight over Kafka Connect connectors, informing users of how to better configure them. With simple updates, a Kafka Connect connector can be instrumented to make this possible by emitting useful metrics.

A couple years ago, I created a simple skeleton Connect connector project to help developers at a hackathon create their first Kafka connector.

I’ve updated the source connector from that sample to emit metrics. In this post, I’ll walk through what I did, as an example for how to add metrics to your own Kafka connector.

Read the rest of this entry »

A few years ago, I ran a workshop about how to create custom Scratch blocks.

I made a template repository, based on the Scratch Team repos, but with a skeleton extension and some extra scripts and automation to handle building and publishing it. I included step-by-step instructions for building different types of Scratch extensions, including Scratch blocks based on web APIs, and Scratch blocks based on JavaScript modules from npm.

Read the rest of this entry »