Simple Large Language Inference Model
sllim serves as a quality of life wrapper around the openai-python library.
I found myself writing and rewriting the same helper functions with each new project I began, so now I am working to put these functions together into a easy to use library.
Nothing here is ground-breaking; everything here is opinionated.
Usage
Use the chat function to connect with the ChatCompletion.create models. By default, it uses the gpt-3.5-turbo model, but you can pass a model param to use gpt-4
from sllim import chat chat( [ { "role": "system", "content": "Example system message", }, { "role": "user", "content": "Example user message", } ] )
complete works just like Completion.create, and embed is Embedding.create.
Map Reduce
from sllim import map_reduce template = [ { "role": "system", "content": "You are an excellent copy writer and you will rewrite my work into {adjective} words." }, { "role": "user", "content": "Below is my writing, please improve it.\n\n{writing}" }, ] writings = [...] # long list of copywriting for adjective in ["clearer", "more expressive", "fewer"]: # Since `writings` is a list, this is what we will reduce over. # The other variables are treated as constants through the reduction. gen = map_reduce(template, adjective=adjective, writing=writings, model="gpt-4") for idx, result in enumerate(gen): # This is a multithreaded generator to optimize latency to networked services original = writings[idx] print("Was:", original, "\nNow:", result)
Benefits
-
Local file caching. Each of the functions is locally cached in request-response key-pairs to prevent excessive network activity.
-
Auto-retry. Timeouts for rate limits, retry for internal errors (>=500 status code).
-
Parameter names are in the functions so that you don't have to go looking at the docs constantly.
-
Map reduce prompts onto data
-
TODO: Cost estimates before running long tasks
-
TODO: Describe task -> run task
-
TODO: Allow easy estimate
-
TODO: Allow easy logging