Kritikos
Kritikos is a project aiming to enhance the quality of responses generated by large language models (LLMs) using the LLM-as-a-critique method: a technique that involves iterative generation and critical evaluation of answers by specialized models.
Table of Contents
- π Getting Started
- How it works
- Project Structure
- API Usage Example
- License
- Contributing
- Additional information
π Getting Started
First of all, thanks to @paologaleotti for his template Blaze, in the repository you will also find instructions for a backend deployment.
1. Clone the project
git clone https://github.com/michelelacorte-quinck/kritikos.git
cd kritikos2. Set up environment
Copy the .env.template and fill in your credentials:
3. Start the server
To get started, simply run:
This will build the project, you just need Go installed and nothing else.
To run the server:
This will start the server on port 3000.
How it works
The goal of Kritikos is to have an off-the-shelf backend that is capable of handling LLM requests that improve automatically.
The system provides the ability to select the model for the βbasicβ response and the model for the βevaluativeβ response. (see baseModel, evaluationModel)
The first response is generated with a default temperature of 0.9 (baseModelTemperature) and then passed to the critique model, which starts with a temperature of 0.5 (evaluationModelTemperature) that is lowered by 0.1 with each iteration to encourage more and more consistent responses.
Project Structure
kritikos/ βββ cmd/api/ # API entry point βββ internal/api/ # Router, handler, controller βββ .env.template # Environment variables βββ Makefile # Utility commands βββ go.mod / go.sum # Go dependencies βββ README.md
API Usage Example
POST /ai/kritikos
{
"systemPrompt": "You are an agent who helps people create DIY projects",
"prompt": "Could you tell me how to do a chair?",
"baseModel": "gemini-2.0-flash",
"evaluationModel": "gemini-2.5-flash",
"baseModelTemperature": 0.9,
"evaluationModelTemperature": 0.5,
"maxRetries": 3
}First Response (no kritikos):
Okay, let's get you started on building a chair! To give you the best guidance, I need a little more information about what kind of chair you want to build. There are many different chair types, from simple stools to more complex armchairs. To help me tailor instructions, please tell me: **1. What kind of chair do you want to build?** Here are some options, or describe your own idea: * **Simple Stool:** A basic seat with legs. * **Basic Dining Chair:** Four legs, a seat, and a back. * **Folding Chair:** A chair that can collapse for easy storage. * **Adirondack Chair:** A classic outdoor chair with a slanted seat and back. * **Something else?** Describe the chair you have in mind. **2. What is your skill level?** * **Beginner:** Little to no experience with woodworking or DIY projects. * **Intermediate:** Some experience with tools and basic construction. * **Advanced:** Comfortable with complex cuts, joinery, and power tools. **3. What tools and materials do you have available?** * **Tools:** Do you have a saw (hand saw, circular saw, jigsaw, miter saw), drill, screwdriver, measuring tape, sander, etc.? * **Materials:** Do you have any wood already? Are you planning to buy new wood? Once I have this information, I can provide you with a more detailed and helpful guide. **In the meantime, here are some general considerations for building a chair:** * **Design:** A good design is crucial. Look at existing chairs for inspiration. Consider the dimensions (seat height, seat width, back height, etc.) to ensure comfort. * **Materials:** Wood is the most common material. Choose a wood that is appropriate for your project (pine is inexpensive and easy to work with, hardwood like oak or maple is stronger and more durable). Consider plywood for certain parts. * **Joinery:** How will you connect the pieces of wood together? Screws, nails, glue, and more advanced joinery techniques (like mortise and tenon) are all options. * **Stability:** A chair needs to be stable! Pay attention to how the legs are attached and consider adding bracing to prevent wobbling. * **Finishing:** Once the chair is built, you'll want to sand it smooth and apply a finish (paint, stain, varnish, etc.) to protect the wood and enhance its appearance. I look forward to helping you build your chair! Let me know the details!
Full Response (with kritikos):
{
"scores": {
"relevance": 5,
"correctness": 4,
"completeness": 4,
"clarity": 4,
"style": 5
},
"strengths": [
"The draft effectively manages user expectations by first asking for clarifying information before providing a specific plan.",
"It offers a concrete, well-structured, and beginner-friendly chair plan as an immediate example.",
"The tone is consistently encouraging, helpful, and perfectly aligns with the persona of a DIY project assistant.",
"The initial questions cover key aspects like skill level, budget, style, and tools, which are crucial for tailoring DIY advice."
],
"weaknesses": [
"The material list and assembly instructions for the example chair were somewhat ambiguous and could lead to confusion or an unstable build.",
"The terminology used for the structural components of the example chair (e.g., 'side supports') was not as precise as it could be, making it harder to visualize the construction."
],
"actionableAdvice": [
"Clarify the specific dimensions and purpose of each structural lumber piece for the example chair (e.g., distinguish between front/back aprons and side aprons).",
"Refine the assembly instructions for the example chair to clearly describe how the frame is constructed from legs and aprons/stretchers, ensuring a stable and standard design.",
"Ensure the material list for the example chair is consistent with the assembly steps and provides accurate quantities and lengths for each component."
],
"improvedAnswer": "Building a chair is a fantastic DIY project! To give you the most helpful and specific instructions, I need a little more information about what you have in mind. Understanding your preferences will help me tailor the best guide for you. Please consider these factors:\n\n**1. What's your skill level?**\n\n* **Beginner:** Simple, sturdy designs are best. Think basic wooden chairs with minimal joinery.\n* **Intermediate:** You're comfortable with more complex cuts and joinery (like mortise and tenon).\n* **Advanced:** You're ready for complex curves, upholstery, and intricate designs.\n\n**2. What's your budget?**\n\n* **Low-Cost:** Repurposed materials, simple lumber, and basic hardware.\n* **Mid-Range:** Allows for higher-quality lumber and nicer hardware.\n* **High-End:** Exotic woods, custom upholstery, and specialized hardware.\n\n**3. What style of chair are you looking for?**\n\n* **Simple Wooden Chair:** Classic, sturdy, and relatively easy to build.\n* **Adirondack Chair:** Comfortable outdoor chair with a distinctive sloping back.\n* **Modern Chair:** Sleek lines, minimalist design, and often uses metal or molded materials.\n* **Upholstered Chair:** Adds comfort and style but requires upholstery skills.\n* **Folding Chair:** Portable and space-saving.\n* **Rocking Chair:** A relaxing classic.\n\n**4. What tools do you have available?**\n\n* **Basic:** Saw (hand saw or circular saw), drill, screwdriver, measuring tape, square, sandpaper.\n* **Intermediate:** Table saw, miter saw, router, sander, clamps.\n* **Advanced:** Jointer, planer, lathe, upholstery tools.\n\n**In the meantime, here's a plan for a very basic, beginner-friendly wooden chair to get you started thinking about it:**\n\n**Project: Simple Wooden Chair (Beginner Level)**\n\nThis chair is designed for simplicity and stability. It uses readily available lumber and basic tools.\n\n**Materials:**\n\n* **Lumber:**\n * Four 2x4s at 18\" for the legs\n * Two 2x4s at 15\" for the front and back aprons (these connect the front and back legs)\n * Two 2x4s at 13\" for the side aprons (these connect the front leg to the back leg on each side)\n * Five 1x4s at 18\" for the seat slats\n* **Hardware:**\n * 2 1/2\" wood screws\n * Wood glue\n* **Finishing:**\n * Sandpaper (various grits)\n * Wood stain or paint (optional)\n * Sealer (polyurethane or varnish)\n\n**Tools:**\n\n* Saw (hand saw or circular saw)\n* Drill with drill bits\n* Screwdriver\n* Measuring tape\n* Square\n* Sandpaper\n* Clamps (optional, but helpful)\n\n**Instructions:**\n\n1. **Cut the Lumber:** Cut all the pieces to the required lengths. Ensure the cuts are square for a stable chair.\n2. **Assemble the Side Frames:**\n * Take two 18\" leg pieces and one 13\" side apron.\n * Attach the side apron between the two legs, flush with the top of the legs, using wood glue and screws. Drill pilot holes to prevent splitting. This forms one side frame.\n * Repeat for the other side frame.\n3. **Connect the Side Frames:**\n * Stand the two assembled side frames upright.\n * Attach one 15\" front apron between the front legs of the two side frames, flush with the top. Use wood glue and screws.\n * Attach the other 15\" back apron between the back legs of the two side frames, flush with the top. Use wood glue and screws. This completes the main chair frame.\n4. **Attach the Seat Slats:**\n * Space the five 18\" seat slats evenly across the top of the assembled frame.\n * Attach each slat with wood glue and screws.\n * Leave small gaps between the slats for drainage if you plan to use the chair outdoors.\n5. **Sanding:** Sand all surfaces smooth, starting with a coarser grit sandpaper and working your way to a finer grit. Pay attention to edges and corners.\n6. **Finishing (Optional):**\n * Stain or paint the chair to your desired color. Apply multiple thin coats for a better finish.\n * Apply a sealer (polyurethane or varnish) to protect the wood and enhance its durability, especially if it will be used outdoors.\n\n**Important Considerations:**\n\n* **Safety:** Always wear safety glasses and a dust mask when cutting and sanding wood.\n* **Accuracy:** Precise measurements and square cuts are crucial for a stable and well-built chair.\n* **Wood Glue:** Use a good quality wood glue for strong and durable joints.\n* **Pilot Holes:** Drill pilot holes before driving screws to prevent the wood from splitting.\n* **Clamping:** Clamping the pieces together while the glue dries ensures a strong bond.\n\n**Next Steps:**\n\n1. **Tell me more about what kind of chair you want to build!** The more information you give me, the better I can tailor the instructions.\n2. **Consider drawing a rough sketch of your desired chair.** This will help you visualize the design and plan the construction process.\n3. **Research different chair designs online.** Look for plans and tutorials that match your skill level and available tools.\n\nI'm excited to help you build your chair! Let's get started!\n"
}License
This project is released under the MIT license. See LICENSE for details.
Contributing
Contributions and feedback are welcome! Open an issue or a pull request.
Additional information
Is it possible to enable grounding with Google Search, it is currently disabled because it is incompatible with creating the json programmatically.
This repository is also useful for leveraging RAG with grounding with basic models (in this case Gemini, but it can be easily extended to OpenAI or others). Currently if you are using VertexAI's RAG Engine if the model does not find the answer in the vector database it will respond to you with a message like, βI did not find what you are looking for in the files.β however you need to have a general-purpose endpoint that is able to take the RAG information and supplement it with the model information automatically, with this system you can bypass this problem.
