Large language model-based evolutionary optimizer: Reasoning with elitism

The advent of Large Language Models (LLMs) has sparked a revolution in generative Artificial Intelligence (AI) research [1], [2], [3]. Since the introduction of transformer model [4], the generative AI research has seen a surge of activity and every subsequent generation of LLM models have become exceedingly more capable than the previous ones. For example, the first decoder-only model developed by OpenAI in 2018 was called Generative Pre-Trained transformers (GPT) based on the transformer architecture, was capable of processing textual data alone [2], while OpenAI’s latest GPT-4 model released in 2023 is multi-modal, i.e., capable of dealing with natural language, code, as well as images [5]. Several studies since then have shown that Large Language Models (LLMs), such as GPT-4, possess strong reasoning ability [6], [7]. Studies have also shown that a LLM’s performance can be further improved by techniques such as in-context learning [8], chain-of-thought prompting [9], and tree-of-thought prompting [10], [11].

Some examples that highlight the generalization capability of LLM models are: (a) Gato [12], a generalist multi-modal agent based on a LLM capable of performing several tasks. (b) Eureka [13], a human-level reward design algorithm using LLMs, is a gradient-free in-context learning approach to reinforcement learning for robotic applications. (c) Voyager [14], a LLM-powered AI agent, has shown the ability to conduct autonomous exploration, skill acquisition, and discovery in an open-ended Minecraft world without human intervention. To test the reasoning and generalization ability of new generative AI models, Srivastava et al. published a suite of benchmarks containing 204 problems called the Beyond the Imitation Game benchmark (BIG-bench) [15].

This reasoning and generalization ability of LLMs has sparked interest in exploring use of LLM models as AI agents, particularly for applications in science and technology. Bran et al. [16] developed an autonomous AI agent called ChemCrow for computational chemistry research. This AI agent has demonstrated remarkable ability to accomplish tasks across organic synthesis, drug discovery, and material design. Similarly, Boiko et al. [17] presents an autonomous AI agent based on an LLM for chemical engineering research. Blanchard et al. [18] use masked LLMs for automating genetic mutations for molecules for application in drug likeness and synthesizability. Zhang et al. [19] present AutoML-GPT, an AI agent that acts as a bridge between various AI models as well as dynamically trains other AI models with optimized hyperparameters. Stella et al. [20] show that generative AI models can accelerate robot design process at conceptual as well as technical level. They further propose a human-AI co-design strategy for the same. Zheng et al. [21] explores the generative ability of GPT4 as a black-box optimizer to navigate architecture search space, making it an effective agent for neural architecture search. Singh et al. [22] study the utility of LLMs as task planners for robotic applications. Jablonka et al. [23] show that a fine-tuned GPT-3 model can outperform many other ML models, particularly in the low-data limit, for predictive Chemistry.

A common thread which passes through the studies mentioned above is the ability of LLMs to find an optimal solution for complex multi-objective optimization problems at hand. This has attracted a great deal of attention from the scientific community. Several studies have been published exploring the ability of LLMs to work as black-box optimizers. Melamed et al. [24] presents an automatic prompt optimization framework called PROPANE. In a method called InstructZero, Chen et al. optimize a low-dimensional soft-prompt to an open-source LLM, which in turn generates the prompt for the black-box LLM, which then performs a zero-shot evaluation [25]. The soft prompt is optimized using a Bayesian optimization method. Deepmind released a general hyperparameter optimization framework called Optformer based on the transformer architecture [26]. The idea of generation of optimized prompts automatically is also explored in Zhou et al. [27]. Similarly, Pryzant et al. [28] explores incorporating gradient descent into automatic prompt optimization. Chen et al. [29] introduces a discrete prompt-optimization framework incorporating human-designed feedback rules. We also see some examples of using LLM within a Reinforcement Learning (RL) framework for optimization [13], [30], [31].

While the examples mentioned so far focused on optimized prompt generation, a few studies have also explored using LLMs for mathematical optimization directly. Liu et al. [32] propose LLM-based Evolutionary Algorithm (LMEA), in which a LLM is responsible for the selection of the parent solution, mutation, cross-over, and generation of a new solution. Guo et al. [33] conducts an assessment of the various optimization abilities of LLM. Their study concludes that LLMs can perform optimization, including gradient descent, hill-climbing, grid search and black-box optimization well, particularly when the sample sizes are small. Pluhacek et al. [34] presents a strategy for using LLM for swarm intelligence-based optimization algorithms. Liu et al. [35] proposes LLM-based multi-objective optimization method, where LLM serves as black-box search operator for Multi-Objective Evolutionary Algorithms (MOEA). Liu et al. [36] proposes using LLMs for optimization algorithm evolution in a framework called AEL (which stands for Algorithm Evolution using LLMs). Liu et al. [37] adopt automatic hill-climbing process using language models as black box optimizers for vision-language models. They show that LLMs utilize implicit gradient direction for more efficient search. Optimization by PROmpting (OPRO) framework from Google Deepmind generates new solutions autoregressively and is seen to outperform human-level prompts [38]. We also see examples of using LLM for mathematical operations and optimization; for example, deep learning for symbolic mathematics [39], LLM for symbolic regression [40], [41], using transformers for linear algebra, including matrix operations and eigen-decomposition [42] etc. In a framework called OptiMUS, LLM are used for formulating and solving Mixed-Integer Linear Programming (MILP) problems [43]. Zhang et al. [44] use LLMs for hyperparameter optimization. Romera-Paredes et al. [45] introduce an evolutionary procedure called FunSearch (short for searching in function space), where a pretrained LLM model is paired with a systematic evaluator for efficient function space search.

The literature cited here demonstrates beyond doubt the ability of LLMs to perform numerical optimization. However, the following questions remain unanswered:

In this paper, our goal is to showcase the ability of LLMs to perform zero shot black-box optimization. We prove that LLMs showcase reasoning abilities and develop an understanding of the objective function landscape over the course of optimization. Via judicious balance between exploration and exploitation strategies, elitism as guardrails and engineering workarounds, we develop a robust framework to solve various optimization problems that hold practical relevance for the industry. We call this method a Large Language-model based Evolutionary Optimizer (LEO). The main contributions of this paper are as follows:

In Section 2, we provide a detailed description of LEO. In Section 3, we highlight the performance of LEO for various benchmark optimization problems as well as engineering applications. This is followed by the Section 4, wherein we discuss the reasoning ability of LLMs to sample better candidate points that result in accelerated convergence followed by challenges associated with LEO and recommendations to overcome them. Lastly, in Section 5, we highlight conclusions.

Motivation Towards populated-based approach

We begin this section to provide a motivation behind perusing a population-based approach for solving complex non-convex optimization problem. While non-population-based or gradient-based methods are preferred for their quick turn-around time towards convergence, the final solutions are likely to get trapped in local optima for non-convex problems. In this section, we setup a quick experiment to demonstrate this idea via the LLM-assisted optimization framework without a population-based

Numerical results

In this section, we evaluate our proposed optimization strategy via a series of test cases, ranging from simple benchmark problems to engineering applications. These tests are classified into four categories: (a) simple benchmark function optimization problems; (b) multi-objective optimization problems; (c) high-dimensional benchmark optimization problems; and (d) industry-relevant engineering optimization problems. This allows us to scrutinize the approach for a range of problems of different

Discussion

As alluded to in the earlier sections, one of the novel aspects of LEO is its ability to reason towards generating better candidate solutions over the course of optimization. In this section, we undertake experiments to demonstrate the reasoning capabilities of LEO backed by strong evidence. We show that LLMs inherently possess attributes of reasoning that, when assisted by an elitism criterion, allow for a hybrid framework that renders faster convergence towards a global optimal solution for

Conclusions

This paper presents a population-based optimization method based on LLMs called Large Language-model-based Evolutionary Optimizer (LEO). We present a diverse set of benchmark test cases, spanning from elementary examples to multi-objective and high-dimensional numerical optimization problems. Furthermore, we illustrate the practical application of this method to industrial optimization problems, including shape optimization, heat transfer, and windfarm layout optimization.

Several key

CRediT authorship contribution statement

Shuvayan Brahmachary: Writing – review & editing, Visualization, Validation, Methodology, Investigation, Formal analysis, Conceptualization. Subodh M. Joshi: Writing – review & editing, Writing – original draft, Project administration, Methodology, Conceptualization. Aniruddha Panda: Software, Resources, Methodology, Formal analysis. Kaushik Koneripalli: Writing – review & editing, Methodology, Formal analysis, Data curation. Arun Kumar Sagotra: Validation, Investigation, Formal analysis.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The authors from Shell India Markets Pvt. Ltd. would like to acknowledge the financial support from Shell, United States .