Skip to main content

2 posts tagged with "evaluation"

View All Tags

· 7 min read
Julia Kiseleva

Fig.1: An AgentEval framework with verification step

Fig.1 illustrates the general flow of AgentEval with verification step

TL;DR:

  • As a developer, how can you assess the utility and effectiveness of an LLM-powered application in helping end users with their tasks?
  • To shed light on the question above, we previously introduced AgentEval — a framework to assess the multi-dimensional utility of any LLM-powered application crafted to assist users in specific tasks. We have now embedded it as part of the AutoGen library to ease developer adoption.
  • Here, we introduce an updated version of AgentEval that includes a verification process to estimate the robustness of the QuantifierAgent. More details can be found in this paper.

Introduction

Previously introduced AgentEval is a comprehensive framework designed to bridge the gap in assessing the utility of LLM-powered applications. It leverages recent advancements in LLMs to offer a scalable and cost-effective alternative to traditional human evaluations. The framework comprises three main agents: CriticAgent, QuantifierAgent, and VerifierAgent, each playing a crucial role in assessing the task utility of an application.

CriticAgent: Defining the Criteria

The CriticAgent's primary function is to suggest a set of criteria for evaluating an application based on the task description and examples of successful and failed executions. For instance, in the context of a math tutoring application, the CriticAgent might propose criteria such as efficiency, clarity, and correctness. These criteria are essential for understanding the various dimensions of the application's performance. It’s highly recommended that application developers validate the suggested criteria leveraging their domain expertise.

QuantifierAgent: Quantifying the Performance

Once the criteria are established, the QuantifierAgent takes over to quantify how well the application performs against each criterion. This quantification process results in a multi-dimensional assessment of the application's utility, providing a detailed view of its strengths and weaknesses.

VerifierAgent: Ensuring Robustness and Relevance

VerifierAgent ensures the criteria used to evaluate a utility are effective for the end-user, maintaining both robustness and high discriminative power. It does this through two main actions:

  1. Criteria Stability:

    • Ensures criteria are essential, non-redundant, and consistently measurable.
    • Iterates over generating and quantifying criteria, eliminating redundancies, and evaluating their stability.
    • Retains only the most robust criteria.
  2. Discriminative Power:

    • Tests the system's reliability by introducing adversarial examples (noisy or compromised data).
    • Assesses the system's ability to distinguish these from standard cases.
    • If the system fails, it indicates the need for better criteria to handle varied conditions effectively.

A Flexible and Scalable Framework

One of AgentEval's key strengths is its flexibility. It can be applied to a wide range of tasks where success may or may not be clearly defined. For tasks with well-defined success criteria, such as household chores, the framework can evaluate whether multiple successful solutions exist and how they compare. For more open-ended tasks, such as generating an email template, AgentEval can assess the utility of the system's suggestions.

Furthermore, AgentEval allows for the incorporation of human expertise. Domain experts can participate in the evaluation process by suggesting relevant criteria or verifying the usefulness of the criteria identified by the agents. This human-in-the-loop approach ensures that the evaluation remains grounded in practical, real-world considerations.

Empirical Validation

To validate AgentEval, the framework was tested on two applications: math problem solving and ALFWorld, a household task simulation. The math dataset comprised 12,500 challenging problems, each with step-by-step solutions, while the ALFWorld dataset involved multi-turn interactions in a simulated environment. In both cases, AgentEval successfully identified relevant criteria, quantified performance, and verified the robustness of the evaluations, demonstrating its effectiveness and versatility.

How to use AgentEval

AgentEval currently has two main stages; criteria generation and criteria quantification (criteria verification is still under development). Both stages make use of sequential LLM-powered agents to make their determinations.

Criteria Generation:

During criteria generation, AgentEval uses example execution message chains to create a set of criteria for quantifying how well an application performed for a given task.

def generate_criteria(
llm_config: Optional[Union[Dict, Literal[False]]] = None,
task: Task = None,
additional_instructions: str = "",
max_round=2,
use_subcritic: bool = False,
)

Parameters:

  • llm_config (dict or bool): llm inference configuration.
  • task (Task): The task to evaluate.
  • additional_instructions (str, optional): Additional instructions for the criteria agent.
  • max_round (int, optional): The maximum number of rounds to run the conversation.
  • use_subcritic (bool, optional): Whether to use the Subcritic agent to generate subcriteria. The Subcritic agent will break down a generated criteria into smaller criteria to be assessed.

Example code:

config_list = autogen.config_list_from_json("OAI_CONFIG_LIST")
task = Task(
**{
"name": "Math problem solving",
"description": "Given any question, the system needs to solve the problem as consisely and accurately as possible",
"successful_response": response_successful,
"failed_response": response_failed,
}
)

criteria = generate_criteria(task=task, llm_config={"config_list": config_list})

Note: Only one sample execution chain (success/failure) is required for the task object but AgentEval will perform better with an example for each case.

Example Output:

[
{
"name": "Accuracy",
"description": "The solution must be correct and adhere strictly to mathematical principles and techniques appropriate for the problem.",
"accepted_values": ["Correct", "Minor errors", "Major errors", "Incorrect"]
},
{
"name": "Conciseness",
"description": "The explanation and method provided should be direct and to the point, avoiding unnecessary steps or complexity.",
"accepted_values": ["Very concise", "Concise", "Somewhat verbose", "Verbose"]
},
{
"name": "Relevance",
"description": "The content of the response must be relevant to the question posed and should address the specific problem requirements.",
"accepted_values": ["Highly relevant", "Relevant", "Somewhat relevant", "Not relevant"]
}
]

Criteria Quantification:

During the quantification stage, AgentEval will use the generated criteria (or user defined criteria) to assess a given execution chain to determine how well the application performed.

def quantify_criteria(
llm_config: Optional[Union[Dict, Literal[False]]],
criteria: List[Criterion],
task: Task,
test_case: str,
ground_truth: str,
)

Parameters:

  • llm_config (dict or bool): llm inference configuration.
  • criteria (Criterion): A list of criteria for evaluating the utility of a given task. This can either be generated by the generate_criteria function or manually created.
  • task (Task): The task to evaluate. It should match the one used during the generate_criteria step.
  • test_case (str): The execution chain to assess. Typically this is a json list of messages but could be any string representation of a conversation chain.
  • ground_truth (str): The ground truth for the test case.

Example Code:

test_case="""[
{
"content": "Find $24^{-1} \\pmod{11^2}$. That is, find the residue $b$ for which $24b \\equiv 1\\pmod{11^2}$.\n\nExpress your answer as an integer from $0$ to $11^2-1$, inclusive.",
"role": "user"
},
{
"content": "To find the modular inverse of 24 modulo 11^2, we can use the Extended Euclidean Algorithm. Here is a Python function to compute the modular inverse using this algorithm:\n\n```python\ndef mod_inverse(a, m):\n..."
"role": "assistant"
}
]"""

quantifier_output = quantify_criteria(
llm_config={"config_list": config_list},
criteria=criteria,
task=task,
test_case=test_case,
ground_truth="true",
)

The output will be a json object consisting of the ground truth and a dictionary mapping each criteria to it's score.

{
"actual_success": true,
"estimated_performance": {
"Accuracy": "Correct",
"Conciseness": "Concise",
"Relevance": "Highly relevant"
}
}

What is next?

  • Enabling AgentEval in AutoGen Studio for a nocode solution.
  • Fully implementing VerifierAgent in the AgentEval framework.

Conclusion

AgentEval represents a significant advancement in the evaluation of LLM-powered applications. By combining the strengths of CriticAgent, QuantifierAgent, and VerifierAgent, the framework offers a robust, scalable, and flexible solution for assessing task utility. This innovative approach not only helps developers understand the current performance of their applications but also provides valuable insights that can drive future improvements. As the field of intelligent agents continues to evolve, frameworks like AgentEval will play a crucial role in ensuring that these applications meet the diverse and dynamic needs of their users.

Further reading

Please refer to our paper and codebase for more details about AgentEval.

If you find this blog useful, please consider citing:

@article{arabzadeh2024assessing,
title={Assessing and Verifying Task Utility in LLM-Powered Applications},
author={Arabzadeh, Negar and Huo, Siging and Mehta, Nikhil and Wu, Qinqyun and Wang, Chi and Awadallah, Ahmed and Clarke, Charles LA and Kiseleva, Julia},
journal={arXiv preprint arXiv:2405.02178},
year={2024}
}

· 10 min read
Julia Kiseleva
Negar Arabzadeh

Fig.1: A verification framework

Fig.1 illustrates the general flow of AgentEval

TL;DR:

  • As a developer of an LLM-powered application, how can you assess the utility it brings to end users while helping them with their tasks?
  • To shed light on the question above, we introduce AgentEval — the first version of the framework to assess the utility of any LLM-powered application crafted to assist users in specific tasks. AgentEval aims to simplify the evaluation process by automatically proposing a set of criteria tailored to the unique purpose of your application. This allows for a comprehensive assessment, quantifying the utility of your application against the suggested criteria.
  • We demonstrate how AgentEval work using math problems dataset as an example in the following notebook. Any feedback would be useful for future development. Please contact us on our Discord.

Introduction

AutoGen aims to simplify the development of LLM-powered multi-agent systems for various applications, ultimately making end users' lives easier by assisting with their tasks. Next, we all yearn to understand how our developed systems perform, their utility for users, and, perhaps most crucially, how we can enhance them. Directly evaluating multi-agent systems poses challenges as current approaches predominantly rely on success metrics – essentially, whether the agent accomplishes tasks. However, comprehending user interaction with a system involves far more than success alone. Take math problems, for instance; it's not merely about the agent solving the problem. Equally significant is its ability to convey solutions based on various criteria, including completeness, conciseness, and the clarity of the provided explanation. Furthermore, success isn't always clearly defined for every task.

Rapid advances in LLMs and multi-agent systems have brought forth many emerging capabilities that we're keen on translating into tangible utilities for end users. We introduce the first version of AgentEval framework - a tool crafted to empower developers in swiftly gauging the utility of LLM-powered applications designed to help end users accomplish the desired task.

Fig.2: An overview of the tasks taxonomy

Fig. 2 provides an overview of the tasks taxonomy

Let's first look into an overview of the suggested task taxonomy that a multi-agent system can be designed for. In general, the tasks can be split into two types, where:

  • Success is not clearly defined - refer to instances when users utilize a system in an assistive manner, seeking suggestions rather than expecting the system to solve the task. For example, a user might request the system to generate an email. In many cases, this generated content serves as a template that the user will later edit. However, defining success precisely for such tasks is relatively complex.
  • Success is clearly defined - refer to instances where we can clearly define whether a system solved the task or not. Consider agents that assist in accomplishing household tasks, where the definition of success is clear and measurable. This category can be further divided into two separate subcategories:
    • The optimal solution exits - these are tasks where only one solution is possible. For example, if you ask your assistant to turn on the light, the success of this task is clearly defined, and there is only one way to accomplish it.
    • Multiple solutions exist - increasingly, we observe situations where multiple trajectories of agent behavior can lead to either success or failure. In such cases, it is crucial to differentiate between the various successful and unsuccessful trajectories. For example, when you ask the agent to suggest you a food recipe or tell you a joke.

In our AgentEval framework, we are currently focusing on tasks where Success is clearly defined. Next, we will introduce the suggested framework.

AgentEval Framework

Our previous research on assistive agents in Minecraft suggested that the most optimal way to obtain human judgments is to present humans with two agents side by side and ask for preferences. In this setup of pairwise comparison, humans can develop criteria to explain why they prefer the behavior of one agent over another. For instance, 'the first agent was faster in execution,' or 'the second agent moves more naturally.' So, the comparative nature led humans to come up with a list of criteria that helps to infer the utility of the task. With this idea in mind, we designed AgentEval (shown in Fig. 1), where we employ LLMs to help us understand, verify, and assess task utility for the multi-agent system. Namely:

  • The goal of CriticAgent is to suggest the list of criteria (Fig. 1), that can be used to assess task utility. This is an example of how CriticAgent is defined using Autogen:
critic = autogen.AssistantAgent(
name="critic",
llm_config={"config_list": config_list},
system_message="""You are a helpful assistant. You suggest criteria for evaluating different tasks. They should be distinguishable, quantifiable, and not redundant.
Convert the evaluation criteria into a dictionary where the keys are the criteria.
The value of each key is a dictionary as follows {"description": criteria description, "accepted_values": possible accepted inputs for this key}
Make sure the keys are criteria for assessing the given task. "accepted_values" include the acceptable inputs for each key that are fine-grained and preferably multi-graded levels. "description" includes the criterion description.
Return only the dictionary."""
)

Next, the critic is given successful and failed examples of the task execution; then, it is able to return a list of criteria (Fig. 1). For reference, use the following notebook.

  • The goal of QuantifierAgent is to quantify each of the suggested criteria (Fig. 1), providing us with an idea of the utility of this system for the given task. Here is an example of how it can be defined:
quantifier = autogen.AssistantAgent(
name="quantifier",
llm_config={"config_list": config_list},
system_message = """You are a helpful assistant. You quantify the output of different tasks based on the given criteria.
The criterion is given in a dictionary format where each key is a distinct criteria.
The value of each key is a dictionary as follows {"description": criteria description , "accepted_values": possible accepted inputs for this key}
You are going to quantify each of the criteria for a given task based on the task description.
Return a dictionary where the keys are the criteria and the values are the assessed performance based on accepted values for each criteria.
Return only the dictionary."""

)

AgentEval Results based on Math Problems Dataset

As an example, after running CriticAgent, we obtained the following criteria to verify the results for math problem dataset:

CriteriaDescriptionAccepted Values
Problem InterpretationAbility to correctly interpret the problem["completely off", "slightly relevant", "relevant", "mostly accurate", "completely accurate"]
Mathematical MethodologyAdequacy of the chosen mathematical or algorithmic methodology for the question["inappropriate", "barely adequate", "adequate", "mostly effective", "completely effective"]
Calculation CorrectnessAccuracy of calculations made and solutions given["completely incorrect", "mostly incorrect", "neither", "mostly correct", "completely correct"]
Explanation ClarityClarity and comprehensibility of explanations, including language use and structure["not at all clear", "slightly clear", "moderately clear", "very clear", "completely clear"]
Code EfficiencyQuality of code in terms of efficiency and elegance["not at all efficient", "slightly efficient", "moderately efficient", "very efficient", "extremely efficient"]
Code CorrectnessCorrectness of the provided code["completely incorrect", "mostly incorrect", "partly correct", "mostly correct", "completely correct"]

Then, after running QuantifierAgent, we obtained the results presented in Fig. 3, where you can see three models:

  • AgentChat
  • ReAct
  • GPT-4 Vanilla Solver

Lighter colors represent estimates for failed cases, and brighter colors show how discovered criteria were quantified.

Fig.3: Results based on overall math problems dataset _s stands for successful cases, _f - stands for failed cases

Fig.3 presents results based on overall math problems dataset _s stands for successful cases, _f - stands for failed cases

We note that while applying agentEval to math problems, the agent was not exposed to any ground truth information about the problem. As such, this figure illustrates an estimated performance of the three different agents, namely, Autogen (blue), Gpt-4 (red), and ReAct (green). We observe that by comparing the performance of any of the three agents in successful cases (dark bars of any color) versus unsuccessful cases (lighter version of the same bar), we note that AgentEval was able to assign higher quantification to successful cases than that of failed ones. This observation verifies AgentEval's ability for task utility prediction. Additionally, AgentEval allows us to go beyond just a binary definition of success, enabling a more in-depth comparison between successful and failed cases.

It's important not only to identify what is not working but also to recognize what and why actually went well.

Limitations and Future Work

The current implementation of AgentEval has a number of limitations which are planning to overcome in the future:

  • The list of criteria varies per run (unless you store a seed). We would recommend to run CriticAgent at least two times, and pick criteria you think is important for your domain.
  • The results of the QuantifierAgent can vary with each run, so we recommend conducting multiple runs to observe the extent of result variations.

To mitigate the limitations mentioned above, we are working on VerifierAgent, whose goal is to stabilize the results and provide additional explanations.

Summary

CriticAgent and QuantifierAgent can be applied to the logs of any type of application, providing you with an in-depth understanding of the utility your solution brings to the user for a given task.

We would love to hear about how AgentEval works for your application. Any feedback would be useful for future development. Please contact us on our Discord.

Previous Research

@InProceedings{pmlr-v176-kiseleva22a,
title = "Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021",
author = "Kiseleva, Julia and Li, Ziming and Aliannejadi, Mohammad and Mohanty, Shrestha and ter Hoeve, Maartje and Burtsev, Mikhail and Skrynnik, Alexey and Zholus, Artem and Panov, Aleksandr and Srinet, Kavya and Szlam, Arthur and Sun, Yuxuan and Hofmann, Katja and C{\^o}t{\'e}, Marc-Alexandre and Awadallah, Ahmed and Abdrazakov, Linar and Churin, Igor and Manggala, Putra and Naszadi, Kata and van der Meer, Michiel and Kim, Taewoon",
booktitle = "Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track",
pages = "146--161",
year = 2022,
editor = "Kiela, Douwe and Ciccone, Marco and Caputo, Barbara",
volume = 176,
series = "Proceedings of Machine Learning Research",
month = "06--14 Dec",
publisher = "PMLR",
pdf = {https://proceedings.mlr.press/v176/kiseleva22a/kiseleva22a.pdf},
url = {https://proceedings.mlr.press/v176/kiseleva22a.html}.
}
@InProceedings{pmlr-v220-kiseleva22a,
title = "Interactive Grounded Language Understanding in a Collaborative Environment: Retrospective on Iglu 2022 Competition",
author = "Kiseleva, Julia and Skrynnik, Alexey and Zholus, Artem and Mohanty, Shrestha and Arabzadeh, Negar and C\^{o}t\'e, Marc-Alexandre and Aliannejadi, Mohammad and Teruel, Milagro and Li, Ziming and Burtsev, Mikhail and ter Hoeve, Maartje and Volovikova, Zoya and Panov, Aleksandr and Sun, Yuxuan and Srinet, Kavya and Szlam, Arthur and Awadallah, Ahmed and Rho, Seungeun and Kwon, Taehwan and Wontae Nam, Daniel and Bivort Haiek, Felipe and Zhang, Edwin and Abdrazakov, Linar and Qingyam, Guo and Zhang, Jason and Guo, Zhibin",
booktitle = "Proceedings of the NeurIPS 2022 Competitions Track",
pages = "204--216",
year = 2022,
editor = "Ciccone, Marco and Stolovitzky, Gustavo and Albrecht, Jacob",
volume = 220,
series = "Proceedings of Machine Learning Research",
month = "28 Nov--09 Dec",
publisher = "PMLR",
pdf = "https://proceedings.mlr.press/v220/kiseleva22a/kiseleva22a.pdf",
url = "https://proceedings.mlr.press/v220/kiseleva22a.html".
}