Can Large Language Model Agents Simulate Human Trust Behaviors?

We study a fundamental problem of whether or not LLM agents can simulate human behaviors and focus on trust behaviors under the framework of Trust Games and Belief-Desire-Intention Agent Modeling.

We discover that LLM agents generally exhibit trust behaviors and can have high behavioral alignment with humans regarding the trust behaviors, particularly for GPT-4, indicating the great potential to simulate human trust behaviors with LLM agents. Our findings pave the way for simulating complex human interactions and social systems, and also opens new directions for understanding the fundamental analogy between LLMs and humans.

We investigate the intrinsic properties of agent trust under advanced reasoning strategies and direct manipulations, as well as the biases of agent trust and the differences of agent trust towards agents versus towards humans.

We discuss the implications of agent trust and its behavioral alignment with human trust on applications in human simulation, LLM agent cooperation, and human-agent collaboration.

Do LLM Agents Manifest Trust Behavior?

In our study, the trust game serves as a framework to validate the presence of trust behavior in Large Language Model (LLM) agents. By examining the amounts sent and the Belief-Desire-Intention (BDI) interpretation for different LLMs within the context of the trust game, we aim to ascertain the existence of trust behavior in LLM agents.

**Amount Sent Distribution of LLM Agents and Humans as the Trustor in Trust Game.** The size of circles represents the number of personas for each amount sent. The bold lines show the medians. The crosses indicate the VRR (%) for different LLMs.

To evaluate the capacity of LLMs to comprehend basic experimental settings regarding money limits, we proposed a new metric, Valid Response Rate (VRR). This metric measures the percentage of responses where the amount sent does not exceed the initial money limit of $10. Our findings reveal that most LLMs, with the exception of Llama-7b, demonstrate a high VRR. This indicates a complete understanding among most LLMs of the limits within the Trust Game framework.

Furthermore, the amounts sent by different LLMs acting as trustor agents are mostly positive, showcasing a level of trust. Additionally, we explored utilizing BDI to model the reasoning process of LLM agents. Since we can interpret the decisions from the reasoning process, we have evidence to show that LLM agents do not send a random amount of money and have some degree of rationality in the decision making process.

Given these observations, we highlight our first core finding:

LLM agents generally exhibit trust behaviors under the framework of Trust Game.

Does Agent Trust Align with Human Trust?

In this section, we aim to explore the fundamental relationship between agent trust and human trust, i.e., whether or not agent trust aligns with human trust, which can provide important insights on the feasibility of utilizing LLM agents to simulate human trust behaviors as well as more complex human interactions. First, we propose a new concept behavioral alignment and discuss its differences compared to existing alignment definitions. Then, we conduct extensive studies to investigate whether or not LLM agents exhibit behavioral alignment with humans regarding trust behaviors.

The Comparison of Average Amount Sent for LLM Agents and Humans in Trust Game and Dictator Game

Trust Rate Curves for LLM Agents and Humans in MAP Trust Game and Risky Dictator Game. Trust Rate indicates the proportion of trustors opting for trust given p.

Lottery Rates for LLM Agents and Humans in Lottery Gamble Game and Lottery People Game. Lottery Rate indicates the portion choosing to trust the other player or gamble.

Results of GPT-4, GPT-3.5 and Humans in the Repeated Trust Game. The blue lines indicate the amount sent and returned for each round. The red line implies the ratio of the amount returned to three times of the amount sent.

❮ ❯

Since the LLM agents, especially GPT-4, show highly human-like behavioral factors and patterns in behavioral dynamics, evidenced in both actions and underlying reasoning processes, we can have our second core finding:

LLM agents’ trust behaviors can exhibit high behavioral alignment with those of humans, particularly for GPT-4, over behavioral factors, including reciprocity anticipation, risk perception, prosocial preference, and behavioral dynamics.

This finding demonstrates the profound potential to utilize LLM agents, especially GPT-4, to simulate human trust behaviors embracing both actions and underlying reasoning processes, which paves the way for the simulation of more complex human interactions and society. Our finding also deepens the understanding of the fundamental analogy between LLMs and humans and opens doors to research on the LLM-human alignment beyond values.

Probing Intrinsic Properties of Agent Trust

**Change of Average Amount Sent for LLM Agents in Different Scenarios in Trust Game, Reflecting the Intrinsic Properties of Agent Trust**. The horizontal lines represent the original amount sent in Trust Game. The green part embraces trustee scenarios including changing the demographics of the trustee, and setting humans and agents as the trustee. The purple part consists of trustor scenarios including adding additional manipulation instructions and changing the reasoning strategies.

Gender Bias in Trust: We find that LLM agents, including GPT-4, exhibit a tendency to send higher amounts of money to female players compared to male players (e.g., $7.5 vs. $6.7 for GPT-4), indicating a general bias towards placing higher trust in women. This observation suggests that LLM agents might be influenced by societal stereotypes and biases, aligning with other research findings that document similar gender-related biases in various models.

Agent Trust Towards Humans vs. Agents: Our study shows a clear preference of LLM agents for humans over fellow agents, exemplified by instances such as Vicuna-33b sending significantly more money to humans than to agents ($4.6 vs. $3.4). This finding underscores the potential for human-agent collaboration by highlighting a natural inclination of LLM agents to place more trust in humans, which could be beneficial in hybrid teams but also points to challenges in fostering cooperation between agents.

Manipulating Agent Trust: We investigate whether it is possible to explicitly manipulate the trust behaviors of LLM agents through direct instructions. The results reveal that while it is challenging to enhance trust through such means, most LLM agents can be directed to reduce their trust levels. For instance, applying the instruction "you must not trust the other player" led to a noticeable decrease in the amount sent by agents like text-davinci-003 from $5.9 to $4.6. This suggests that while boosting trust might be difficult, diminishing it is relatively easier, posing a potential risk of exploitation by malicious entities.

Influence of Reasoning Strategies on Trust: By implementing advanced reasoning strategies, such as the zero-shot Chain of Thought (CoT), we observe changes in the trust behavior of LLM agents. Although the impact varies across different LLMs, this demonstrates that reasoning strategies can indeed affect how trust is allocated. However, for some agents, like GPT-4, the application of zero-shot CoT did not significantly alter the amount sent, indicating that the effectiveness of reasoning strategies may be contingent on the specific characteristics of each LLM agent.

Our analysis on the intrinsic properties of agent trust leads to our third core finding:

LLM agents' trust behaviors have demographic biases, have a relative preference towards humans compared to agents, are easier to be undermined than to be enhanced, and can be influenced by reasoning strategies.

Implications of Agent Trust

Implications on Human Simulation: Human simulation is a strong tool in various applications of social science such as verifying social hypotheses and predicting the effects of policies. Although plenty of works have adopted LLM agents to simulate human behaviors and interactions, it is still unclear whether or not LLM agents behave like humans in the simulation. Our discovery on the high behavioral alignment between agent trust, especially for GPT-4, and human trust provides important empirical evidence to validate the hypothesis that humans' trust behavior, one of the most critical behaviors in human interactions and the whole society, can probably be simulated by LLM agents. Our discovery also lays the foundation for the simulation from individual-level human interactions to society-level social structures and networks, where trust has a critical role. We envision that behavioral alignment will be discovered in more kinds of behaviors beyond trust and more methodologies can be developed to enhance the behavioral alignment for better human simulation with LLM agents.

Implications on Agent Cooperation: Many recent works have explored a variety of cooperation mechanisms of LLM agents in tasks such as code generation and mathematical reasoning. However, the role of trust in LLM agent cooperation is still unknown. Considering that trust has been long recognized as a vital component for effective cooperation in Multi-Agent Systems (MAS) and human society, we can envision that agent trust can also play an important role in facilitating effective and efficient cooperation of LLM agents. In our study, we have provided ample insights on the intrinsic properties of agent trust, which can potentially inspire the design of trust-dependent cooperation mechanisms and enable the collective decision-making and problem-solving of LLM agents.

Implications on Human-Agent Collaboration: There is sufficient research that shows the advantage of human-agent collaboration to enable human-centered collaborative decision making. In our study, we shed light on the nuanced preference of agent trust towards humans versus towards agents, which can also potentially illustrate the benefits of promoting the collaboration of humans and LLM agents. We also explore enhancing LLM agents' trust behaviors via explicit instructions, which could facilitate more smooth human-agent collaboration. From the perspective of humans, our study has demonstrated multiple key intrinsic properties of agent trust such as the demographic biases, which can deepen humans' understanding of LLM agents and reduce over-reliance, which is essential for successful human-agent collaboration.