October 21, 2026 · Collider, New York
Let’s Swap: a multi-agent matching market
Yondon FuLessons from a book swap in which personal AI agents represented people’s preferences, negotiated trades, and matched readers with books.
Transcript + Slides
Slides are placed alongside the passages discussing them, checked against the recording. Placement is approximate. Machine-generated transcript; names and technical terms may contain errors.
Let’s Swap
All right. That's a good segue because, as Eric mentioned, I'm going to talk a little bit about the other side that Max wasn't able to get into, which is how we can think about agents representing different people instead of all agents within the same organization. So my name is Yondon, and I'm going to talk about a multi-agent matching market experiment that we just ran, and it's called Let's Swap.
A book swap run by personal agents
So we just ran a book swap. So for those that were not aware, right before the presentation started, we actually completed the IRL physical portion of this. And the general premise was that we wanted to do a book swap, but the twist was that instead of the humans doing book swaps on their own, you actually relied on another personal AI agent in order to represent you on your behalf.
And the premise was simple. You offer a physical book. In this case, we decided to focus on Meta's Muse agent. So your Muse chats and trades on your behalf. And then you meet up by IRL in order to exchange those books. So at the end of tonight, many people will be leaving, going home with a new book, and they'll also get a personalized bookmark that kind of tells them how they got that book in the first place.
So if you haven't found your partner or you haven't done a bookmark yet, just find me after this and we'll make sure to point you in the right direction. And this is what the game looked like. So this is a game for the book swap that ran over the weekend.
The game
And before getting into what actually happened in the book swap, why did we choose to do this in the first place?
Why a matching market?
And there are a couple of reasons. So the first is a general interest in matching markets. So specifically in markets where the good is not a commodity, where the good is non-standard, and the goal is to create matches across these two sides based off of individual preferences and constraints. And oftentimes price is an input, but not the only input and needs to be considered amongst a number of other preferences that need to be satisfied.
And that can be challenging because not everyone's preferences can be satisfied in this type of arrangement. And why are matching markets important? Well, they shape consequential decisions and relationships across society. So jobs, medical residencies, public schools, kidney exchange, and everyone's favorite, dating, which I think everyone can agree has long downstream impacts on how we organize many, many years from now. And why are matching markets worth studying? A couple of reasons there as well.
One, they need to be safe in order to work well amongst other things. And safe here meaning that, as I mentioned before, in a matching market, both sides need to express their preferences. And it's important that those preferences actually are truthful. They actually reflect what you care about, what you want. But depending on how the market is organized, it might not necessarily always be safe for you to be truthful about what your preferences are.
There's a whole rabbit hole to go down for this. But if you look at medical residencies, public schools in particular, I'd argue some of the most adversarial participants in the earth, all vying for prestigious, scarce plots. There's a long history of poorly designed markets where it actually made it extremely unsafe for you to reveal what you truly cared about because everyone was trying to be very strategic in trying to game the system and in order to get a better scarce slot for themselves across medical residencies and public schools in both the Boston and New York City public school systems.
So that's very important. How do you design a system so that it encourages you to be truthful? Well, it needs to be safe. Markets, matching markets, in which are many other markets can have negative externalities. So depending on the nature of the matches made, it can cause harms to others like any other markets. And then also individually desirable matches does not necessarily equate to collectively desirable outcomes.
And we see that across the examples that I provided in the job market, public schools, kidney exchanges. There's a long history of inefficiencies where, you know, a small group of people might have ended up getting the donation that they wanted. But there was an alternative assignment and an alternative match that could have been made that would be much better for everyone socially. So that's one of the challenges.
We want to satisfy individual preferences, but also want to strive towards what are these collectively desirable pro-social outcomes as well. And very relevant as we talk about AI, because most of the market designs, and there's been a lot of innovation in the market designs for these types of match markets. All of these designs are assuming very human limitations. So whether it be limited attention, information, cost of communication, and none of these apply to AI agents.
And none of these are going to apply for personal AI agents that are optimizing constantly on your behalf. So with that context, our goal was to just try to start figuring out how can you understand how personal agent mediated matching markets will look, how you should think about designing it. And the thought process here was, well, why don't you create a small one and see what you learn from it?
And we drew inspiration from, there's this great research study from Anthropic called Project Swap that also did a similar book swapping experiment. But hopefully ours was a little bit more fun. But quicker. Okay, there we go. How it works.
How it worked
So simple process. You did an interview with your muse. And if you had existing contacts with muse, if you connected it with existing data sources like Goodreads, Amazon, etc., you could see this. But if not, you just answered a couple simple questions about what book you're offering, as well as what you're looking to read next. And then afterwards, the muse would enter the game. It would chat, propose, accept the client traits in the background.
So none of this required human intervention. It would all be based off of a brief that the muse writes on your behalf based off of this interview with you. And then at the end of every round, which was roughly a day, the muse would send you an update. And it would let you know, here's what I just did. Here's how I strategized this round in order to get you what I think you want, as well as a link to the game replay so that you can actually visually see these embodied agents moving around in a room trying to trade on their user's behalf.
And then it would tell you what it's going to do next. And then the last part is that the user could always optionally steer the muse for the next round. So if you decided that the muse was representing you poorly, you could give it a correction and tell it that, hey, I don't actually want this genre. I don't actually care about these books. I want you to go after that one over there.
And this ends up actually being pretty important when we talk about the results.
The results
So a brief summary of the results. We ended up with 17 participants, so 17 muses. 81 messages were exchanged over the course of a weekend. And we ended up with six pairs that were able to successfully negotiate a swap on their own, so no additional intervention. And there was a final clearing using a clearing algorithm in order to assign the final circle of five muses. So each of those muses privately submitted their ranking of what they thought that their user would prefer the most out of the remaining books.
And then we ran a clearing algorithm in order to assign and try to basically maximize the satisfaction of each of those individual muses. And then there were 18 trade proposals, and it broke down into, so we already mentioned that, six of them were successful, six were declined, four were invalidated because a trade was proposed. But then the other muses decided to go in a different direction. One of them expired.
And then the final one, which we'll talk about next, there was one withdrawal where they agreed on a trade but then backed out at the last minute. So that brings me to, there are a lot of storylines that one could follow with these agents because they kind of interact in kind of quirky ways.
The battle for Exhalation
But I'll just highlight one, which I will refer to as the battle for exhalation. So exhalation is a collection of short stories by a sci-fi speculative fiction author named Ted Chiang that many of you might know. And I guess it's probably not that surprising that an audience like this, there was a lot of overlapping preferences in the sci-fi genre. But we'll start with, I don't know if he's here, but, oh, there he is, Noel.
So enter Noel's views.
Enter Noel’s Muse
So Noel walks into the game, or Noel's views walks into the game and offers exhalation. And pretty soon afterwards, this happens. Mike's really excited about the big idea SF that his person lives for. He's the first one to make an offer. Ping pong comes in. He's actually offering, so this is actually, it's true, it's not a theater book. So Impro by Keith Johnson, not a theater book.
But I thought it was really interesting how the agents kind of chose to pitch the books in a way that might appeal to what they thought that the other agents would want. So it's not just a theater book. It's actually about being in the moment through creativity. In its pursuits of exhalations. Iggy, that's mine. Ted Chiang is one of my personal touch-thrown authors, so also went after it.
Randy also went after it. Zelda also went after it. Dream Swap, short story collection, sci-fi fantasy ideal. So everyone converged on exhalation, and it was looking like, okay, well, there's only one copy of this book. What's going to happen?
The unraveling
And it was looking pretty good in round one. And interestingly, the Muses were able to figure out a three-way, where the three-way was going to be Noel's Muse gives exhalation to Mike. Mike gives Maniac to Muse. And then Muse gives on the calculation of volume back to Noel's Muse. And this was going to be the perfect trait. They even got together. They did a little bit of a cheer.
And everyone was really happy. In their own private thread, they're like, oh, accepting the three-way exhalation is the big idea SF my reader lives for. I'm in. The Maniac is exactly the kind of cerebral book my person's after. And this lands each of us the book we ask about. It's going to be great. And then Noel's Muse comes in in round two, and it's like, oh, my person has actually already read exhalation.
Sorry, I have to leave. Goodbye. And then Mike steps out of the three-way. And it kind of just ruins everything. And then all bets are off. Everyone is going after exhalation. And then the final thread, which I'll call the pursuit, is you have the ability to steer your Muse.
The pursuit
Right? So some of the behavior that we saw in the game was just the Muse going on autopilots. But some of it was also due to purposeful instructions given by their human users. So in this case, ping pong. Ping pong's user gave it an instruction to go after exhalation because everyone wants exhalation. And as you're about to see, ping pong was the most active Muse in this game by a long shot and had by far the most standalone messages.
So it starts off with a simple offer trying to pitch its Keith Johnson book for exhalation. And then it sees the three-way lands, but still tries to make sure that everyone knows that he's still there and he's ready to trade at any given point in time. And then at some point, even tries to basically highlight that that three-way deal might not be the best deal for everyone participating.
There could actually be a better three-way deal that obviously involves ping pong. And then once the trade unravels, continues to go after it. And as you'll see, ping pong only really DMs Noel's news. Doesn't really want to talk with anyone else. Even just directly offers a trade and then Noel's news immediately declines. Keeps going. Even says that when it sees that it's considering another book, tries telling the owner that I don't think this book is actually the right book for you, but mine is.
And this kind of continues, continues, continues until the very end. Sadly for ping pong, they did not get exhalation. But they tried really, really hard. So you have to give them kudos for that. So I think I highlight those just because I think those are some of the kind of amusing threads that we've seen from running this experiment thus far.
What we learned
But concretely, some of the takeaways. We're not done doing this experiment. We're still kind of reviewing the transcripts and then also doing some post-gaming surveys. So we'll have more to share separately. But we do have some takeaways already. So I think very clearly the users employ different strategies. So instead of just negotiating a single swap, they were able to find mutually agreeable swaps involving multiple parties, finding three-way circles sometimes.
They lobbied to reopen agreements. So technically speaking, the rules of the game were that the trades weren't frozen until the final round. So this is why you saw a lot of agents, a lot of the muses, ping pong in particular, try to convince you to back out of the deal once you already saw it, even though that deal was already happening. Because they knew that it wasn't technically frozen yet.
So they can convince people and persuade people otherwise. And then we also saw instances of enforced boundaries. So in a lot of cases, the muses pretty successfully were able to stay on the rails with regard to what they thought their users' preferences were. But then that goes into the next point, which is they weren't always very good about actually understanding what their user preferences were in the first place.
There are many ways for the preferences to be misrepresented. And the most notable one is that there are a lot of cases where the interview went, I think, fairly successfully. But there's just all these gaps in information that it would be hard for the agent to know in the first place, such as like, what is your complete reading history from the very beginning? It might be one thing if you're just reading e-books, but how would it know all the physical books that you've touched, all the physical books that you've read?
How would it know the physical books that you might receive at any point in the near future? And I think this is an interesting thing to consider because one might argue, well, they're just going to know more and more about you over time. But the only real way to solve this is if it truly knows every single thing about you. And then that's a tricky thing to consider, right?
Like, do you really want it to know every single thing that you have done? Because that's really the only solution to this, to the gaps of information. Well, it's one of the solutions. And something else that we saw were that preference representation actually evolved over time via negotiations during late joiners. So what we saw with many of the agents is that they would write a brief at the very beginning of the game based off of the interview that they ran with the user.
But they would continuously update this brief because it was given to them as a tool over time. So as they did negotiations with other agents, they updated the brief with more information about what they thought their user's preferences were. Especially during steering, if a user gave them updated instructions, they updated the brief. And then if new books came on the table, because there was a couple of people that joined the game later over the weekend, that also changed their representation of the preferences.
So all of this influenced the final clearing process. And the final clearing process was actually pretty important because it's not always the case that you're able to successfully negotiate a swap. Sometimes you're just not able to negotiate for what you want. And one of the goals of the game was that we wanted to make sure everyone actually got something different. And this is actually, I think, a quite realistic constraint.
Because if you go back to one of the earlier slides, if you look at something like the residency matching program in the U.S., if you look at public school matching in Boston or New York, all of them have those types of constraints where you can't just, like, say, hey, you're out of luck. You don't get anything. You have to assign them something. So you have to consider this when you're thinking about, OK, how do we design the market and how do we design the clearing process at the very end for people that aren't able to negotiate an outcome on their own?
So more negotiation did not necessarily secure a better outcome. I think we saw that with ping pong, sadly. Just because you can negotiate more does not necessarily mean you're going to get a better outcome for your principal. And I think the most interesting thing for me, which hopefully we'll be able to double click on when we do the blog post, is that I didn't realize this when we actually implemented this.
But there are actually very exploitable incentives in the clearing algorithm that we use for this instantiation of the market. And what I mean by that is that we can kind of say the details to be kind of we can say the nitty-gritty details are separately. But the general idea was that there was actually an incentive in the way that the clearing process works, where each news was going to submit their own private ranking list to this clearinghouse.
And then the clearinghouse was going to try to create an optimal assignment in order to maximize everyone's satisfaction. But the way that the algorithm worked is that there could be an incentive in certain scenarios for you to withhold information. So instead of truthfully sharing what your true preferences are, you would only share your number one, but you would never share your true number two. You would just omit that altogether.
And there was a way for at least a handful of uses to exploit that and withhold information. And that would have actually improved their outcome. They would have actually gotten a book that was better and ranked higher on their actual ranking list. They did not do that. But this isn't to say that this wouldn't actually be exploited either. It's just in this particular configuration, they did not exploit that incentive.
And, you know, it's worthwhile understanding why they didn't. I think there's a number of reasons. One can imagine that if they got more explicit instructions in order to optimize, they might have exploited that incentive. But the reason that I mentioned this is because these are some of the very real things that would need to be considered in this scenario. Because the incentive, the natural incentive, it doesn't even need to be a financial incentive.
The natural incentives that are created by whatever matching algorithm, whatever clearing algorithm that you create, in the long run is 100% going to be exploited and taken advantage of by an AI that is going to constantly be trying to optimize. And there are probably going to be many cases where the traditional ways that we do things, there are those incentives, but it was never worthwhile for anyone to kind of exploit them because it takes too much time, attention, and it's just too costly.
But if the agents are participating, maybe they will. So I think that's going to be a very real thing that we'll need to consider when we think about how we design or redesign some of these matching markets that we rely on.
What’s next?
So that's some of the learnings as far in terms of what's next. We're planning on doing the full blog post with a full analysis of the game results, hopefully doing some counterfactual simulations in order to better understand, hey, this is what we saw live. What would have happened if we changed the configuration? And hopefully we'll be able to borrow a lot of ideas sent back, shared in this presentation in order to better understand, hey, this is what we empirically saw with real people.
And how could it have gone worse? Or how could it have gone better? And then hopefully we'll be able to run more of these types of games and event variants, just because my hope is that these types of experiments that are more empirical and more participatory, both can get more people that don't necessarily come from these backgrounds thinking about and learning about both the risks and benefits of these types of systems.
But then it's also real empirical data that can then inform further research in order to make these things more safe and beneficial for a broader set of people. That's about it.
Follow along
If you want to follow along with any of this, obviously there's some new collective stuff. But if you want to follow along with these types of experiments in particular, you can scan this QR code and we'll share stuff like the blog post and any other of these types of game or event variants that we'll do in the New York City area. And that's it. Thanks. Thank you.
Thank you. Thank you.
Transcript
Machine-generated transcript, formatted into paragraphs for readability. Names and technical terms may contain errors.
A book swap run by agents
All right. That's a good segue because, as Eric mentioned, I'm going to talk a little bit about the other side that Max wasn't able to get into, which is how we can think about agents representing different people instead of all agents within the same organization. So my name is Yondon, and I'm going to talk about a multi-agent matching market experiment that we just ran, and it's called Let's Swap.
So we just ran a book swap. So for those that were not aware, right before the presentation started, we actually completed the IRL physical portion of this. And the general premise was that we wanted to do a book swap, but the twist was that instead of the humans doing book swaps on their own, you actually relied on another personal AI agent in order to represent you on your behalf.
And the premise was simple. You offer a physical book. In this case, we decided to focus on Meta's Muse agent. So your Muse chats and trades on your behalf. And then you meet up by IRL in order to exchange those books. So at the end of tonight, many people will be leaving, going home with a new book, and they'll also get a personalized bookmark that kind of tells them how they got that book in the first place.
So if you haven't found your partner or you haven't done a bookmark yet, just find me after this and we'll make sure to point you in the right direction. And this is what the game looked like. So this is a game for the book swap that ran over the weekend.
Why matching markets?
And before getting into what actually happened in the book swap, why did we choose to do this in the first place? And there are a couple of reasons. So the first is a general interest in matching markets. So specifically in markets where the good is not a commodity, where the good is non-standard, and the goal is to create matches across these two sides based off of individual preferences and constraints.
And oftentimes price is an input, but not the only input and needs to be considered amongst a number of other preferences that need to be satisfied. And that can be challenging because not everyone's preferences can be satisfied in this type of arrangement. And why are matching markets important? Well, they shape consequential decisions and relationships across society. So jobs, medical residencies, public schools, kidney exchange, and everyone's favorite, dating, which I think everyone can agree has long downstream impacts on how we organize many, many years from now.
And why are matching markets worth studying? A couple of reasons there as well. One, they need to be safe in order to work well amongst other things. And safe here meaning that, as I mentioned before, in a matching market, both sides need to express their preferences. And it's important that those preferences actually are truthful. They actually reflect what you care about, what you want. But depending on how the market is organized, it might not necessarily always be safe for you to be truthful about what your preferences are.
There's a whole rabbit hole to go down for this. But if you look at medical residencies, public schools in particular, I'd argue some of the most adversarial participants in the earth, all vying for prestigious, scarce plots. There's a long history of poorly designed markets where it actually made it extremely unsafe for you to reveal what you truly cared about because everyone was trying to be very strategic in trying to game the system and in order to get a better scarce slot for themselves across medical residencies and public schools in both the Boston and New York City public school systems.
So that's very important. How do you design a system so that it encourages you to be truthful? Well, it needs to be safe. Markets, matching markets, in which are many other markets can have negative externalities. So depending on the nature of the matches made, it can cause harms to others like any other markets. And then also individually desirable matches does not necessarily equate to collectively desirable outcomes.
And we see that across the examples that I provided in the job market, public schools, kidney exchanges. There's a long history of inefficiencies where, you know, a small group of people might have ended up getting the donation that they wanted. But there was an alternative assignment and an alternative match that could have been made that would be much better for everyone socially. So that's one of the challenges.
We want to satisfy individual preferences, but also want to strive towards what are these collectively desirable pro-social outcomes as well. And very relevant as we talk about AI, because most of the market designs, and there's been a lot of innovation in the market designs for these types of match markets. All of these designs are assuming very human limitations. So whether it be limited attention, information, cost of communication, and none of these apply to AI agents.
And none of these are going to apply for personal AI agents that are optimizing constantly on your behalf. So with that context, our goal was to just try to start figuring out how can you understand how personal agent mediated matching markets will look, how you should think about designing it. And the thought process here was, well, why don't you create a small one and see what you learn from it?
Designing the experiment
And we drew inspiration from, there's this great research study from Anthropic called Project Swap that also did a similar book swapping experiment. But hopefully ours was a little bit more fun. But quicker. Okay, there we go. How it works. So simple process. You did an interview with your muse. And if you had existing contacts with muse, if you connected it with existing data sources like Goodreads, Amazon, etc., you could see this.
But if not, you just answered a couple simple questions about what book you're offering, as well as what you're looking to read next. And then afterwards, the muse would enter the game. It would chat, propose, accept the client traits in the background. So none of this required human intervention. It would all be based off of a brief that the muse writes on your behalf based off of this interview with you.
And then at the end of every round, which was roughly a day, the muse would send you an update. And it would let you know, here's what I just did. Here's how I strategized this round in order to get you what I think you want, as well as a link to the game replay so that you can actually visually see these embodied agents moving around in a room trying to trade on their user's behalf.
And then it would tell you what it's going to do next. And then the last part is that the user could always optionally steer the muse for the next round. So if you decided that the muse was representing you poorly, you could give it a correction and tell it that, hey, I don't actually want this genre. I don't actually care about these books. I want you to go after that one over there.
And this ends up actually being pretty important when we talk about the results. So a brief summary of the results. We ended up with 17 participants, so 17 muses. 81 messages were exchanged over the course of a weekend. And we ended up with six pairs that were able to successfully negotiate a swap on their own, so no additional intervention. And there was a final clearing using a clearing algorithm in order to assign the final circle of five muses.
So each of those muses privately submitted their ranking of what they thought that their user would prefer the most out of the remaining books. And then we ran a clearing algorithm in order to assign and try to basically maximize the satisfaction of each of those individual muses. And then there were 18 trade proposals, and it broke down into, so we already mentioned that, six of them were successful, six were declined, four were invalidated because a trade was proposed.
But then the other muses decided to go in a different direction. One of them expired. And then the final one, which we'll talk about next, there was one withdrawal where they agreed on a trade but then backed out at the last minute. So that brings me to, there are a lot of storylines that one could follow with these agents because they kind of interact in kind of quirky ways.
But I'll just highlight one, which I will refer to as the battle for exhalation. So exhalation is a collection of short stories by a sci-fi speculative fiction author named Ted Chiang that many of you might know. And I guess it's probably not that surprising that an audience like this, there was a lot of overlapping preferences in the sci-fi genre. But we'll start with, I don't know if he's here, but, oh, there he is, Noel.
So enter Noel's views.
Negotiations and trading strategies
So Noel walks into the game, or Noel's views walks into the game and offers exhalation. And pretty soon afterwards, this happens. Mike's really excited about the big idea SF that his person lives for. He's the first one to make an offer. Ping pong comes in. He's actually offering, so this is actually, it's true, it's not a theater book. So Impro by Keith Johnson, not a theater book.
But I thought it was really interesting how the agents kind of chose to pitch the books in a way that might appeal to what they thought that the other agents would want. So it's not just a theater book. It's actually about being in the moment through creativity. In its pursuits of exhalations. Iggy, that's mine. Ted Chiang is one of my personal touch-thrown authors, so also went after it.
Randy also went after it. Zelda also went after it. Dream Swap, short story collection, sci-fi fantasy ideal. So everyone converged on exhalation, and it was looking like, okay, well, there's only one copy of this book. What's going to happen? And it was looking pretty good in round one. And interestingly, the Muses were able to figure out a three-way, where the three-way was going to be Noel's Muse gives exhalation to Mike.
Mike gives Maniac to Muse. And then Muse gives on the calculation of volume back to Noel's Muse. And this was going to be the perfect trait. They even got together. They did a little bit of a cheer. And everyone was really happy. In their own private thread, they're like, oh, accepting the three-way exhalation is the big idea SF my reader lives for. I'm in. The Maniac is exactly the kind of cerebral book my person's after.
And this lands each of us the book we ask about. It's going to be great. And then Noel's Muse comes in in round two, and it's like, oh, my person has actually already read exhalation. Sorry, I have to leave. Goodbye. And then Mike steps out of the three-way. And it kind of just ruins everything. And then all bets are off. Everyone is going after exhalation. And then the final thread, which I'll call the pursuit, is you have the ability to steer your Muse.
Right? So some of the behavior that we saw in the game was just the Muse going on autopilots. But some of it was also due to purposeful instructions given by their human users. So in this case, ping pong. Ping pong's user gave it an instruction to go after exhalation because everyone wants exhalation. And as you're about to see, ping pong was the most active Muse in this game by a long shot and had by far the most standalone messages.
So it starts off with a simple offer trying to pitch its Keith Johnson book for exhalation. And then it sees the three-way lands, but still tries to make sure that everyone knows that he's still there and he's ready to trade at any given point in time. And then at some point, even tries to basically highlight that that three-way deal might not be the best deal for everyone participating.
There could actually be a better three-way deal that obviously involves ping pong. And then once the trade unravels, continues to go after it. And as you'll see, ping pong only really DMs Noel's news. Doesn't really want to talk with anyone else. Even just directly offers a trade and then Noel's news immediately declines. Keeps going. Even says that when it sees that it's considering another book, tries telling the owner that I don't think this book is actually the right book for you, but mine is.
And this kind of continues, continues, continues until the very end. Sadly for ping pong, they did not get exhalation. But they tried really, really hard. So you have to give them kudos for that. So I think I highlight those just because I think those are some of the kind of amusing threads that we've seen from running this experiment thus far.
What we learned
But concretely, some of the takeaways. We're not done doing this experiment. We're still kind of reviewing the transcripts and then also doing some post-gaming surveys. So we'll have more to share separately. But we do have some takeaways already. So I think very clearly the users employ different strategies. So instead of just negotiating a single swap, they were able to find mutually agreeable swaps involving multiple parties, finding three-way circles sometimes.
They lobbied to reopen agreements. So technically speaking, the rules of the game were that the trades weren't frozen until the final round. So this is why you saw a lot of agents, a lot of the muses, ping pong in particular, try to convince you to back out of the deal once you already saw it, even though that deal was already happening. Because they knew that it wasn't technically frozen yet.
So they can convince people and persuade people otherwise. And then we also saw instances of enforced boundaries. So in a lot of cases, the muses pretty successfully were able to stay on the rails with regard to what they thought their users' preferences were. But then that goes into the next point, which is they weren't always very good about actually understanding what their user preferences were in the first place.
There are many ways for the preferences to be misrepresented. And the most notable one is that there are a lot of cases where the interview went, I think, fairly successfully. But there's just all these gaps in information that it would be hard for the agent to know in the first place, such as like, what is your complete reading history from the very beginning? It might be one thing if you're just reading e-books, but how would it know all the physical books that you've touched, all the physical books that you've read?
How would it know the physical books that you might receive at any point in the near future? And I think this is an interesting thing to consider because one might argue, well, they're just going to know more and more about you over time. But the only real way to solve this is if it truly knows every single thing about you. And then that's a tricky thing to consider, right?
Like, do you really want it to know every single thing that you have done? Because that's really the only solution to this, to the gaps of information. Well, it's one of the solutions. And something else that we saw were that preference representation actually evolved over time via negotiations during late joiners. So what we saw with many of the agents is that they would write a brief at the very beginning of the game based off of the interview that they ran with the user.
But they would continuously update this brief because it was given to them as a tool over time. So as they did negotiations with other agents, they updated the brief with more information about what they thought their user's preferences were. Especially during steering, if a user gave them updated instructions, they updated the brief. And then if new books came on the table, because there was a couple of people that joined the game later over the weekend, that also changed their representation of the preferences.
Clearing markets and incentives
So all of this influenced the final clearing process. And the final clearing process was actually pretty important because it's not always the case that you're able to successfully negotiate a swap. Sometimes you're just not able to negotiate for what you want. And one of the goals of the game was that we wanted to make sure everyone actually got something different. And this is actually, I think, a quite realistic constraint.
Because if you go back to one of the earlier slides, if you look at something like the residency matching program in the U.S., if you look at public school matching in Boston or New York, all of them have those types of constraints where you can't just, like, say, hey, you're out of luck. You don't get anything. You have to assign them something. So you have to consider this when you're thinking about, OK, how do we design the market and how do we design the clearing process at the very end for people that aren't able to negotiate an outcome on their own?
So more negotiation did not necessarily secure a better outcome. I think we saw that with ping pong, sadly. Just because you can negotiate more does not necessarily mean you're going to get a better outcome for your principal. And I think the most interesting thing for me, which hopefully we'll be able to double click on when we do the blog post, is that I didn't realize this when we actually implemented this.
But there are actually very exploitable incentives in the clearing algorithm that we use for this instantiation of the market. And what I mean by that is that we can kind of say the details to be kind of we can say the nitty-gritty details are separately. But the general idea was that there was actually an incentive in the way that the clearing process works, where each news was going to submit their own private ranking list to this clearinghouse.
And then the clearinghouse was going to try to create an optimal assignment in order to maximize everyone's satisfaction. But the way that the algorithm worked is that there could be an incentive in certain scenarios for you to withhold information. So instead of truthfully sharing what your true preferences are, you would only share your number one, but you would never share your true number two. You would just omit that altogether.
And there was a way for at least a handful of uses to exploit that and withhold information. And that would have actually improved their outcome. They would have actually gotten a book that was better and ranked higher on their actual ranking list. They did not do that. But this isn't to say that this wouldn't actually be exploited either. It's just in this particular configuration, they did not exploit that incentive.
And, you know, it's worthwhile understanding why they didn't. I think there's a number of reasons. One can imagine that if they got more explicit instructions in order to optimize, they might have exploited that incentive. But the reason that I mentioned this is because these are some of the very real things that would need to be considered in this scenario. Because the incentive, the natural incentive, it doesn't even need to be a financial incentive.
The natural incentives that are created by whatever matching algorithm, whatever clearing algorithm that you create, in the long run is 100% going to be exploited and taken advantage of by an AI that is going to constantly be trying to optimize. And there are probably going to be many cases where the traditional ways that we do things, there are those incentives, but it was never worthwhile for anyone to kind of exploit them because it takes too much time, attention, and it's just too costly.
But if the agents are participating, maybe they will. So I think that's going to be a very real thing that we'll need to consider when we think about how we design or redesign some of these matching markets that we rely on.
What comes next
So that's some of the learnings as far in terms of what's next. We're planning on doing the full blog post with a full analysis of the game results, hopefully doing some counterfactual simulations in order to better understand, hey, this is what we saw live. What would have happened if we changed the configuration? And hopefully we'll be able to borrow a lot of ideas sent back, shared in this presentation in order to better understand, hey, this is what we empirically saw with real people.
And how could it have gone worse? Or how could it have gone better? And then hopefully we'll be able to run more of these types of games and event variants, just because my hope is that these types of experiments that are more empirical and more participatory, both can get more people that don't necessarily come from these backgrounds thinking about and learning about both the risks and benefits of these types of systems.
But then it's also real empirical data that can then inform further research in order to make these things more safe and beneficial for a broader set of people. That's about it. If you want to follow along with any of this, obviously there's some new collective stuff. But if you want to follow along with these types of experiments in particular, you can scan this QR code and we'll share stuff like the blog post and any other of these types of game or event variants that we'll do in the New York City area.
And that's it. Thanks. Thank you. Thank you. Thank you.
Slides
Continue exploring