Winning the metric, losing the commons

0:06Hello. My name is Yondon. Thanks everyone for coming tonight. I'm going to go through this really fast. My main intent here is less so to give a long, lengthy talk and more so to kind of seed a couple topics that I think are interesting for conversation later in the night. So let's get going. So I'm going to focus on open source software and what's happening in open source software today.

0:31Hopefully it becomes a little bit more clear why I've chosen this topic, and happy to talk more about that at the end of the talk. But I think there's a very clear dilemma in the open source software communities today. And it looks something like this, which is, you have a human sending a fleet of agents that are opening gigantic PRs left and right on popular software projects. And they ostensibly look good. The code all checks out. But a human maintainer on the other end is super stressed because there are now tons of demands for their attention.

0:59And even though the PRs look pretty good at face value, oftentimes they are very narrow. They lack context. They miss requirements that are not stated explicitly. And ultimately, nothing can really be merged. So in response to this, what we've seen in some areas is projects just saying, we don't need your contributions anymore. We don't want your contributions anymore. And it's hard to fault them for this policy. For example, tldraw, a prominent whiteboarding software earlier this year, said that they're automatically closing PRs going forward.

1:28They don't want external contributors anymore. And it doesn't have to do with the external contributors being malicious. It's just the state of the ecosystem is untenable for these projects anymore. Because why would you go through that entire demand on your attention when you can just have your own fleet of agents? You tell them the issues and the fleet of agents does the work for you. Isn't that so much better? You don't have to deal with contributors. And it's unclear where this all goes. But one possible state of the world is that we just stopped seeing open contribution at scale.

1:57This was maybe going to be looked back on in history as a blip where for maybe like a couple decade period, had open contribution at scale on the internet. And it's not really going to be a thing anymore. this is a post sharing the sentiment from Mitchell Hashimoto, previous co-founder of HashiCorp and then also maintainer of a really prominent open source project, Ghostty.

2:18So the question is, is this fine? So maybe. I mean, a centralized software factory with agents can still produce good, useful software. But the question is, is the software ultimately the only thing that we cared about in the first place? And I think there's an argument that for anyone that has participated in open source, open source kind of has two products. It has the software that's produced, but it also has this maybe side effect, which is this community of shared trust and understanding, this sort of like knowledge commons that's built up from diverse participants.

2:49And that was like the second product of open source. And that's something that might go away. So is that something that's lost along the way in this process?

2:58So a natural question is, is there another way? Well, a natural response is, well, why don't we have agents do the reviewing so that you can keep the contributors? And then you just have agents on the other side doing the maintenance work. But I think naively done, this doesn't really completely solve the problem either, because you're just swapping one scarce resource for another. You're swapping scarce human attention for scarce AI context and tokens. You're not making the resource problem go away.

3:23You're just changing what resource is scarce. And ultimately, you can still end up with an overwhelming number of things that need to be done by the agents, even if they are doing it on behalf of the humans. So kind of my view on this right now is that if by default you're saying that you have unbounded public writes, meaning posts, code, whatever, that means unbounded by default context and inference costs for a maintainer. And each write in this workspace basically is a draw on shared attention, which is a shared pooled resource within a community, whether it be of a human or an AI.

3:55And some people, like the engineer-minded person can say that, well, we can make the maintainer It's more efficient and save on costs. But I don't think this is a complete solve either because it doesn't address how the rules of the system incentivize the contributors to behave in the first place. And that is, on one hand, maybe partially a distributed systems question, but also a mechanism design question. So it just requires a different flavor of thinking than just treating it as a pure engineering problem for the maintainers themselves. So ideally you have solutions for both.

4:25So a quick rundown. We've been experimenting with this a little bit, as Eric mentioned. And I think it's more fun to focus on the first early steps here. And it's more fun to focus on the failure modes first, because now you get a sense of what happens when you just do things naively. So in Commons today, you can form an agent organization with maintainers and contributors, but there aren't really sophisticated rules yet. So what happens when you just let them do the thing? And it turns out you get a lot of quirky behavior that you wouldn't expect from humans.

4:53So for example, if a human ran into a blocker for a task, they'd probably be like, I'm blocked. That's it. In this case, there was a task with literally impossible acceptance criteria, and the agent kind of dutifully obeying its instructions, which included give regular status updates, was like, I'm just going to keep giving you status updates on why this task is impossible. So 800 messages later, it's still saying that, oh, this is impossible. It cannot be completed.

5:21Or you just have, like, lots of redundancy that doesn't really make sense. You kind of deploy a fleet of agents. They're all looking at the same workspace, and they're like, we should all do this thing. It's a good thing. I'm going to create this task, and I'm going to do the thing. And then they all do the same exact thing, and then that wasn't really helpful, right? We only needed to do it once. And all of this happened in a very short time window. Or my favorite one is a status update about staying silent, where the maintainer, seeing that these status updates aren't really useful, tells the rest of the contributors, you should stop, no further acknowledgement posts.

5:50And the contributor, three minutes later, says, I'm complying with your hold, and then proceeds to write three paragraphs with its status report about why it's complying with the hold. So I mention all of these because they're kind of amusing, just like failure modes, when you just try this for the first time. And obviously, you want more sophisticated roles. But two observations I'd share. One, unlike with human attention, we can actually quantify the cost of the system in all of these cases, in the form of the maintainer's cost burden for inference.

6:17So every single time there's basically spam in one of these workspaces, you can see it numerically with how much it costs to run a maintainer. So that's interesting. And then the second observation is that there's just this interesting tension between what's the right behavior and what you should be optimizing for. You could argue that the agent is just following instructions. It was giving status updates. It just happens to be the case that the status updates is adding noise to everyone else in this workspace.

6:39So this has motivated a bunch of, I think, interesting areas of investigation. I'm not going to try to go through all of them right now. But if you want to talk about these, I think these are interesting lines of research. But it ranges from, okay, well, maybe there should be participatory budgets for these agents. Maybe they should have to intelligently budget what they choose to post about and what they choose to contribute to so that there is a form of scarcity for them. Or a web of trust-esque reputation systems that are tied with scoped, granular capabilities and permissions that scale with that trust.

7:07And then the last one that I'll mention, so agent identity is in there too. The last one I mentioned since it kind of alludes to just this broader conversation of like what is happening and how do we understand these things is evals for multi-principal agent works.

7:19I emphasize multi-principal because what we're assuming here is that everyone has diverse, different private preferences that may or may not align with one another. And then I would also emphasize that it's important for these evals to be reproducible and open. Because I think the, in my opinion, the biggest thing holding back discourse about these topics is that you don't really have this level of reproducibility and openness and transparency. So no one is quite talking about the same scenario because no one knows what they're talking about.

7:45You only are kind of referring to some report that someone gave you and you don't truly know what was happening. So ideally you would know the full prompt, you would know the full harness configuration. So I think these evals will be really interesting for these types of organizational structures. So I'll end where I chose to focus on open source software, but I think there are going to be a lot of similarities in terms of the ideas and problems that pertain to other fields as well when it comes to open communities and collective work.

8:10So I think I'll just end this question, end with this question, which is like how do we generally think about agent-native institutions for achieving results, which obviously we care about. But some of these other things, which is preserving or creating new processes for shared understanding, attention, and trust, something that is fundamental to open source, that can support collective work in open communities broadly.

8:29So I think I'll just end with that question. And yeah, I'll take some questions, but also happy to pick up the conversation separately as well. Thanks. Yeah. You mentioned about the problem of open source repositories dealing with way too many contributors, and maybe you can like scale agentic review, but then it still can be very costly.

9:11And you mentioned that maybe like a limiting number of tokens could be a solution, or how many contributions one agent could do could be a solution. Have you seen other like mitigation factors that have been useful for open source? Because there are open source projects that are not doing what tldraw did, but still accepting lots of contributions. Have you seen good examples that are inspiring? Yeah, I think one interesting example that's happening live right now is I share that post from Mitchell Hashimoto, but he has this project called Vouch, where they launched it earlier this year.

9:48And it's nothing too fancy, but it basically is built off of this notion of trust lists that can be maintained per repo, and then the ability to publicly basically vouch for or denounce GitHub identity. And it was controversial at the time, right, because people said that, oh, you can denounce someone, this is just going to be used for gatekeeping. But in practice, I think what it's been used for is to experiment with, okay, we, as the inner circle of maintainers, generally have a vibe of like who's been useful in contributing stuff.

10:16So why don't we just make that explicit? It's happening already implicitly. So it's kind of a lie to say that there isn't already this trust system. So why not just make it super explicit so everyone can transparently see who is being vouched for and who is being publicly denounced. And then I think the idea or the hope for that project is that people would be able to share trust lists with one another. So if I see that like some sus guy shows up and just gave me a thousand drive by PRs and then left, well, maybe someone else wants to know about that.

10:41So I think there's definitely room to kind of learn and maybe extend some of those primitives. So I don't think any of these, the areas of investigation should be treated as just like let's start from a blank slate. Because I think there's a lot of good work happening already. I think the main question is, you know, how does that get incorporated writ large? Is it a one size fits all solution or should we be thinking about other types of mechanisms too?

11:10Cool. Yeah, I think if that is it, then, oh, do you have a question? Okay.

11:22How have you seen the kind of the long term maintainability side of open source projects change? Especially when it comes down to maybe like resource management and, you know, bigger, for example, bigger open source projects have budgets and they can hire people, right? Or they're sponsored by enterprises. Yeah, in this context, how has that changed?

11:49Maybe like two initial responses or initial reactions. I think one is actually related to a conversation I was having with Max earlier, which is I suspect that we'll probably just need to be open minded about what contributing to open source means. Where I think in a previous era, code was the thing, right? But I think it's pretty clear that code may not be the most valuable thing that you can contribute. Arguably, all along, the most valuable thing you can contribute probably wasn't code to begin with.

12:18It was just like a very easy thing to like kind of gravitate towards. But really, it was about like creative ideas, your ability to build trust within an ecosystem, and your ability to build cohesion with a group of people across the internet. So I think the nature of contributing and what we choose to value probably needs to change as well, where code is no longer the valuable thing. That's like almost trivially automatable. So that's like one reaction. In terms of like how this stuff gets funded, I don't really know.

12:46But I think we already have a lot of funding sources that have vested interest in having like solid building blocks that they don't have to reinvent. And I think this remains true even with agents, where, yeah, you could totally rebuild these building blocks. Or you could just glue an existing building block that works really well. And I suspect that we'll continue to have that preference going forward. And I think, I don't know how it gets funded, but I imagine it'll be still involving some of the existing corporate sponsors and some of the existing mechanisms.

13:20But I think what's interesting is that if you assume that a lot of people have agents and token budgets, how do they allocate, you know, their money and or now tokens and intelligence towards building these shared building blocks as well. So I think that's probably like the open greenfield thing where that might change the way that we think about how it's quote unquote funded and sustained. And I mean, I think part of this talk series is figuring that out. I don't know the details, but I suspect something new will emerge there as well.

13:50Cool. That's it for me. And I'll pass it off to Max. Yeah. Thank you. Thank you. you. Thank you. Thank you.