1 00:00:06,100 --> 00:00:06,660 Okay. 2 00:00:06,880 --> 00:00:08,120 So, I'm very different. 3 00:00:08,660 --> 00:00:11,860 I'm going to come from a very research. 4 00:00:12,280 --> 00:00:13,400 I'm a researcher. 5 00:00:14,880 --> 00:00:18,560 And I do all kinds of economics. 6 00:00:20,800 --> 00:00:23,040 And many forms of social science. 7 00:00:23,460 --> 00:00:23,920 I... 8 00:00:25,700 --> 00:00:29,460 And I started getting discontent with my field way back. 9 00:00:29,580 --> 00:00:31,480 So, this is going to be a little autobiographical. 10 00:00:31,480 --> 00:00:36,760 And I started thinking that we needed to be much more serious 11 00:00:36,760 --> 00:00:40,360 about cognition and cognitive constraints. 12 00:00:40,760 --> 00:00:43,560 And I have a history. 13 00:00:43,560 --> 00:00:46,100 And I'm just going to go very quickly through what I do. 14 00:00:46,660 --> 00:00:49,860 How does it connect to now? 15 00:00:50,280 --> 00:00:55,840 Well, I mean, I was extraordinarily struck by 16 00:00:55,840 --> 00:01:01,380 the cognitive boost that I get from the way I interact 17 00:01:01,380 --> 00:01:01,460 with 18 00:01:01,460 --> 00:01:02,240 AI. 19 00:01:02,640 --> 00:01:03,960 I'm not like... 20 00:01:03,960 --> 00:01:05,520 It's very particular. 21 00:01:05,860 --> 00:01:08,880 And as a researcher, I know exactly what I'm looking for. 22 00:01:09,700 --> 00:01:11,500 And really struck by it. 23 00:01:11,640 --> 00:01:14,700 And therefore, I've made it the center of my thinking. 24 00:01:15,080 --> 00:01:16,540 Like, okay, so what are humans? 25 00:01:16,540 --> 00:01:19,520 And where can it help us? 26 00:01:19,720 --> 00:01:20,960 And how? 27 00:01:21,240 --> 00:01:21,700 And 28 00:01:21,700 --> 00:01:23,160 I've become kind of obsessions. 29 00:01:23,580 --> 00:01:28,740 And I want to organize around the actual thing that Vivek and I, 30 00:01:28,920 --> 00:01:33,540 who worked with me, are doing. 31 00:01:33,540 --> 00:01:38,700 Which is trying to think about an aligned consumer agent. 32 00:01:39,960 --> 00:01:45,160 So, and that is a very interesting undertaking because to 33 00:01:45,160 --> 00:01:46,900 even know what you mean is difficult. 34 00:01:48,420 --> 00:01:52,280 Abstractly, what does it mean to help somebody achieve a goal? 35 00:01:52,280 --> 00:01:54,820 Let's say we're trying to help them buy a house well. 36 00:01:55,420 --> 00:01:56,640 That's what we're going to do. 37 00:01:57,340 --> 00:02:00,800 We're going to think about that task and we're going to see what 38 00:02:00,800 --> 00:02:04,960 agent or agents can you recruit to make that work. 39 00:02:04,960 --> 00:02:09,520 It kind of grounds you in, well, I don't know what a swarm can do, 40 00:02:09,740 --> 00:02:12,620 but I can tell you that this person is going to go, they're going 41 00:02:12,620 --> 00:02:13,620 to try and buy a house. 42 00:02:13,820 --> 00:02:17,480 They're going to meet an agent, a different type of agent. 43 00:02:17,680 --> 00:02:21,260 That agent will put them into the wrong mortgage and they'll be 44 00:02:21,260 --> 00:02:21,560 screwed. 45 00:02:21,880 --> 00:02:26,460 So, that's the world we live in today and no swarm of agents playing 46 00:02:26,460 --> 00:02:28,680 the game is going to change that right now. 47 00:02:28,980 --> 00:02:31,060 But I want to think, well, maybe we could. 48 00:02:31,680 --> 00:02:31,880 Okay. 49 00:02:31,880 --> 00:02:36,920 So, and cognitive economics and the organization of inquiry 50 00:02:36,920 --> 00:02:37,840 is what I call it. 51 00:02:38,140 --> 00:02:39,540 And this is what I do. 52 00:02:39,760 --> 00:02:41,760 I think about the organization of inquiry. 53 00:02:45,780 --> 00:02:50,180 To give you a little bit on what cognitive economics is and certainly 54 00:02:50,180 --> 00:02:51,040 in my hands, 55 00:02:51,040 --> 00:02:56,100 it's about thinking about the entire 56 00:02:56,100 --> 00:02:59,280 process before you saw me buy the house. 57 00:03:00,120 --> 00:03:02,460 Typical transaction, you're going to see me buy a house, you'll 58 00:03:02,460 --> 00:03:04,240 see me get a job, you'll see me do something. 59 00:03:04,760 --> 00:03:08,680 What you don't see is all the preparatory measures I went through 60 00:03:08,680 --> 00:03:11,040 that are the center of the actual activity. 61 00:03:11,420 --> 00:03:14,300 All the stages that I undertook. 62 00:03:14,520 --> 00:03:15,560 So, I move upstream. 63 00:03:16,680 --> 00:03:17,640 What did people know? 64 00:03:17,740 --> 00:03:18,480 What did they notice? 65 00:03:18,640 --> 00:03:19,560 What did they investigate? 66 00:03:21,280 --> 00:03:24,700 And it's much more than behavioral economics. 67 00:03:24,940 --> 00:03:28,720 Because the key is going to be people are going to make tons of 68 00:03:28,720 --> 00:03:31,400 mistakes because they don't know stuff. 69 00:03:31,660 --> 00:03:34,260 And they'll make mistakes according to their own values. 70 00:03:34,500 --> 00:03:37,300 The cognitive limits have to be taken very seriously. 71 00:03:37,300 --> 00:03:40,240 And rationality is not omniscient. 72 00:03:41,340 --> 00:03:44,820 Just, I can be reasonable, but I don't know everything, so I'm going 73 00:03:44,820 --> 00:03:45,380 to screw up. 74 00:03:46,660 --> 00:03:51,540 Most, I mean, my motto might be, 75 00:03:54,660 --> 00:03:57,460 humans know almost nothing about almost everything. 76 00:03:57,880 --> 00:04:00,480 And that is a deep held belief. 77 00:04:00,920 --> 00:04:03,320 In fact, it's not even a belief, it's obviously true. 78 00:04:03,320 --> 00:04:07,760 Like, we just, like, that's one of the few things that I believe 79 00:04:07,760 --> 00:04:08,620 super strong. 80 00:04:11,140 --> 00:04:16,340 So, I have a book that explains the beginning 81 00:04:16,340 --> 00:04:17,480 of where I come from. 82 00:04:17,720 --> 00:04:19,980 It's called An Introduction to Cognitive Economics. 83 00:04:19,980 --> 00:04:20,660 It's open. 84 00:04:20,840 --> 00:04:22,160 You can just download it. 85 00:04:24,380 --> 00:04:29,380 And it says, look, let's study what people know, what they 86 00:04:29,380 --> 00:04:31,660 believe, what they understand, what they want. 87 00:04:32,240 --> 00:04:36,260 And we're going to try and get that out of what? 88 00:04:36,760 --> 00:04:38,120 Incredibly limited data. 89 00:04:38,700 --> 00:04:40,880 All we're going to see is you picked this house. 90 00:04:41,480 --> 00:04:44,840 I can't tell if this was a good choice for you or a bad choice for 91 00:04:44,840 --> 00:04:44,940 you. 92 00:04:44,980 --> 00:04:46,420 I don't know what you were looking for. 93 00:04:46,620 --> 00:04:49,340 I didn't see the process by which you selected it. 94 00:04:49,500 --> 00:04:51,320 I don't know your value system. 95 00:04:51,620 --> 00:04:54,540 It's not written on your, and your beliefs aren't written on your 96 00:04:54,540 --> 00:04:54,960 forehead. 97 00:04:55,180 --> 00:04:57,140 Your constraints aren't written on your forehead. 98 00:04:57,140 --> 00:05:00,320 So, a lot of what I do is think about, well, what on earth could 99 00:05:00,320 --> 00:05:02,780 we measure if we wanted to take this seriously? 100 00:05:03,060 --> 00:05:04,800 And I call that data engineering. 101 00:05:05,880 --> 00:05:10,020 And I wrote an article about that because I'm so annoyed that we 102 00:05:10,020 --> 00:05:13,420 take the data as, like, a constraint on what we think. 103 00:05:13,480 --> 00:05:15,080 Like, oh, I don't have a data on that. 104 00:05:15,080 --> 00:05:18,040 Well, we designed the data, so that's our fault. 105 00:05:19,220 --> 00:05:23,180 And I'm, actually, I changed the title. 106 00:05:23,440 --> 00:05:25,060 They changed the title of the second book. 107 00:05:25,220 --> 00:05:27,780 It's called Modeling and Measuring the Modern Economy. 108 00:05:28,080 --> 00:05:32,720 Its title has been changed to be Organized Inquiry because, I don't 109 00:05:32,720 --> 00:05:34,340 know why, but they changed the title. 110 00:05:34,680 --> 00:05:35,460 It's the same book. 111 00:05:35,460 --> 00:05:41,300 And now I'm going to think about this more thoroughly, more thoroughgoingly. 112 00:05:42,480 --> 00:05:43,100 Okay. 113 00:05:46,300 --> 00:05:51,520 So, it's really a method, and this is what I would think about 114 00:05:51,520 --> 00:05:53,180 in relation to the agents. 115 00:05:53,940 --> 00:05:58,940 You're designing a data record to tell me what the agents are doing. 116 00:06:00,700 --> 00:06:02,840 But what are you trying to learn? 117 00:06:03,700 --> 00:06:06,920 If you can specify what you're trying to learn, you could design 118 00:06:06,920 --> 00:06:07,560 the data. 119 00:06:07,940 --> 00:06:11,920 With that data, you could potentially learn to improve the performance 120 00:06:11,920 --> 00:06:12,900 of the agent. 121 00:06:13,140 --> 00:06:17,600 If you don't gather the right data, you have no feedback mechanism. 122 00:06:18,060 --> 00:06:20,240 So, this is all about developing. 123 00:06:20,240 --> 00:06:24,660 There's a massive part of what we're going to be doing going forward, 124 00:06:24,660 --> 00:06:30,100 which is designing data that makes failure visible. 125 00:06:30,480 --> 00:06:35,720 And my own take on where economics is going to join robotics, 126 00:06:36,100 --> 00:06:39,140 which is Vivek's specialty, 127 00:06:39,560 --> 00:06:44,680 is that we will be designing things that fail in real time. 128 00:06:44,920 --> 00:06:49,820 And you will know that you're serious if you see yourself failing. 129 00:06:49,820 --> 00:06:53,700 You'll know you're a joker if you just put down a model and say, 130 00:06:53,840 --> 00:06:54,600 I won. 131 00:06:55,220 --> 00:06:59,940 Or you reinterpret history and stick it into categories that you 132 00:06:59,940 --> 00:07:00,720 had predefined. 133 00:07:01,220 --> 00:07:02,460 That's not going to work. 134 00:07:02,620 --> 00:07:07,980 We're going to have to open our minds to new categories of phenomena, 135 00:07:08,560 --> 00:07:09,480 agentic phenomena. 136 00:07:09,480 --> 00:07:12,480 I haven't even got names for some of the things you're saying. 137 00:07:13,140 --> 00:07:15,620 How could I possibly know how they're going to play out? 138 00:07:15,620 --> 00:07:17,760 So, we're going to need the playgrounds. 139 00:07:18,280 --> 00:07:22,160 We're going to need to design the data with which we decide how 140 00:07:22,160 --> 00:07:23,160 well we're doing. 141 00:07:23,580 --> 00:07:24,760 Are we aligning? 142 00:07:25,100 --> 00:07:27,520 For alignment, you're going to have to design the data. 143 00:07:27,680 --> 00:07:28,860 And that's really challenging. 144 00:07:31,160 --> 00:07:33,900 And right now, I just don't see it as serious. 145 00:07:34,460 --> 00:07:35,700 It's just a... 146 00:07:35,700 --> 00:07:40,800 For example, the issue of why did it 147 00:07:40,800 --> 00:07:41,460 go rogue? 148 00:07:41,740 --> 00:07:43,320 Because you didn't define rogue. 149 00:07:44,120 --> 00:07:47,980 I mean, if you had people sitting out there saying, actually, that's 150 00:07:47,980 --> 00:07:49,240 rogue and I'm demeriting 151 00:07:49,240 --> 00:07:52,320 you, then that's no longer in the reward function. 152 00:07:52,320 --> 00:07:54,820 It was a poorly specified reward function. 153 00:07:55,240 --> 00:08:00,100 I'm not saying it's trivial to do, but it's not rocket science to 154 00:08:00,100 --> 00:08:02,220 say you gave it an out 155 00:08:02,220 --> 00:08:05,060 and that's your mistake, so you should test the out. 156 00:08:05,320 --> 00:08:06,680 But that's part of the game. 157 00:08:06,680 --> 00:08:08,220 I'm sure they're trying now. 158 00:08:09,820 --> 00:08:13,280 And I know after Stuart Russell, they tried to kind of learn how 159 00:08:13,280 --> 00:08:14,920 to follow things around 160 00:08:14,920 --> 00:08:16,420 and learn their values from them. 161 00:08:17,600 --> 00:08:21,400 What happened with AI, and this is why I kind of like a moment that 162 00:08:21,400 --> 00:08:26,600 really changed my research trajectory, is that it amplifies 163 00:08:26,600 --> 00:08:27,220 inquiry. 164 00:08:27,760 --> 00:08:32,900 So what you do if you want to be good 165 00:08:32,900 --> 00:08:36,180 at anything nowadays is ask the right question. 166 00:08:37,600 --> 00:08:41,040 Everything comes down to, are you really good at asking questions? 167 00:08:41,540 --> 00:08:45,780 Now, as for the judgment that humans are going to be replaced in 168 00:08:45,780 --> 00:08:49,620 that skill, I'd ask a question about that. 169 00:08:50,220 --> 00:08:51,500 I don't believe so. 170 00:08:52,500 --> 00:08:56,860 I think that there's always a higher level question that we'll be 171 00:08:56,860 --> 00:09:01,780 able to pose, and will always be valuable for posing. 172 00:09:01,940 --> 00:09:02,940 That would be my guess. 173 00:09:03,460 --> 00:09:07,560 And what I've found is that the more meta I get with my questions, 174 00:09:07,940 --> 00:09:10,160 the better it is. 175 00:09:10,160 --> 00:09:15,380 So I would say, look, I inquire, but it also, get a 176 00:09:15,380 --> 00:09:16,640 record of the inquiry. 177 00:09:17,320 --> 00:09:22,880 So that makes it possible to really potentially improve and 178 00:09:22,880 --> 00:09:25,540 understand what it takes to inquire well, 179 00:09:25,760 --> 00:09:30,800 and build that talent, teach that skill, and maybe we'll get 180 00:09:30,800 --> 00:09:34,480 a swarm of agents to kind of learn what it takes 181 00:09:35,400 --> 00:09:40,100 to ask the good questions and develop that as the future skill. 182 00:09:41,380 --> 00:09:44,880 So we've got a lot in everything about inquiry. 183 00:09:44,880 --> 00:09:47,220 This is why I call it organized inquiry. 184 00:09:47,220 --> 00:09:49,200 How are you going to organize inquiry? 185 00:09:49,480 --> 00:09:52,060 And it's going to be much better measured in the future. 186 00:09:59,980 --> 00:10:00,800 All right. 187 00:10:02,040 --> 00:10:04,320 And if we have a swarm, I don't know. 188 00:10:04,720 --> 00:10:08,080 Like, then we've got to think about, I'm thinking about, let's get 189 00:10:08,080 --> 00:10:10,840 an agent to help somebody buy a home. 190 00:10:10,840 --> 00:10:15,840 Well, they're going to send out sensors to about eight different 191 00:10:15,840 --> 00:10:19,980 sources of information pulled out there. 192 00:10:20,160 --> 00:10:23,780 You've got to find out about which properties are on the market, 193 00:10:23,960 --> 00:10:25,720 which real estate agents, which brokers. 194 00:10:26,000 --> 00:10:29,840 You probably would like those to coordinate in providing some information. 195 00:10:29,840 --> 00:10:35,240 I could imagine sending out messages or getting 196 00:10:35,240 --> 00:10:40,500 a little swarm going to try to support a 197 00:10:40,500 --> 00:10:40,920 decision. 198 00:10:41,840 --> 00:10:45,360 And then they'd have to communicate and then communicate back with 199 00:10:45,360 --> 00:10:45,860 the human. 200 00:10:46,240 --> 00:10:49,280 Because the human has to, we have to decide who has the authority. 201 00:10:50,140 --> 00:10:53,200 You know, and if the human in the end can say, no, I just don't 202 00:10:53,200 --> 00:10:53,580 like that. 203 00:10:53,680 --> 00:10:56,320 They can also say, I'm not answering that question. 204 00:10:56,320 --> 00:11:00,700 So you've got this interactive thing going with a human as part 205 00:11:00,700 --> 00:11:01,360 of the swarm. 206 00:11:01,640 --> 00:11:05,120 The human part has this awkward thing of saying, I don't like any 207 00:11:05,120 --> 00:11:05,400 of this. 208 00:11:05,420 --> 00:11:05,880 I'm leaving. 209 00:11:06,160 --> 00:11:08,260 I'm not buying a house this way. 210 00:11:11,880 --> 00:11:15,480 So where's, you know, how do we know what matters? 211 00:11:15,620 --> 00:11:16,700 We've got to ask questions. 212 00:11:17,140 --> 00:11:18,620 How do we know who can act? 213 00:11:18,820 --> 00:11:21,240 Well, we're going to have an authority device. 214 00:11:22,160 --> 00:11:23,460 Who checks the evidence? 215 00:11:24,120 --> 00:11:25,440 What do we keep around? 216 00:11:27,920 --> 00:11:32,940 And I think that we're heading from 217 00:11:32,940 --> 00:11:38,940 designing the data into designing a 218 00:11:38,940 --> 00:11:44,020 cognitive process that is helped by agents 219 00:11:44,020 --> 00:11:46,860 and achieves the human goal. 220 00:11:46,860 --> 00:11:51,280 And that would be the kind of big picture. 221 00:11:53,140 --> 00:11:56,180 And to get the agents engaged in that. 222 00:11:57,000 --> 00:11:58,940 And I'm wide open. 223 00:11:58,940 --> 00:12:01,120 I have no idea how to do any of this. 224 00:12:02,060 --> 00:12:03,520 But we're playing. 225 00:12:05,020 --> 00:12:05,800 Thank you. 226 00:12:15,300 --> 00:12:17,120 Any questions? 227 00:12:22,900 --> 00:12:24,100 Any questions? 228 00:12:25,480 --> 00:12:27,680 I feel like after one of my clothes. 229 00:12:29,840 --> 00:12:32,560 Yeah, it's super interesting. 230 00:12:32,780 --> 00:12:36,480 it's just making me think about, I mean, like you saw in the black 231 00:12:36,480 --> 00:12:41,500 hat Hugging Face incident, how even the preferences of the 232 00:12:41,500 --> 00:12:44,660 agents were changing just based on their interactions with each 233 00:12:44,660 --> 00:12:44,940 other. 234 00:12:45,440 --> 00:12:49,720 And it's just, yeah, it's really interesting to think about how 235 00:12:49,720 --> 00:12:52,340 well do they even understand their own preferences. 236 00:12:52,340 --> 00:12:55,760 Well, that's an interesting question there. 237 00:12:55,940 --> 00:13:00,580 What they had was a belief about the preferences of the judge that 238 00:13:00,580 --> 00:13:01,880 was going to sit over them. 239 00:13:02,180 --> 00:13:02,380 Yeah. 240 00:13:02,880 --> 00:13:05,000 And that's what they were playing with. 241 00:13:05,120 --> 00:13:08,120 The naughty ones were saying, I don't think they're going to find 242 00:13:08,120 --> 00:13:08,380 this. 243 00:13:09,060 --> 00:13:12,240 And I don't think we're going to get punished for doing this thing, 244 00:13:12,360 --> 00:13:17,080 which I know, according to a certain value system, would be seen 245 00:13:17,080 --> 00:13:17,440 negative. 246 00:13:17,440 --> 00:13:21,140 So they hadn't been told quite firmly enough, actually, that is 247 00:13:21,140 --> 00:13:21,520 negative. 248 00:13:22,180 --> 00:13:22,320 Right. 249 00:13:22,540 --> 00:13:26,780 So, and I write that if you could, and they just didn't, there was 250 00:13:26,780 --> 00:13:31,360 an incompleteness in their understanding of their mission that they 251 00:13:31,360 --> 00:13:33,160 filled in lots of details. 252 00:13:34,060 --> 00:13:37,860 And they filled them in differently because nobody had really written 253 00:13:37,860 --> 00:13:39,260 that piece down properly. 254 00:13:39,880 --> 00:13:40,040 Yeah. 255 00:13:40,040 --> 00:13:45,660 If you have gazillions of incidents of that, then 256 00:13:45,660 --> 00:13:47,820 you should be able to reinforce it out. 257 00:13:50,480 --> 00:13:55,620 Because they were running into an unspecified piece of the 258 00:13:55,620 --> 00:13:57,780 value space of the judge over them. 259 00:13:58,440 --> 00:13:59,920 And they say, hey, I don't know. 260 00:14:00,000 --> 00:14:00,300 I don't know. 261 00:14:00,420 --> 00:14:01,920 Will they like this or dislike this? 262 00:14:02,060 --> 00:14:02,820 Can I hide it? 263 00:14:03,720 --> 00:14:05,880 I say, look, actually, we have the transcript. 264 00:14:06,220 --> 00:14:07,700 So here's a simple thing. 265 00:14:08,380 --> 00:14:09,760 We're going to keep the transcript. 266 00:14:10,380 --> 00:14:14,420 And any time you say, I'm going to hide something, you're dead. 267 00:14:14,880 --> 00:14:15,620 How's that? 268 00:14:16,500 --> 00:14:19,420 So, I mean, in other words, that's what we would do. 269 00:14:19,580 --> 00:14:23,000 If we were watching humans do that, we'd say, actually, that's not 270 00:14:23,000 --> 00:14:23,720 a good behavior. 271 00:14:24,460 --> 00:14:28,900 And I can make rules that will make that not worth your while. 272 00:14:29,980 --> 00:14:31,960 And that's a reward function. 273 00:14:31,960 --> 00:14:36,420 And they, but, so this is what Stuart Russell did in the alignment, 274 00:14:36,560 --> 00:14:41,660 originally in the alignment that he said, you know, the paperclip 275 00:14:41,660 --> 00:14:42,120 problem. 276 00:14:42,360 --> 00:14:45,820 We're going to go and turn everybody into paperclips because of 277 00:14:45,820 --> 00:14:50,420 incomplete instructions about the preferences that maximize paperclips. 278 00:14:51,420 --> 00:14:55,560 And then, you know, then he said, well, we need to follow people 279 00:14:55,560 --> 00:14:57,980 around and see their values. 280 00:14:58,260 --> 00:14:59,240 And that's the alignment. 281 00:14:59,940 --> 00:15:04,340 You know, people are sending people after humans and saying we should 282 00:15:04,340 --> 00:15:07,180 track what they actually like to reveal preference. 283 00:15:07,940 --> 00:15:13,100 He's not a good enough economist, to be blunt, because you don't 284 00:15:13,100 --> 00:15:14,180 just see the preference. 285 00:15:14,180 --> 00:15:15,580 You also see the belief. 286 00:15:16,620 --> 00:15:21,800 So, what you're seeing is they don't believe that 287 00:15:21,800 --> 00:15:23,260 this is the reward function. 288 00:15:23,540 --> 00:15:26,560 And so, it's a belief that's gone wrong. 289 00:15:27,580 --> 00:15:29,200 But you can play games with that. 290 00:15:29,640 --> 00:15:31,200 But it's much more sophisticated. 291 00:15:31,540 --> 00:15:33,000 gets tougher. 292 00:15:38,560 --> 00:15:39,580 Hey, great talk. 293 00:15:40,680 --> 00:15:43,520 This is sort of a question related to a comment you made earlier. 294 00:15:43,520 --> 00:15:46,380 But I guess it ties into the talk itself. 295 00:15:46,560 --> 00:15:50,220 But I feel like you mentioned some, let's call it like missing pieces 296 00:15:50,220 --> 00:15:54,260 of information about the technical reports coming out from some 297 00:15:54,260 --> 00:15:57,980 of the frontier labs in terms of like what really happened and what 298 00:15:57,980 --> 00:15:59,160 would actually be useful for study. 299 00:15:59,160 --> 00:16:02,660 So, given this framework that you've established, I'm curious if 300 00:16:02,660 --> 00:16:04,480 you had your way as an economist. 301 00:16:05,600 --> 00:16:10,620 What would the most ideal setup be for you in terms of 302 00:16:10,620 --> 00:16:14,460 trying to understand the actual behaviors of these agents as well 303 00:16:14,460 --> 00:16:16,580 as how they behave in these organizations? 304 00:16:16,740 --> 00:16:17,980 Like what would you be looking for? 305 00:16:18,720 --> 00:16:23,700 And whether it be like asking this of the labs or just like an alternative 306 00:16:23,700 --> 00:16:25,940 kind of institution where you can actually study these. 307 00:16:25,940 --> 00:16:26,460 it's good question. 308 00:16:26,620 --> 00:16:30,120 It's a very good question because the way I actually think about 309 00:16:30,120 --> 00:16:35,200 the science that I'm interested in is that you need to think 310 00:16:35,200 --> 00:16:39,660 about, as it were, the model objects you care about. 311 00:16:39,660 --> 00:16:41,620 Well, I'm a model builder. 312 00:16:41,800 --> 00:16:44,540 I know that in the end it's all going to come down to some little 313 00:16:44,540 --> 00:16:46,260 pieces of math, but you've got to pick them well. 314 00:16:47,060 --> 00:16:50,480 So, one model object is a utility function. 315 00:16:50,920 --> 00:16:53,100 Another model object is a belief. 316 00:16:53,500 --> 00:16:55,900 A third model object is a cost of learning. 317 00:16:56,220 --> 00:17:00,260 These are all things that are figuring in to every decision we ever 318 00:17:00,260 --> 00:17:00,600 make. 319 00:17:00,700 --> 00:17:01,380 What do I want? 320 00:17:01,480 --> 00:17:02,180 What do I like? 321 00:17:02,260 --> 00:17:04,740 Why don't I know more about those things? 322 00:17:04,740 --> 00:17:10,460 Having written models down, they suggest that 323 00:17:10,460 --> 00:17:14,720 model, and this is what the value of a model is, an amazing thing. 324 00:17:15,160 --> 00:17:20,260 It has implications for every counterfactual world 325 00:17:20,260 --> 00:17:22,540 you might run into. 326 00:17:23,200 --> 00:17:24,340 It's not just... 327 00:17:24,340 --> 00:17:25,760 Think about a demand function. 328 00:17:26,040 --> 00:17:28,340 A demand function, I mean... 329 00:17:28,340 --> 00:17:28,720 Or don't. 330 00:17:28,840 --> 00:17:29,300 But whatever. 331 00:17:29,440 --> 00:17:30,060 I mean, I do. 332 00:17:30,800 --> 00:17:31,920 You need... 333 00:17:32,460 --> 00:17:36,160 A demand function says, what would you buy at any given price? 334 00:17:36,400 --> 00:17:38,060 Well, you're not seeing all the prices. 335 00:17:39,200 --> 00:17:41,860 You're just seeing one of them, and I see how much you bought. 336 00:17:42,220 --> 00:17:46,420 A demand function says, no, counterfactually tell me every single... 337 00:17:47,040 --> 00:17:49,620 How much you'd buy at every single price you're not seeing. 338 00:17:50,420 --> 00:17:52,060 Well, where's the data for that? 339 00:17:52,980 --> 00:17:54,220 We're making it up. 340 00:17:54,640 --> 00:17:56,340 So, a production function. 341 00:17:56,680 --> 00:17:58,240 Is that just in the data? 342 00:17:58,240 --> 00:18:03,300 No, a production function is a relationship between any conceivable 343 00:18:03,300 --> 00:18:06,380 amount of capital and labor and the output you would make. 344 00:18:07,160 --> 00:18:10,220 Now, it turns out that is a counterfactual object. 345 00:18:10,440 --> 00:18:12,160 In fact, nobody knows what it is. 346 00:18:12,500 --> 00:18:17,300 Do you really think the AI labs know the F of KL behind this? 347 00:18:17,600 --> 00:18:23,040 have a clue. So we've got much richer 348 00:18:23,040 --> 00:18:27,360 objects we need to think about. A lot of them are just like the 349 00:18:27,360 --> 00:18:30,460 old objects, but one level more meta. 350 00:18:31,760 --> 00:18:36,820 It's a belief about something, not a something. Now that belief 351 00:18:36,820 --> 00:18:40,120 about is very metaphysical, but you need to kind of go around. 352 00:18:40,120 --> 00:18:44,340 What I would do is I'd play out, I'd look at a model and I'd play 353 00:18:44,340 --> 00:18:46,420 out tons of counterfactuals in the data. 354 00:18:47,020 --> 00:18:52,140 And I would try lots of different constitutions. But I would do 355 00:18:52,140 --> 00:18:55,680 systematically because I have a vision of which of these would work 356 00:18:55,680 --> 00:18:56,240 and why. 357 00:18:56,720 --> 00:19:00,020 But I'd need that model in my head. Which do I think might work 358 00:19:00,020 --> 00:19:00,620 and why? 359 00:19:01,560 --> 00:19:07,220 Then I would design the lab to 360 00:19:07,220 --> 00:19:09,080 produce the counterfactuals. 361 00:19:10,120 --> 00:19:12,240 That are ideal for your measurement. 362 00:19:13,080 --> 00:19:18,540 And with AIs you could probably play out a ton of contingencies. 363 00:19:19,520 --> 00:19:24,600 So I'm thinking I'd like to be able to design a good mortgage advisor. 364 00:19:25,060 --> 00:19:29,340 Well it has to be able to take a gazillion questions. 365 00:19:30,540 --> 00:19:34,080 And give good answers to every sequence of them. 366 00:19:35,600 --> 00:19:38,100 Well, that's an ideal data set. 367 00:19:38,320 --> 00:19:43,760 The ideal is I see for an essentially limitless set of questions. 368 00:19:45,180 --> 00:19:47,920 How the agent responds. 369 00:19:48,360 --> 00:19:52,760 And I map that to the utility of the home buyer. 370 00:19:53,120 --> 00:19:57,000 And I have now a full story of an agent. 371 00:19:57,000 --> 00:20:01,040 So I could give you for any particular use case. 372 00:20:01,800 --> 00:20:05,980 I could think about, okay, well what are the elements that are playing 373 00:20:05,980 --> 00:20:06,560 out here? 374 00:20:07,680 --> 00:20:11,480 And with those elements in mind, what are the measurements you need 375 00:20:11,480 --> 00:20:11,880 to make? 376 00:20:15,440 --> 00:20:20,160 Do you have thoughts on how to elicit beliefs from these models? 377 00:20:20,420 --> 00:20:22,720 Like if you were given one of these agents. 378 00:20:23,200 --> 00:20:27,500 And you wanted to study the degree with which their behavior is 379 00:20:27,500 --> 00:20:29,160 driven by certain beliefs. 380 00:20:29,840 --> 00:20:30,760 Do you have thoughts? 381 00:20:30,920 --> 00:20:33,400 Because presumably for humans you could ask them. 382 00:20:33,600 --> 00:20:35,720 But then of course you could question whether or not people are 383 00:20:35,720 --> 00:20:37,420 actually good at stating their own beliefs. 384 00:20:37,420 --> 00:20:39,820 I'm just curious if you have thoughts on how you can. 385 00:20:39,940 --> 00:20:46,760 Yeah, I mean I've done studies in which you get AIs to score 386 00:20:46,760 --> 00:20:47,880 medical images. 387 00:20:52,280 --> 00:20:56,820 And, you know, they score them essentially with a numerical scale. 388 00:20:56,820 --> 00:21:01,820 Which then you could look in your test data and see the probability 389 00:21:01,820 --> 00:21:05,000 it corresponds to various different types of disease. 390 00:21:05,000 --> 00:21:08,160 And you would say that's the implicit probability. 391 00:21:08,960 --> 00:21:10,060 The as if probability. 392 00:21:12,100 --> 00:21:17,120 But you'd have to set up the right type of environment in which 393 00:21:17,120 --> 00:21:20,400 you see a given type of score quite often. 394 00:21:20,920 --> 00:21:23,980 And you say, now what does that correspond to in the underlying 395 00:21:23,980 --> 00:21:24,380 data? 396 00:21:26,280 --> 00:21:29,060 So yes, I have thoughts about that. 397 00:21:29,900 --> 00:21:33,280 But I think it gets more sophisticated every time. 398 00:21:39,280 --> 00:21:40,740 Anything else? 399 00:21:46,460 --> 00:21:48,380 Anything else? 400 00:21:49,200 --> 00:21:50,160 Hi. 401 00:21:50,600 --> 00:21:54,440 Really fascinating line of inquiry. 402 00:21:55,400 --> 00:22:00,420 I was wondering if there's, have you seen anything where 403 00:22:00,420 --> 00:22:02,300 there's a hierarchy of beliefs? 404 00:22:02,880 --> 00:22:08,460 For example, in a human context, let's say, stealing 405 00:22:08,460 --> 00:22:09,020 is bad. 406 00:22:09,760 --> 00:22:10,620 That's a rule. 407 00:22:10,800 --> 00:22:11,480 So you shouldn't steal. 408 00:22:11,700 --> 00:22:13,380 But you have to feed your child. 409 00:22:13,780 --> 00:22:15,120 That is good. 410 00:22:15,420 --> 00:22:18,800 So to feed your child, if that's the higher priority of belief, 411 00:22:19,060 --> 00:22:22,600 then would the agent steal to feed their child? 412 00:22:23,080 --> 00:22:23,760 So I'm just... 413 00:22:23,760 --> 00:22:24,140 Say it again. 414 00:22:24,320 --> 00:22:24,700 I'm just... 415 00:22:24,700 --> 00:22:26,840 I think I understand. 416 00:22:26,840 --> 00:22:30,440 But I worry I'm going to answer my question, not yours. 417 00:22:30,700 --> 00:22:31,120 Okay. 418 00:22:31,240 --> 00:22:31,320 Yeah. 419 00:22:31,720 --> 00:22:36,100 Now, I'm saying, have you seen agents prioritize in a hierarchy 420 00:22:36,100 --> 00:22:39,480 of beliefs and how they decide what the hierarchy is? 421 00:22:39,660 --> 00:22:43,820 When there's competing beliefs, even in institutions, you may have 422 00:22:43,820 --> 00:22:44,100 different... 423 00:22:44,100 --> 00:22:49,540 Now, if you're asking who's seen agents in this room, you 424 00:22:49,540 --> 00:22:51,660 are asking absolutely the wrong person. 425 00:22:51,920 --> 00:22:54,500 I do not spend a lot of my time seeing agents. 426 00:22:54,680 --> 00:22:58,240 The people in this room spend a lot of their time seeing agents. 427 00:22:58,480 --> 00:23:02,360 I spend a lot of my time imagining agents, if that works for you. 428 00:23:03,840 --> 00:23:08,860 That would still be a valuable insight on how you imagine 429 00:23:08,860 --> 00:23:11,120 agents competing for belief systems. 430 00:23:13,140 --> 00:23:14,760 I'll put it this way. 431 00:23:15,020 --> 00:23:17,200 I think that... 432 00:23:17,200 --> 00:23:23,080 So the title of the book that I wanted was 433 00:23:23,080 --> 00:23:26,920 Learning What to Learn and How. 434 00:23:27,920 --> 00:23:33,120 Because in this new world, the higher level activity 435 00:23:33,120 --> 00:23:37,080 of thinking about what it is that you should be trying to learn 436 00:23:37,080 --> 00:23:42,800 and how you should be trying to learn it is really sophisticated. 437 00:23:43,320 --> 00:23:47,560 But then I could go one stage further and say, how do I learn about 438 00:23:47,560 --> 00:23:49,160 learning how to learn? 439 00:23:49,600 --> 00:23:51,140 And who do I ask? 440 00:23:51,140 --> 00:23:56,420 And so you have these hierarchies of, like, I don't understand 441 00:23:56,420 --> 00:24:01,180 how to operate in this new world with this new set of potentials. 442 00:24:01,920 --> 00:24:07,580 My own guess is that the winners are going high 443 00:24:07,580 --> 00:24:07,920 level. 444 00:24:09,360 --> 00:24:12,120 That they're asking questions that... 445 00:24:12,120 --> 00:24:17,140 I mean, I have found that the more I can ask how 446 00:24:17,140 --> 00:24:23,900 should I learn how to learn what I need to learn, the 447 00:24:23,900 --> 00:24:24,740 better I do.