Representative appearances

• I'm 100% playing this..

  Analysis of 'janky' versus 'intuitive' mechanics in RPGs.

• Pim De Witte on Turning Down $500 Million from OpenAI... Then Raising $134M.

  Discussed as the 'highest level of human intelligence,' evolving into 'world creation' with AI tools, allowing for new bets on experimentation.

• Francois Chollet + Mike Knoop | ARC Prize @ MIT

  Since we've been talking about the game design, let's switch over because I think that's this is one of my favorite parts of my job right now is every Wednesday for a few hours we get to like play games and give feed. It's a little grueling sometimes. You know, not all the games are good yet, but yeah, literally, you know, we're sort of paid to play games. I think collectively we've probably played over a hundred of them uh and given feedback. Do you want to talk a little bit about some of the common feedback we're finding and sort of playing these early preview games and giving feedback to the game design team? Yeah, we have the most productive uh game design studio in the world. It's already over 100 games in less than a year. So, it's actually very fun like making games like small games at scale and play testing them because you start uh really thinking deep about things like what makes a game fun for instance you try you try to come up with a theory of fun to reverse engineer what makes a good game. So one thing we've observed is the most fun you're having is when the game the environment is trying to maximize the rate at which you're learning. In order to be fun the game should offer some kind of challenge, right? It should be something you have to figure out something you cannot already do. If you just recognize what you have to do and you do it, then it's very boring. It's very it's very repetitive. You're just in execution mode. So it should have some kind of challenge, some kind of a novelty element. It should surprise you in some way. But at the same time, it should be tractable. It should not be too challenging or too novel because then you would just get stuck. You'd not be able to make sense of it. So for the game to be really fun, you need to keep your learning rate in this sweet spot where it's challenging but just to the right amount. It's challenging enough that you overcome the challenge and you learn in the process and you grow in the process. And establishing this sense of of progression, of growth is also very important. And it's not just in polishing your skill at playing the game. It's also that the game itself should you know directly convey a sense of progression like for instance a forance progression. As you play the game you should be getting more powerful in some way. You should you know have more things you can do more actions and the way you would do it is you know you introduce new game dynamics as the game goes on and you don't master them and then you use them and it opens up you know new possibilities in the later levels. So yeah, I think this is really like the the essence of of fun. It's it's to to maximize your learning rate and create this sense of of growth and progression. Another thing that's uh that's interesting is how challenging it is to make games like this that are still easy to play for humans given that we're in a setting where there are no instructions whatsoever. You're not told anything. In fact, there's also like no text for instance, no text labels in the games and so on. You cannot leverage any acquired knowledge to play them. They're completely new. You can only bring to the games like core knowledge prior. And it's very easy to end up with games that are too difficult. Not too difficult because the game mechanics are too complex, but just because they're not learnable enough. And we had to put a lot of work into, you know, really deliberately crafting the early levels of a game to make it as learnable as possible. like only introducing one concept at a time. Uh Making sure the the controls are discoverable. One issue we had uh early on is u players would often you know fail to complete the game not because they could not you know understand because they could not discover the controls right they would try to the arrow keys for instance would not do anything and because it was a game where you were supposed to click on things they would never try to click on things. So we we had to iterate a lot on on the control UI and we ended up with something that's a lot more discoverable. We uh in a lot of my user testing I think there's an interesting tension here between easy for humans hard for AI and this concept of fun. Some of the most fun that I've heard from folks and myself is playing games where it's not immediately obvious until you explore a little bit and then there's an aha moment and that aha moment actually significantly increases your fun. Um so this suits the swap between not being too obvious on the level one but still within sort of easy discovery. Uh I think in the first couple levels maybe why do we care about arc being fun you know this this was this was a very unique design goal I think you had in mind for v1 and v2 as well you know most benchmarks don't especially a benchmarks don't optimize for fun why is this an important thing that arc has this like in the acceptance criteria I think in a on a very basic way the benchmark will be more successful if it's fun if it's catchy if people actually enjoy interacting with it enjoy playing the games it will catch on more people will be be attracted to it more people will want to work on it. The human story like inspiring. Exactly. It it needs to be it needs to be catchy as a cultural artifact, not just as a as a scientific artifact. And also I think that in order to make progress towards the GI we need to do a lot of thinking about how humans play these games and uh in order to get this data like the human testing data for instance or just your own human introspection perception like as you play the games you ask how you're figuring things out. What's your strategy? to try to leverage meta cognition to produce uh um AGI insights. In order to to do that, the games need to be fun. They should not be boring. If they're boring, you're not going to want to play them. And human testers, they're they're not going to give it their best. All right. So, last question. Looking towards the future. So, even beyond V3, we've already started having conversations about V4, V5, whatever, even at lunch today. Uh what what are your maybe like hopes and dreams for what's not captured on this slide? What are the other interesting things that you can envision that we might want to test for capabilities wise in future AI systems? Right. So, I think V3 has uh basically the right ingredients like on-the-fly learning uh interactive learning, goal acquisition, but at a very small scale. So for instance, this sort of interactive learning is not really what you would call continual learning because even though there is a curriculum in each game, it's at the scale of 5 minutes of play. And as humans, you know, we we're doing continual learning over decades. And so I think an obvious thing would just be to take the same ingredients and scale them up. Have much more complex environments instead of just having, you know, 150 uh study games have endless novelty like have the have the game environment actually be more like a a living thing, an intelligent thing with maybe other agents that you compete with, that you collaborate with. And we should try to measure the development of an agent not over 5 minutes but over years of game time. hopefully not real time, rightMoved to summary

• Rahul Vohra, Founder & CEO of Superhuman: AI, Product & Investing Insights

  Rahul Vohra's background and influence on his approach to building software like a game.

• Behind The Scenes Of MICROVERSE: Interview with Designer SAM LONDON

  The overarching theme of Sam London's work and discussion in the interview.