An Interview with Meta CTO Andrew Bosworth About Orion and Reality Labs

Good morning,

Today’s Daily Update Interview is with Meta Chief Technical Officer Andrew Bosworth. Bosworth was an early employee of Meta who created the News Feed, built out Facebook’s ad product, and, since 2017, has led what is now known as Meta Reality Labs; he became Meta’s CTO in 2022.

Over the last few years I have moved towards letting interviews stand on their own, without additional analysis or commentary, but it’s hard to escape the necessity of said commentary in the wake of yesterday’s Meta Connect 2024 keynote, and my opportunity to try Orion, Meta’s not-for-sale AR product. I will expand on my thoughts more next week, but for now:

First, I thought Meta CEO Mark Zuckerberg delivered one of the best tech keynotes in years. The keynote was live, chock-full of demos (includine a demo fail), and downright fun. I hearkened back to an old Steve Jobs keynote in Tuesday’s Article, and this presentation was in line with that spirit. Meta demonstrated some really cool new technology and did so in a way that captured the wonder that made so many of us fall in love with this industry.

Second, Orion, Meta’s AR glasses, is spectacular. I must start with the caveat that this is not a shipping product; the glasses that I tried felt like a consumer-ready product, but they reportedly cost $10,000 each, and Meta has decided to hold off on shipping a consumer version until they can bring the price down. That will be a tall order, and that challenge should be kept in mind with everything that follows.

What follows is unadulterated praise. Orion makes every other VR or AR device I have tried feel like a mistake — including the Apple Vision Pro. It is incredibly comfortable to wear, for one. What was the most striking to me, however, is that the obvious limitations — particularly low resolution — felt immaterial. The difference from the Quest or Vision Pro is that actually looking at reality is so dramatically different from even the best-in-class pass-through capabilities of the Vision Pro, that the holographic video quality doesn’t really matter. Even the highest quality presentation layer will pale in comparison to reality; this, counter-intuitively, gives a lot more freedom of movement in terms of what constitutes “good enough”. Orion’s image quality — thanks in part to its shockingly large 70 degree field of view — is good enough. It’s awesome, actually. In fact — and I don’t say this lightly — it is good enough that, for the first time ever, I felt like I could envision a world where I don’t carry a smartphone.

Orion is a standalone product, at least in terms of needing a phone; instead there is a “puck”, an oblong unit that holds the compute for the operating system and connectivity, and which connects to the glasses wirelessly. The glasses themselves contain the compute necessary for low-latency calculations that pertain to the actual display. One challenge I see in this model is input: voice works well, and the wristband that detects the electrical signals in your arm worked flawlessly for me — you can control your glasses with your hand without anyone knowing — but I wouldn’t mind if that “puck” contained a Blackberry-style keyboard for extended text entry.

Given all this, the big question over the new few years will be the race between Apple to build something this good, and Meta to figure out how to manufacture what is already great at scale and an accessible price point. It certainly feels like Meta is in the driver seat, but manufacturing is hard, and no one is better at it than Apple. Even framing the competition this way, though, feels like a big win for Meta, which also has a much larger developer ecosystem and user base — and that user base is set to expand with the introduction of the new lower-cost and lower-resolution Quest 3S.

In this interview, held in person at Meta Connect, we get into all of these questions, along with discussion about Bosworth’s history at Meta, which has not been without controversy. One thing I get from his answers — and perhaps this explains why Meta showed Orion — is the sheer relief and pride in providing evidence that all of the billions spent on Reality Labs might have actually been worth it. Having used Orion, I feel it has been.

As a reminder, all Stratechery content, including interviews, is available as a podcast; click the link at the top of this email to add Stratechery to your podcast player.

On to the Interview:

An Interview with Meta CTO Andrew Bosworth About Orion and Reality Labs

This interview is lightly edited for clarity.

Background

Andrew Bosworth, welcome to Stratechery.

Andrew Bosworth: It’s good to be here.

We obviously have a lot of new and interesting stuff to talk about in terms of your present-day job. But this is your first appearance on Stratechery, I’ve gotten lots of feedback, but this is the first official appearance. We need to touch on the usual questions about your background, how you ended up where you are. Take me back to the beginning. How’d you start in technology, where’d you grow up? That’s where I want to go.

AB: Listen, a longtime listener, first-time caller.

Many-time messenger.

AB: Many, many times WhatsApp chatterer. That’s right, you know me. Grateful to be here, thank you.

I grew up in the Bay, and technology was always a part of it. It’s funny, I grew up on a horse range in the south part of the Bay Area, and contrary to popular belief farmers are entrepreneurs. Farmers are like, “Hey, you got a horse business? Cool, we have too much manure, now we have a manure business”. Now you’re trying to sell manure to people who need that. It’s entrepreneurial in nature, you’re constantly trying to spin little side businesses up and down.

Being in Silicon Valley was such a fun place, we got to visit when I was in high school, we got to go on tours of Silicon Graphics and HP, all these kind of cool places. But really video games were, of all the things, video games were the thing that convinced me I wanted to study Computer Science and in particular, Metal Gear Solid convinced me I wanted to study AI. That was the first game that had AI that was kind of halfway smart. By today’s standards, we would consider it very dumb, but at the time it was staggering, MGS3.

So I went to undergraduate at Harvard, where I studied Computer Science with a specialty in Computational Neurobiology, I have a certificate in that, and of all the things I studied there, of course, much important than any of that was my senior year, I was a teaching fellow for a course, and I was randomly assigned a student to be in my Introduction to Artificial Intelligence section, Mark Zuckerberg.

I knew you had taught Mark Zuckerberg, but I could never get the timeline right. I didn’t think you were a professor, so thank you for the clarification.

AB: No, I wasn’t a professor. I was a senior, he was a sophomore. It’s very common in both Math and Computer Science for undergraduates to teach a section. So the professor does a lecture two or three days a week and is augmented two or three days a week by an undergraduate who teaches a section which covers the same material in greater depth, can do more QA, they also grade your homework and they do the exams and that kind of stuff. So I had taught my sophomore and junior year, our Introduction to Computer Science class, CS50 and then my senior year, I was the head teaching fellow for Computer Science 182, Introduction to Artificial Intelligence, under Professor David Parkes.

Mark was randomly assigned to be a student in my section. And in turn, by the way, quite a few of the people who taught me were early Facebook engineers, we had a whole lineage of these kind of teacher-student relationships.

Right. They’re supposed to go top-down though, it sounds like this went bottom-up.

AB: We went bottom-up, actually. We did actually go bottom-up, and Mark remains the top.

Right, and you’re now the CTO, so I guess it all worked out.

AB: That’s right. So I went to Microsoft for about 15 months, briefly, I worked on a wonderful piece of software called Visio. The ShapeSheet still does not get the credit it deserves.

I was a fan!

AB: You are I feel like somebody who could really appreciate the ShapeSheet, that was a phenomenal innovation, and I really enjoyed my time at Microsoft, and I had a great boss and setup there.

But one day I got an AOL instant message from a recruiter at Facebook who was like, “Hey, you should come down”, and I figured, hey, this is a free trip to visit my family, because I was living in the Pacific Northwest and I’m from the Bay. So I went down and instead was completely unexpectedly blown away by the vision that Mark had and his team, I’d met the team, Jeff Rothschild in particular, I just found so impressive.

Building News Feed

That’s interesting. So it wasn’t like you were best friends at Harvard, or whatever, and came over. You just knew each other.

AB: That’s right. I was Facebooker 1,681 to his number 4. I suppose technically the first three were test accounts, so you can minus three to all of them so we knew each other, but we weren’t close.

But what had happened, he told me later, was the recruiter is like, “Well, what do you need?”, he’s like, “Well, I want to build this thing”, which later become News Feed and he’s like, “I need somebody who knows AI”, and he’s like, “Well, do you know anybody?”, he said, “Well, I know one guy”. That’s why I came down, and sure enough, that’s what he did.

[Meta Chief Product Officer] Chris Cox and I sat next to each other, he shared that funny picture today, which is why I’m bald, and pictures like that, and he built the front-end and I built the back-end and Ruchi Sanghvi was the product manager, and we built News Feed together and that was what I did from — I started January 9th, 2006, which is just a couple of days after my 24th birthday.

News Feed, the arguably definitive Facebook invention.

AB: Yeah, thank you.

But also in my mind, the definitive culture setter for Facebook. I always think you have to go back to the early parts of a company, they go through a crisis, they figure something out, and that sort of guides them going forward. In this case, users revolted, you had actual protesters outside your office.

AB: We did, yeah.

And yet the numbers told the story, people absolutely loved it. You pushed through and that wasn’t going to be the first time you got a lot of protests and yet pushed through. Were you already on the, “No, this is right, we’re pushing through”, side or was that burnishing for you as well?

AB: No, I knew we were pushing through. If you use the product as much as we did at the time, because keep in mind it was a college piece of software for college kids, and I was a college kid. I was just like, I was the core demographic of this thing, you just knew this was the right answer.

We screwed up the rollout, we knew that, and the analogy I always draw, it’s like, you’re at a party and it’s loud, and you’re just talking to somebody and then the music cuts out, but you didn’t know it was going to cut out, so the last thing you say just hangs in the air. We basically did that to all 10 million people using the product, or 12 million people using the product, so that’s a strong reaction. We kind of screwed that up, they didn’t know it was coming, but then we saw the usage doubled immediately and it never went back down.

We knew we had a hit product, and we just had to massage in, “Okay, what’s the feedback and how do we integrate that going forward?” — and you’re right though, that did become, for us, an iconic way that we dealt with things was fix forward. “Okay, it’s broken, we’re not trying to deny the challenges, how do we incorporate that feedback in a forward-looking way?”, and that was our relationship with our user base for a long time. It’s a lot more stable now, we do a much better job rolling features out now, I would say.

Speaking of speaking loudly when the music drops, what was your job at the time you wrote the Ugly Memo? Which by the way, to be super clear, I defended it at the time and I defend it today. I actually went back and read it, I think it has a level of honesty about trade-offs that’s sorely missing in our political and tech discussion, but we don’t have to get too much into it. But what was your job at the time?

AB: So the irony of that, of course, is I had nothing to do with any of the stuff I was talking about in that note, I was running the ads business at the time.

Right, you weren’t in growth.

AB: I wasn’t in growth and I wasn’t even in those discussions really, because at the time I was running ads and it was a great and important job I enjoyed, but I wasn’t really connected to all these other side conversations the way I would be today.

I wrote that because there was a big internal discussion happening, and I wrote it as a clean bit of rhetoric, and so I think I’ve always appreciated your take on it, which is like, “Hey, if you boil it down to, like, ‘This is just rhetoric’, this is like, ‘What’s the purest expression of a thought?’”, that was what it was.

It was also correct. The whole thing with the Internet is, the critics of tech want to latch onto everything that’s negative, the people in tech want to latch onto everything that’s positive, and really it’s both, as you said in there. If you connect everyone, you’re going to get good outcomes and you’re also going to get some bad outcomes. To me, it’s incredibly mature to admit that that’s the case. And it does bother me that people who refuse to admit trade-offs and just latch onto one or the other, it drives me up the wall, so I was always a defender of it.

AB: No, and I do truly appreciate that. All these obviously, my regret is that I hadn’t brought more nuance to the discussion.

Right. But it was an internal memo, right?

AB: Yeah. One of the things that has happened to me over time is the context collapses around them. I think people internally at the time understood the context of the memo implicitly so I could be direct. I could just be like, “Hey, here’s a really almost oversimplified version of things” — well, of course, once it gets exposed externally, it’s like people don’t have any of that context.

Well, not just that, but people had it in for you at the time.

AB: They also had it in for me, and some of them still do.

Well, for you and Facebook generally.

AB: For the broader “You”.

Well, you mentioned it was around this timeline — were you at Reality Labs at that point?

AB: No, I was still in the ads business.

That’s right, okay.

Reality Labs

So you mentioned in an interview with Matthew Ball last summer, which was really, really good — I don’t want to cover too much of the same ground, so I’m going to link to it, people should go read it — that it was a good thing when you went to Reality Labs that you didn’t have a connection to it, because you could sort of look at things with a fresh pair of eyes. Can you expound on that a little bit?

AB: Mark’s said this to me several times in my career. He brings me a new thing — when I went to ads, he was like, “Hey, you should work on ads”, and I was like, “No”. I said no to him twice. Then he was like, “No, I really think you should do this”, and I came into it, and not only did I find that I was able to do it and that I had some delight in doing it, but yeah, the fact that I was an outsider to that, and even I dare say a critic of it, allowed me to take a really fresh perspective on the opportunity that we had, which was massive, of course. And ads, it’s not the hardest thing to do, to monetize, but we had to do it.

Hey, Facebook has been great for me all along. I started just right after the IPO and everyone’s dumping out and the stocks way down. I’m like, “This is the most insane monetization opportunity in the world. How can you not see this?”.

AB: I’m glad not to have let you down.

Then the same thing happened, it was actually kind of a funny story. My daughter had just been born, my second child, and Mark came over. I think, by the way, just an aside, since we’re here, one of the things people don’t respect about Mark is what a wonderful kind of person he is, a great partner. He’s the first non-family member to visit both of my kids after they were born. Which is crazy, this guy’s the CEO of a major company, and so he came to visit us after my daughter had been born, and was holding my daughter actually and he was like, “Hey, I think you should change jobs”, and I was like, “I think you should get my daughter back, I don’t like the leverage you have in this conversation, I feel like we got to level the field a little bit”.

But what was crazy to me, he’s offering me this job, what would become Reality Labs, and I had been by far the most vocal critic of this work internally and I was like, “Mark, you’re nuts, I’m telling you, I don’t think we should be doing this at all”, but his challenge to me was, “Okay, spend some time and think about what it would have to look like for you to think it was a good idea”.

And why at the time did you think that they shouldn’t be doing it?

AB: It felt actually, you’ve written about this a bunch before, you really like to have clarity of if you’re at the application layer, there’s a wonderful cleanliness that comes with embracing being at the application layer, and you talked about that when we walked away from the mobile phone idea as being, we finally just embraced, “Okay, we’re going to be an app, but we’re going to do the best app, and we’re going to be everywhere”. It was so clarifying and gratifying, and here we are once again.

Diving back into the platform.

AB: Diving back into the platform side of things, and it felt to me like a potential misadventure, we don’t have the expertise for this.

So we went away and Mark — I said “No”, again, for all the good it did me. Then Mark said to me, he’s like, “All right, write up what it would have to look like”, so I wrote this document up. It was very virtual reality focused at the time, AR was still very much just a research program, I believed actually more strongly in AR from the beginning, it’s worth noting.

Which by the way, I think for anyone who’s skeptical of this space, that’s sort of the default position.

AB: That’s right. But VR was the bigger investment at the time, it certainly had more money going towards it, and so I wrote this whole thing about how it can’t just be this entertainment device, it has to be a device that is able to permeate more deeply into the fabric of what we do and you get to a place where you’re like, “Wow, okay, if we had Codec Avatars that were super realistic, and you have a much evolved Workrooms-type product where people can be co-present, and then you have a truly global workforce unburdened by the need to go physically into the office to get the same types of gains that people are obviously seeing when people are co-located together physically in an office”.

Suddenly that environment is actually the more attractive environment because not only can you do all the things that you can do physically together, you now have this unbounded digital landscape, and so we really, once I started to get myself hyped about the much bigger picture — because again, people forget that the Rift era, it was a gaming peripheral.

Yeah, it would attach to a PC.

AB: And the device itself cost twice of what the Quest 3S cost, but you also needed the $2,000 PC to pair with it, so the total cost of ownership is just so high, and I think you can see with the direction we’ve taken, standalone, hand-tracking and mixed reality, making it much more natural, much more intuitive, and we’re continuing to attack things like form factor and resolution. I just think we really have actually been on the path that I imagined all those now seven, eight years ago.

One of the divisions you talked about with Matthew was between folks who wanted to build the best device and others, I think John Carmack has been very vocal about this, and you mentioned that in the interview, who wanted to build cheap devices. Two years ago I was here and I tried the Quest Pro, today I’m here and I just tried the 3S, which is fine, but it’s also $299. Is this John Carmack’s final victory?

AB: Listen, I like to give John as many W’s as I can. His love language is DMing me on Twitter with various critiques, which I adore, and John, please keep them coming.

I think this is, it’s a little bit of both. So John was a big champion of, for example, the Oculus Go, and the Oculus Go was a 3DoF [Three Degrees of Freedom] headset. So it didn’t have six degrees of freedom, so your head’s locked in a box, and it didn’t have a great control scheme, and it did okay, it didn’t do great. I always liked that bet though, even though it didn’t end up being a big deal, because if you could have made 3DoF work, boy, now we would be talking about $100 headsets, $50 headsets. We couldn’t do it.

So I think you want to be as absolutely good as you can past the bar of the minimum bar of “Okay, humans are willing to engage meaningfully in this thing”, and I think we found that with Quest 2. I think Quest 2, we found the feature set.

And the 3S feels like a Quest 2, it’s lower resolution. But as soon as I put it on, it was interesting. I immediately felt like I was using the Quest 2, and then I’m like, “Oh, I’m in mixed reality”.

AB: That’s right.

But it’s interesting because I almost didn’t notice the mixed reality that much because when I put it on I had like six screens on.

AB: That’s right.

But yeah, it felt like a Quest 2.

AB: And John, he was a bit of a skeptic on hand tracking. He’s like, “The controllers are good enough”, and he was a little skeptical on mixed reality. He was like, “That’s an extra thing, you don’t need that”, and I actually do appreciate where he’s coming from because he ripped those things out. Maybe it’s a little cheaper, it’s a little lighter, it’s a little more responsive, and those are important things.

At the same time, for me, I’m always thinking about the marginal person that we want to get into this headset and the truth is, we know from our research, from our use that things like mixed reality and hand tracking make you much more comfortable in the headset. They feel much more at home, they feel it’s more natural, they have a more implicit sense.

Remember when you put a Quest 2 on somebody who’d never used it before, having to teach them how to use the controller to do a thing was it was like a crazy hard problem to solve. Now people are just like, “Oh yeah, just grab that little thing and you move it”.

So I do think we underestimate sometimes as technologists the degree to which making these things accessible to people, not just in terms of price and in terms of comfort, but also in terms of how they’re easy to use and understand is a really important thing.

So was the Quest Pro a mistake?

AB: So it’s funny, Mark and I have this debate all the time. Quest Pro, you probably can’t get to Quest 3 without Quest Pro. Quest Pro was the device that allowed us to pioneer the pancake lenses, it allowed us to pioneer eye tracking and face tracking, which we’ll use in the future, it pioneered all these pieces, and there’s this kind of saying in a hardware space, which is like the Gen Three is what you wish your Gen One was going to be, but there’s no way to short circuit it, there’s no way to jump ahead. You actually have to launch the Gen One to get to the Gen Three. There’s no shortcuts, and I really feel that way.

Quest Pro was, we were able to justify to ourselves, “Okay, we’re going to put this massive expense in developing these completely new lenses that are much more expensive to develop, and build, and build the new factories for those”, and when we did that, so even though the Quest Pro maybe didn’t do the unit volume we wanted, it did fine. We sold all of them, which is good, but it didn’t blow up the roof out. It did end up being the critical device for us getting to the 3S. So Mark really thinks of it as a very successful program, obviously I would’ve liked to have moved more units.

Yeah, maybe we can go ask Susan [Li] though, your CFO, if she thinks it was a successful program.

Competing With Apple

Was there any aspect of the Vision Pro going so high end that also made you re-anchor on the low end, or do you think you would’ve ended up here regardless?

AB: No, I think we would’ve ended up here regardless. I mean, listen, I love that the Vision Pro — people won’t believe me — I love that they went maximalist. Just like, “What if we just take this dial and turn it to 11, and let the rest of the system fall where it does?”, and you see why we haven’t done that, just in terms of weight and cost. Like yep, that’s what it takes to take this dial and turn it to 11.

And this is why I think you and Mark right away seemed almost relieved by the Vision Pro.

AB: Your only real fear when a competitor launches a product is that they’ve had a breakthrough that you haven’t had. That there’s something that they’ve figured out, some technical thing that you haven’t figured out, because then they have a sustaining advantage potentially for some period of time until you can beat them on it. So I think whenever a device comes out, it’s like, “Oh good, this is all made with materials that we are aware of, this is all made with technologies that we have access to”.

“We understand why it costs this much, why it weighs this much.”

AB: We could have built this, we chose not to build this. It is both great for the world that there’s people exploring different quadrants of the space. By the way, if the Vision Pro had sold really well, of course we’d be changing our strategy. We’d be like, “Oh, okay, cool, there’s actually a bigger market than we’ve realized up there, let’s go do it”. And I think, by the way, I do actually think there will be a market there, when there’s software.

Are you surprised at how little content Apple’s released with Vision Pro?

AB: It’s how do you get the content? Before you have the devices and it’s a chicken-and-egg problem where it’s like, “Hey, okay cool, you have these devices out there, but there’s not enough for me to build my content for”.

Is part of the low cost a bet that if the egg is the end market, that’s the most important part?

AB: One hundred percent. We’ve talked about this all the time, you almost always hear me talk about the Quest ecosystem. I’m not talking about the Quest line of devices, I’m talking about building as big an audience as I can for developers to target so they can sell their software, so there’s more developers, that brings more consumers, and you have flywheel that way. Then at some point, that’s how you power your way up market. That’s how you power your way to, “Hey, we can now sell higher margin, higher end devices because there’s plenty of stuff”.

Well, to that point, Mark talks about, “In every market there’s the integrated version and the modular mass market one”, but if you go back to the PCs, Microsoft swept the market. Now one thing that’s important about that era that’s different from the smartphone era is in the smartphone era, Apple was first. In the PC era, DOS was first, so Microsoft was actually first, so they actually had developers first. At this point, seeing the Vision Pro, seeing what’s happened over the last six to nine months, are you shifting from a, “Yeah, we can both be winners here”, to, “We’re going to win the whole thing”?

AB: Man, I feel good about our position, if that’s what you’re asking.

I want to pretend I’m turning off the mic and getting your honest thought.

AB: With me, you’ll always get an honest thought, I have to make sure I’m phrasing this in a clever way.

The only reason I’m being careful here is I think — I don’t really want to be antagonistic with anybody, including Apple, I think it’s great that they’re investing, I want them to continue invest. Actually the Vision Pro has caused a surge of interest in investing in the entire space, including in us. I’ve gotten calls in the last couple of months especially that I would not have gotten, had Apple not launched the Vision Pro, and if they weren’t courting people to consider that there’s going to be a follow-on version. So it’s really, really healthy to have that competition. Good for consumers, good for us.

I also think that right now, if you’re a developer, you’d be an idiot not to build for us first, we have an audience that can actually go buy your software. It’s big enough to sustain you, and then yeah, no problem, bring it over to Apple Vision Pro after that.

Is your bigger concern losing to Apple, or that a market never materializes for these devices?

AB: Oh, good question. Yeah, my biggest concern is that the market gets capped somehow, like it doesn’t take off. The thing I worry about with Apple specifically is that they have their phones and devices so locked down that they can self-preference a ton. So they can easily, you look what our Orion glasses, these full AR glasses, incredible. We’ve got custom silicon in the glasses, we’ve got custom silicon in the puck, but Apple could build all that and just be like, “Oh, it only works with us,” which they’ve already done with the AirPods.

They don’t need a puck because they have a phone.

AB: They already have a phone, and they did this with Airpods.

Or the Apple Watch.

AB: Apple Watch. Those aren’t the best possible things you could build, but no one else is allowed to build those things, so it’s like, “Oh cool”, so if I have a concern about Apple, it’s not the competitiveness or non-competitiveness of their headsets, it’s that they’re going to bundle into their ecosystem in a way that really makes it hard for us to compete.

Orion

Well, let’s talk about Orion. I have tried it, and look, I have to try to be oppositional here. This is an interview, but it’s difficult because it is spectacular. I was blown away.

AB: Yes.

Even the shortcomings. What was striking is there are clear shortcomings in terms of lower resolution. I had a focus issue, which I think was actually due to my contacts, I rarely wear contacts anymore, these were super old, but none of it mattered. And I feel that’s a really critical point where you as a user are aware of the flaws and you just don’t care, because the context of reality being crystal clear. Another thing that was really striking is the total comfort of it compared to anything else. Wearing Orion, it didn’t feel any different than wearing glasses. You can use it anywhere. I want to buy it, but I can’t. So why show it? To me, this should be an astronomical wake-up call to Apple. Are you worried about delivering that wake-up call?

AB: No, and I appreciate this so much. I think the reason we want to show it is twofold, okay? The first one is we’ve been investing for 10 years in this thing and we’ve been asking a lot of people, technologists, investors, the public to come with us in this journey, trust us in this journey and the last three or four years, obviously we’ve been under intense scrutiny for the amount of investment we’ve been making in this space.

Do you regret the name change?

AB: No, not at all, I love the name change. So I think for us, the fact that we’ve got these Meta glasses and they work and they’re spectacular and they have this response, I think it’s a real proof point for people to be like, “Hey, this is the real deal”, I think that puts the call-out to technologists.

This is the first device I’ve ever used that — I know you guys have been saying it — that genuinely feels post-phone.

AB: It could do it, right?

It could absolutely do it.

AB: It could do it, that’s exactly how I feel. It’s the first time I put anything on. Dude, your reaction was so funny, the way you just described it.

Whenever I do the demos, I’m always very stoic in there.

AB: No, you did great. You were keeping us on ten toes. I left the room, Mark’s like, “How does he like it?” I was like, “It’s hard to tell man”. No, the first time I tried it with software, so I tried the displays a year ago and was blown away.

The field of view’s unbelievable.

AB: It’s crazy.

Because I’ve used the HoloLens before and that’s my main AR experience. But if you try to get the edge of the field of view, you can. The other thing that’s so striking is I felt the field of view in the Apple Vision Pro a lot more than the Quest, even though I think they’re kind of the same.

AB: Apple Vision Pro is smaller.

But in this case, because you are seeing the real world, the field of view edges are immaterial and they’re sufficiently large to do everything you want to do.

AB: The first time I tried it with the software was four or five months ago, and very much to your experience, the team kept apologizing like, “Oh, I know this pixel’s not right yet, and we’re going to do the color correction over here” — meanwhile, I’m just having my mind blown and I don’t care about any of it and I literally stopped at the end, I stopped. I was like, “You guys, stop, stop apologizing. This thing is un-fucking-believable, you’ve done it, congratulations.”

That demo was so funny too. Here’s a little fun anecdote. They kept going into the other room and I couldn’t figure out what they were doing — they said were getting sodas. In fact, they hadn’t done the thermal work yet, so they kept putting cold sodas on the puck to keep it from overheating. Now they’ve got a full two-hour battery, now the battery runs down before the thermals hit, so it’s two to three hours, we’ve had people run it for three hours.

Anyways, so for us, the first reason we’re showing it is as a proof point for people, it’s like, “Hey, invest with us, believe in us, if you’re a technologist, come work with us, this is the thing”. The second thing is for developers, we do want to ignite their enthusiasm and say, “Hey, if you’re investing in our Avatars platform”, which we talked about today on the stage, “If you’re investing in our social platform, you’re going to get that dividend not just in the Quest ecosystem, but eventually also in the AR ecosystem”.

By the way, someone called me while I was wearing it to show someone calling in, and it tricked me, I didn’t realize it wasn’t an avatar. Now, again, I just said it was low resolution, I was having focus problems. But that said, I actually for a couple seconds thought it was the real person.

AB: You’re not the only person who’s been fooled by the Codec Avatars.

Yeah, the hair work is particularly good, I don’t know if you picked the guy with wavy hair to sort of really show it off, it was very effective.

Anyhow, you mentioned before the worry with Apple Vision Pro is Apple figured out something technologically, and that could be patented or it could be protected or whatever it might be. Do you have that level of assurance that helped you show this now?

AB: Yeah, that’s a good point. We do have both these things. So some of these pieces, like the MicroLEDs that we have in this, we’re super proud of them. We didn’t just design them ourselves, we manufactured them ourselves, we have partners, but we were the ones who built the systems, so we have some places where we do think we have technological leads and advantages.

Everything in Orion is not contiguous with what we’re going to build next, we already have the next several prototypes of full AR glasses, the next ones we hope to be consumer-ready in development, and they have some really cool changes. They’re lighter, they’re thinner, they’re dramatically cheaper, they have some trade-offs. Field of view is one of them that we’re trying to balance against things like brightness and cost and these kinds of things, and weight, and so for us, we feel like we have a good number of places where we have technological advantages.

Our approach here has been a hybrid. For a few technologies, we keep them completely first-party, we’re the only ones who can do it. For quite a few technologies, we work with industry partners because they have applications beyond just AR, and those partners are free to commercialize that technology in any other segment of the market except for AR, except for the specific people who would compete with these glasses and that’s been a huge boon to getting a much bigger industry-wide investment going.

How long is this going to take though? Is it next year, two years, five years, ten years?

AB: It’s years, not decades. We will be playing with this one and getting our intuition honed in the software for the next year or two probably, and then I think the focus will be on getting geared up for a consumer launch of a version of these.

2027?

AB: I’m not going to put it to paper here, no, so to speak. But yeah, we’re definitely looking in the next three to five years.

Developers and Focus

You opened up your developer keynote by apologizing to developers.

AB: That’s right, a bit of fan service there. If you’re on the Reddit community or if you’re on Threads, every day, me and Mark Rabkin are getting DMs and @s from people who have very legitimate grievances about the challenges they’ve had developing for our platform.

These are people who we want to develop for our platform, they’re in good faith, they want to reach their audience. They have interesting ideas, and some things have been really challenging, and that’s been a major focus for us this last year. It was hard to get ourselves excited about doing things like, “Let’s clean up the developer documentation”, “Let’s make sure that we have a good end-to-end story for Unity, for Unreal”, for all these things when we know the platform is about to change again, we’re about to add mixed reality, we’re about to add hand tracking and those change all the primitives.

So I started with this apology, I just wanted the audience to hear that I’ve been listening, I read it all, I care about it, I listen. Our big focus now is getting to a place where we’re a great platform to develop on, a delightful platform to develop on, and there’s a lot of work to do there, and that we’re going to have stability that they can build those APIs and not have to constantly worry about having those kind of deprecated and have to replace them.

I’ll be totally honest, after using Orion, I’m excusing you all your billions of dollars a year spent, that’s how incredible it is, but I do think one of the critiques, and you talked about it when you went in, this was an entity that had two completely different camps that want to go in totally different directions. Then even a few years after you were there, you’re having an operating system bake-off for years instead of months and then it’s, “Should we do processors? Should we partner with someone doing processors?”. What is the forcing function that is getting you into, “Okay, we’re going to stop experimenting and actually start building”? What got you to that point? Was it the Year of Efficiency? Was there a bit where, “Look, we have to lay off half the team, so we’ve got to decide which half”?

AB: I love this question. It was before the Year of Efficiency hit. I think it’s not uncommon, you have these expansionary periods where you’re like, “We don’t know what matters yet, we truly don’t know what technology is the right technology, we don’t know what operating system is the right operating system, we don’t know what trade-offs matter yet”. So if you want to be successful with high confidence in a certain timeframe, it pays to parallel path a ton of stuff.

But how long do you parallel path it?

AB: We honestly turned the corner with Quest 2, especially when we had mixed reality in sight. That started the process, and now you’ve got to a point with a mixed reality with our metaverse division where it’s extremely focused, have a very clear vision of what good looks like, have a very clear ability to discern this is the path, this is not the path. As a consequence, you can be really, really much more efficient with your resourcing, your parallel pathing list, you’re just blitzing the things that matter more.

With augmented reality, Orion, a year ago, we actually hit this point where we’re like, “Okay, we believe in this, we see it, we have a really clear sense of where we’re going with this”, and you know what really helped a lot with that was the Ray-Ban Meta glasses as well. Cool, it’s not just that we have this distant AR thing, we actually have an entire family of devices coming before that that also matter.

Did AI save Reality Labs?

AB: Oh my gosh. So AI, because FAIR, the Fundamental AI Research group reported to me until this year. We just moved it over to join the rest of the AI stuff with Chris, and I don’t know if it saved us, but it’s a wonderful tailwind, it’s the first tailwind I can remember having. For us, it’s mostly just headwind after headwind after headwinds like, “Oh, guess what? This thermal performance is worse than you thought, this battery life is worse than you thought, the efficiency is worse than you thought”, and so we finally got a tailwind. We finally got a thing that showed up before it was expected, which was AI.

So I think to answer your first question, each of these devices has gone through an expansionary period and a contractionary period where it expands until you feel like you have a good understanding and intuition of what good looks like, and then you can start to prune and then you can get really good about pruning. Today our architecture is really tight, hand tracking, eye tracking, face tracking, Codec Avatars, these are shared technologies, they work in both VR and AR, and we have a single shared team building those technologies. Separately, the operating system for AR has to be its own operating system because it turns out the use cases, what you actually do, the interaction paradigm, completely different.

The AR and VR Bifurcation

So to me, this bifurcation is becoming much more clear.

AB: That’s right.

I wrote yesterday — this idea that just came to me — but I think there’s something where it does feel like Facebook crossed the Rubicon as it were, or Meta I should say, when content broke out of your social network, and of course TikTok forced you down this direction, but now you’re pulling content out from anywhere. You’re already individualizing the feed, your baby, and what if we pulled it from everywhere? It could be anything, once you’re pulling from everywhere, you’re no longer constrained by your network.

AB: That’s right.

What if you’re pulling generative stuff? What if you’re actually literally generating content for people? What if you’re pulling ads that are generated or personalized directly to you, you see yourself with the shirt on instead of just seeing the shirt? And this is just barreling towards what the feed was, the ultimate manifestation of the feed.

AB: That’s right.

Completely personalized, customized content. This feels like when I saw DALL-E for the first time, to me, I wrote at the time, this is the savior of VR because it fixes all the cost problems, the asset creation problems. But then AR, the whole thing is its real world and I feel like your true social products these days are group chats.

AB: Sure, yeah.

And this middle area where Facebook started where it’s kind of social/kind of public has sort of disappeared and you have this bifurcation and VR/AR sort of mirrors that bifurcation.

AB: Yeah, that’s interesting, I hadn’t thought about it from the same lens that you have there of content. I completely agree with you. The place that the AI saves us the most is Horizon Worlds, where I want everyone to be able to create a world, but 3D design is just not trivial.

Yeah, games have hit a wall.

AB: As cheap as you can make it, it’s just not trivial to do until I can just describe it in language and have it appear and today, we talked about that. We talked about NPCs on stage — NPCs are kind of a fussy thing to do, and it’s hard to do your game if you don’t have it. Now, you can do it with AIs. Meanwhile, on AR, the side of AI we’re using is the sensing side.

That’s right.

AB: So I’ll tell you another demo that we’ve been playing with internally, which is taking the Orion style glasses, actually super sensing glasses, even if they have no display, but they have always-on sensors. People are aware of it, and you go through your day and what’s cool is you can query your day. So you can say, “Hey, today in our design meeting, which color did we pick for the couch?”, and it tells you, and it’s like, “Hey, I was leaving work, I saw a poster on the wall”, it’s like, “Oh yeah, there’s a family barbecue happening this weekend at 4 PM”, your day becomes queryable.

And then it’s not a big leap, we haven’t done this yet, but we can, to make it agentic, where it’s like, “Hey, I see you’re driving home, don’t forget to swing by the store, you said you’re going to pick up creamer”. And it’s like, oh man, there’s all these things that start to change. So you’ve got this VR and MR as this output space for AI, and then you’ve got this kind of AR space where it’s like it’s in the AIs on the inputs.

Right.

AB: The AI is on the sensing and they’re both phenomenal tailwinds for our work, and we’re thrilled about it.

The Ray-Ban Partnership

You mentioned the Ray-Ban thing. One thing that strikes me about it, just being here at Facebook today and there’s been press reports or whatever about, “Oh, no one in Facebook is using Horizon Worlds”, you don’t have to comment on that. What strikes me today, I don’t know if it’s because it’s Meta Connect Day, but is how many people I see walking around campus wearing Ray-Ban glasses.

AB: Yeah, for sure.

And it feels like you go to Apple, everyone’s wearing an Apple Watch, everyone’s carrying an Apple phone, and one of the critiques you might have of other companies is the extent to which, “Do you like your products? Are you using them?”. How important have the Ray-Ban glasses been just in terms of, “This is a product I like to wear and use, and it looks good”?

AB: It’s such a huge part of it. I think if there’s a way that we’ve tried to differentiate ourselves from other AR people in the space, we were not willing to let Orion be unwearable.

Is it true that EssilorLuxottica came to you instead of the other way around?

AB: Yeah. [EssilorLuxottica Chief Wearables Officer] Rocco [Basilico] emailed us, cold emailed Mark Zuckerberg years ago and was like, “We should do this”, and actually at the time, me and Hugo Barra were working together, and Hugo was like, “I think he’s right”. I think Mark had sent it to his team and like, “Hey, someone follow up with with Rocco”, and Hugo was like, “No, we need to do this”.

“No, you follow up with Rocco.”

AB: And Mark flew out to Italy and created a strong relationship with Leonardo Del Vecchio, the late now Chairman of Luxottica, Founder and Chairman.

So this was the real shift in Mark’s style, Rocco saying looks actually matter, appearances actually matter.

AB: To his credit, Mark has always known that and believed that. It was clear from the moment I took over what we then called AR/VR, now Reality Labs, he’s like, “If they don’t look good, people will not wear them, it doesn’t matter what they do”, and the size is the challenge of this product. Making this thing twice the size is more than twice as easy, it’s four times easier, and he wouldn’t let us do it. Even with this one, 98 grams and the next one, we already know how to make it thinner, we already know how to make it smaller, we already know how to make it lighter, we’re rolling.

Just got to make it cost a thousand bucks.

AB: And we have a path to it. Again, not without trade-offs, and real trade-offs, but we have a path to this stuff now and until you build this, you never get there.

Open and Integrated

You talk about, I mentioned before, the open/modular bit. It feels, actually to go back to AI, you are very integrated. Just the integration is your hardware and the AI, that’s the key thing.

AB: That’s right.

So is, “We’re open”, just a buzzword?

AB: Well, for us, we think the opportunity to create an ecosystem around it is the real missing piece. So we’ve benefited so much from this in the past, and you know this because you’ve been covering this a long time, whether it be Open Compute, where it’s like, “Hey, you want to commoditize your complements”. And for us, AI makes our products better.

What are the complements for Reality Labs?

AB: AI makes our product better, nobody else can provide your News Feed from Facebook, only we get to do that. So AI anywhere, it doesn’t matter who developed the AI, makes our products better. The same is really true for us in headset with Horizon Worlds. It’s like cool, AI makes our products better, it can make other people’s products better too.

The core product is AI.

AB: I think it’s both. So in AR, I really agree with you. I think in mixed reality and in virtual reality, the AI is the unlocker.

Is Meta a content company or a social network company?

AB: We never like to call ourselves a social networking company, we’re a technology company, that’s how we’ve always tried to identify. People want to put us in these buckets, and it’s actually one of the reasons so many people misunderstand our work in Reality Labs in particular. It’s like we’ve always been a technology company.

So early on when we did HPHP, when we’re doing Hadoop kind of work, when we’re doing work on Cassandra, all this work that we did, open source, why are we open sourcing it? Because our goal is to build a community around that tool that can do more great work than we can do on our own.

So what are the pieces that make Reality Lab’s products open?

AB: Sorry, I thought we were still talking about Llama basically. On Horizon OS, yeah, the big open shift that from a developer standpoint is we used to have a very curated store, and they really hated it, they really wanted to have an open app store that anybody could just ship any APK into and the consumers decide and we’ve made that change over the last year. So yeah, I know the word, “Openness”, gets so bogged down in our industry, and everyone wants to stake out. And there’s always a Richard Stallman who’s like way out in the —

I think Matt Mullenweg was criticizing your use of “open” this week.

AB: And I get it. It’s a huge spectrum, it’s all relative. It’s only relative openness and relative closeness and we want ours to be relatively open, is what I’d say.

Fair enough. Well, honestly, this sounds like kissing up, but congratulations.

AB: Thank you.

Like I said, no pun intended, with Orion, you can see the future in a tangible way for a product, you have to actually figure out how to manufacture it and ship it so can’t give you too much credit yet. But as I said on Twitter, they’re real and they’re spectacular.

AB: Honestly, I’m so gratified. You can’t imagine, I guess, obviously the intense pressure that we felt with the scrutiny of the last several years. And until a year ago probably, we didn’t know if we could build any of them still, it was still a bit of a risk. Then to come up a few months ago and play the software, it was genuinely an emotional moment for me. Not just to see it and experience it as a technologist, as someone who cares with these products, but just having the whole history of the 10 years and all the thousands of people who worked in this thing.

Turns out $75 billion gets you something after all.

AB: (laughing) That’s not just on Orion, man. We have a lot of great investments that we’ve made that we think are going to pay off.

Thank you very much.

AB: Thank you, Ben. Always a pleasure.

This Daily Update Interview is also available as a podcast. To receive it in your podcast player, visit Stratechery.

The Daily Update is intended for a single recipient, but occasional forwarding is totally fine! If you would like to order multiple subscriptions for your team with a group discount (minimum 5), please contact me directly.

Thanks for being a supporter, and have a great day!