Thoughts on Apple’s entry into XR. Consider this more like fan fiction than hard predictions - most of it is probably wrong. This is just me applying “WWSJD?”, given what I know about the technology and believe about the potential. XR is a societal phase shift. Widespread ubiquitous immersive computing is best thought of as programmable hearing and vision. Tablets did not displace laptops did not displace TVs, but this isn’t like that. If immersive computing becomes ubiquitous, those all evaporate. (Among other insane things.) As such, Apple entering XR signals that they believe the technology and society at large is ready to begin this transition. This is a very big deal. First, Apple has been working on this for a long time with a lot of people. I don’t know how they’ve managed to conceal what they’re doing so well. My guess is that only a few people on Earth have seen the final hardware. The best way to think about this is an entire alternative future computing paradigm has been developed in secret for a decade. People don’t realize Apple has had opinions about this before the metaverse was a twinkle in Zuck’s eye. They’ve been building it, and now, it’s ready. Second, Jobs was right: Google and Samsung stole the iPhone. Android was nothing like the iPhone, but then once it launched competitors backed out everything they had and copied it. The competitors who didn’t do this died. Apple hasn’t forgotten this. This is where we are today with Meta. Meta has run the ball down a very narrow assumptions about XR, as Android did. They’re in a deep, deep local minimum for this reason. Apple knows Meta will be forced to try to copy them to survive. But I think Apple is ready for it. (More on this later.) Not only that, but I think many people at Apple simply dislike Meta. So, I think there are really two things here: Apple is going to reveal how they think the final computing platform should be - the one that will one day run on your visual cortex. And that it can work now. Second, I think they are going to try to kill Meta. First, let’s talk about the new computing platform. Apple’s philosophy is that computers should fade to the background. Apple has probably realized (like I did, and Palmer did, a while ago) that passthrough AR, aka reprojection AR, is how AR should work. Not only is it maximally capable, but it can allow a full glass-like experience for users on both sides of a headset via high res displays. This was the missing piece needed to make the technology evaporate, mimicking glass. It will be strange! In that spirit, I think we won't see physical controllers, but it will use the physical world as the interface, perhaps also a stylus or bluetooth keyboard on occasion. Any hard surface will be an interface, any tool you can pick up can be used as a pointer, and so on. You might have to wear a ring or something to make it work. Existing objects will be physical affordances, not controllers you need to carry around, charge, etc. I also think Apple knows that the existing mobile VR paradigm is a disaster. Poor comfort, the ceremony to set it up, retention is horrible. So, I expect the headset will be designed to be worn powered off most of the day, charged at night, and will be stylish and comfortable to wear all day. You’ll turn it on it as needed, it will look cool. They’re going to have agonized over low weight (hence the battery pack) and how the device looks to others, two things Meta has completely, utterly failed to prioritize. Passthrough AR, unlike glass AR, also allows full perceptual override. This means you can remove objects, perform color shifts, recolor things, etc. The programming interface for this is “what light do you want on to the retina, what sound to the eardrum.” This is the right API for the final computing platform. The killer app for XR is the elimination of physical co-locality to be with others for most contexts. We'll need to move our bodies around a lot less to be with one another. This shift, finally, will start to happen. The rush away from remote work will reverse. Friendships will be easier to maintain. People will spend much more time feeling together, unlike on social media. Eventually, I expect people to carve out areas in their home which are synced across houses so you can hang out with anyone you want whose in that room in their house The problem though is that you need device penetration for any kind of multi-user applications like this. For wide appeal you need a variety of “single-player” experiences that will make the device useful on its own. One obvious thing will be iPad OS panels you can pin to any surface and pin it so it’s there whenever you come back. This gets you almost all screen-oriented use-cases. There are a lot of other possible apps + games from the VR space. But I think one that people are underestimating is simple perceptual enhancement and what I call “beauty injection.” Passthrough AR unlocks programmable vision. You can zoom, enhance, recolor, remove, simplify, recontour, restylize, “spectral hack”, and basically do any image-oriented processing on your visual perception. This will be accessibility++. A lot of people will be surprised to find out how much they like having enhanced visual perception, even if they don’t particularly have poor vision. Those that do have poor vision will love it. Beyond that, passthrough will allow you to remove ugliness, add beauty, tone down detail, get focused, inject art, draw on any surface, meditate anywhere, and more. For avatar communication, the potential is there for Apple to do head stitch and replace. If you are in XR, it's possible to replace the heads of other people with avatars. I don’t know if they’ll do it. Eventually you can stitch out + replace whole people. I could give lots more examples but leveraging passthrough AR itself will be a pillar of why you’d use this device even if nobody else has one. Now, here's another big thing. XR experiences, particularly immersive XR experiences, impact human health. People get sick, confused, disoriented, can fall, or run into things. Not very Apple. I think there will be a few mitigators: first, the HIG will focus on short, light XR experiences. Pop in, pop out. You’ll have games, but short sessions, light modifications of reality, etc will be preferred. But Apple knows that it needs developers to experiment to really make the ecosystem shine. This is a problem: many XR apps developers make will be a blast for some and make others very sick. Oculus et al have “YOLOed” this problem by just making a ‘comfort rating’ for apps. We’ve seen hacky side loading solutions. Kludges. Apple can’t review apps to score them, they have no experience at all to bootstrap an XR review process. A score feels un-Apple like: would they approve a “moderately” comfortable app? It's also subjective per person. So I think Apple figured this out a while ago: they need a way for devs to experiment that doesn’t require them be involved. There is a solution, just like there was when the iPhone was launched: the web. The web now has WebXR which is a way to build XR apps in the browser. It turns out, Apple has radically accelerated its investment into Safari over the last year or two, and (publicly) WebXR support has been going into WebKit. I think this is why. So I expect WebXR to be a big part of this. It actually wouldn’t surprise me if WebXR *is* the entire 3rd party development story. Maybe this whole time they’ve been working with Unity & Unreal on WebXR. What I expect is a very tightly curated set of Apple-approved 3rd party apps + games, that they feel *very* confident in wrt health and safety, and a clean path for WebXR apps to be discovered, installed, and run. Apple may then choose to review these and promote them in the more centralized centers of the store. These might be PWAs. If I were to really go out on a limb, Apple might even go all-in on the web. Perhaps every app on their XR platform is part of the web, and you have to register an origin token to associate it with your developer account. This lets you use Apple Pay and other services in your WebXR app. Who knows. The more you think Apple is going to pull back towards the Macintosh days, the more likely this seems. So that brings me back to Meta. The situation here is no controllers, affordances based on the physical world, WebXR build targets and web based deployment, Passthrough AR as a core mechanic, for short experiences, meant to be used on-the-go, often in public. This basically flips the table on the entire VR paradigm that all of Meta’s developers are currently building in. I think Apple is going to tell VR/AR developers: you can build an Apple XR app, *or* a Meta XR app. These will be very different things. There's no 'least common denominator.' This would mega-rug Zuck to a degree we've never seen in tech. First, Meta is going to find themselves on the wrong paradigm, and unable to copy Apple’s essential innovations because this time around Apple is going to have a million man army of lawyers ready to prevent Android 2.0. Second, their devs are all going to flee, because they'll have to pick sides, and none of them will pick the doomed paradigm Meta is offering. And of course, Meta now has sunk costs: do they re-do the Quest 3? Do they kill the existing platform? What the hell do they do? Zuck correctly saw the wave but the fact is he simply doesn't have the Apple X-factor. Ballmer liked to throw chairs in frustration at this, I expect similar kinds of things to happen behind closed doors as it slips away from him. So that’s about it. This is pure speculation, and probably mostly wrong, but I don’t think Apple pulled the trigger if it wasn’t serious about the fact they think the world is ready. XR will cannabalize.. well.. all computers, and I hope they’re right that now is the time. If it is, it’s going to be amazing.