The Rise of Audio AR
dbreunig.comThe article kind of dismissed Microsoft Soundscape as a failure, but for blind people who want awareness of what is around I still don't think there has been anything that beats it. I work on one of it's successors, Soundscape Community [0]. It is based on the code Microsoft released as open source when they shut down the project.
TLDR: context is hard.
To get good context, you need video data. To get good video data, you need cameras, lots of them, on your head.
To get them on your head you need them to be small and light
To get them to run you need batteries. But the maximum battery size you can get, without wires is about 1-1.5 watt hour.
Then, without making a custom wireless protocol, the maximum bandwith you can reliabily expect to user (with an iphone) is 1megabit.
That means you need to compress the world around you to 1megatbit a second.
Now, with a bit or work, like using eye tracking to segment what you are actually looking at(usually a 64x64 pixel image at 15-20 frames a second), and occasional wide angle view when the scene changed, you can build really good context, even reading a book.
But, you also need good location data, along with room classfication to get context. You can't really use GPS because they aren't accurate enough, and don't work all that well indoors. So you use visual odometry.
Once you have all that, you then now need to teach the machine to understand that 1mbit stream to work out if it knows the answer you need. oh and that has to work in a device that has ~10-15 watthours. or offload to a bit boy machine over a patching network.
Aar is a welcome enhancement (that has its own, but potentially less privacy issues than video capture). There's a whole set of aar enabled glasses that focus more on the user via audio rather than reckless complete video capture like so many "smart" pervert glasses being shilled by large advertising companies.
Reminds me of MIT Media Labs 'Nomadic Radio' (1999): https://www.youtube.com/watch?v=l99vjTHWhk8
So much room for advertising. Meta sure loves this
anyone using any device/ app in this space they are happy about?
(2024)
> They deliver high-quality sound and give developers access to a range of controls and sensor data, from volume to head movement to biometrics. Built-in microphones and simple controls make it easy to trigger assistants and audio apps.
Fear the Greeks when they give you presents.
But i fear it is too late.