GitHub - dcrebbin/meta-glasses-api: "Hey Meta send a message to ChatGPT" Mai: A Hacky Messenger browser extension & pseudo API for the Meta Glasses :eyeglasses:

3 min read Original article ↗

Mai - Meta Glasses API for Messenger

Note: by using this project you agree to the terms of use outlined in TERMS_OF_USE.md

This is a browser extension to add custom AI bots to messenger that can be used with the Meta Rayban Smart Glasses or the standalone messenger app.

Features:

  1. Hey Meta send a photo to my food log: Video Demo

  2. Hey Meta send a message to ChatGPT

  3. Video Monitoring: Send screenshots of your video calls to your chosen provider (ChatGPT, Claude etc) and then log the output to the log viewer

Requirements:

a) Meta Rayban Smart Glasses (or the standalone messenger app)

b) OpenAI/Perplexity/Claude etc Api Key

c) Alternative Facebook/Messenger account

Browser Extension Setup

  1. bun install

  2. bun run dev:chrome (or brave, firefox)

  3. This should build and run the extension and automatically open it

  4. Add any api keys you want to use in the extension: the API settings tab has more information on how to get them

  5. Sigin into your alt Facebook account and head to facebook.com/messages/t and start monitoring the conversation

Hey Meta Send a Photo/Message to _____

Before we setup our extension we're going to trick the Meta Glasses into allowing us to send a message to (nearly) any name e.g: "Hey Meta send a message to ChatGPT".

Tricking Meta

  1. Create a messenger group chat with 2 other facebook accounts (the minimum amount allowed to create a group chat)

  1. Remove the account you're not going to use

  2. Change the name of the chat

  3. Update the group chat photo (for a legit feel)

  4. Set a nickname for your alt bot account

  5. Go to the Meta view app within the communications section

  6. Go to Messenger and disconnect then reconnect your messenger account

I believe this step resyncs all your latest chats and friends which then allows that Meta Glasses to become aware of your newly created group chat to allow for voice commands!

Chat Monitoring

  1. On your alt account head to messenger.com or facebook.com/messages then open your newly created group chat

  2. Start monitoring the chat

  3. With each new message/image request it will send it to your chosen provider (ChatGPT, Claude etc) and then respond to you with the output

  4. If enabled: It will then generate an audio clip of that output using OpenAI and send it back to you

Examples

All chats can be done via voice commands "Hey Meta send a message to ___" or by simply messaging the group chat.

OpenAI query with Minimax text to speech

Using Perplexity to answer a question accurately

Using GPT 4.1 to describe an image

by Devon Crebbin

Please reach out if there are any issues or feature requests :)

Hopefully the Meta Reality Labs team will provide an SDK in the future so these types of integrations can be ✨productionised✨

Credits:

Anime Sky from Vecteezy is used in the icon