GitHub - SamBroner/magic-frame: A "magic frame" that displays generated art from audio prompts to an e-ink display

Magic Frame

A "magic frame" that displays generated art from audio prompts to an e-ink display.

See it in action on Twitter!

Features

• Images Generated from Dall-E 2
• Realtime Transcription from AssemblyAI
• Wakeword Detection from Picovoice Porcupine

Getting Started

1. Clone Repo
2. Create accounts for AssemblyAI, OpenAI, and Picovoice.
3. Create .env from .env.template with keys
4. Install
5. Start

pip install .
python start

To Do

• clean up globals in start.py
• Better error handling, especially on file reads (e.g. what if no image gets downloaded from openai, then the dir doesn't have an image)
• Consider saving text/image layout rather than recomputing
• Consider allowing for more than 4 lines of text for very long strings
• Consider putting prompt on display before submitting to OpenAI (immediately after getting prompt back from AssemblyAI)
• Move more constants into the .env and .config files
• Fix the latency issues in start.py, specifically the 1second delay in the random image display
• Refactor the text rendering in display.py/render so it's in it's own function
• Fix the text rendering to change text sizes if text is too long
• Consider making Jarvis slightly easier to trigger

Dependencies

Hardware

• 1200x825, 9.7inch E-Ink display HAT for Raspberry Pi - $193.79
• Raspberry Pi Model 4B - 4GB Ram - $55.00
• USB Lavalier Microphone - $24.43
• Frame - $12
• Random - ~$15

Total: $300.22

Libraries

• Greg D Meyer's Python wrapper around the IT8951 e-paper controller
  • Seems to have limitations around partial update for GC16 (full image) renders
  • This is somewhat blocking my ability to do a partial update with text before rendering the image
• Documentation for Display
• AssemblyAI
• Picovoice Porcupine
• PIL
• Dall-E 2

Issues

NUMPY (OpenAI dependency)

Importing the numpy C-extensions failed. This error can happen for many reasons, often due to issues with your setup or how NumPy was installed.

Ended up uninstalling numpy and installying python3-numpy

pip uninstall numpy  # remove previously installed version
apt install python3-numpy