AI in a Box

Your very own private AI that you can ask questions and get answers, all in a tiny box! The first AI that you can talk to, and that talks back, running locally with no internet connection so your conversations and data are completely secure. No account, setup, or subscription are needed, just plug in the box and start chatting.

Live Captions

Need closed captions for a live event, or just to help in situations where you have trouble hearing a conversation? We’re using the latest in AI technology to display subtitles based on the audio input, which are output on the built-in display and through an HDMI connector for external monitors or screens.

Conversational AI

Want answers to questions, jokes, stories, or just want to chat to pass the time? AI in a Box lets you talk to a large language model (LLM) running locally on the device, and it will talk back. Get the power of modern AI on your coffee table, all with no setup or API fees.

Live Translation

Translate between multiple major languages on the fly. Select the source and destination languages and you’ll see split-screen captions showing the original input and the results of machine translation.

Air-Gapped Privacy

The software itself runs entirely locally, so no information is ever shared outside of the device. With no Wi-Fi or Bluetooth connectivity, the only way to access the device is by plugging in a physical keyboard or an Ethernet cable. This removes the possibility of remote attacks, and makes it suitable for use with sensitive conversations or material.

Voice Keyboard

Attach the included USB Type-A cable to any device that can take keyboard input, and the content of any nearby conversations will be entered as keystrokes, as if they were being dictated. This allows the box to become part of a larger system, for example by connecting to a Raspberry Pi that takes in the transcript and controls a robot or sculpture based on the content.

Open Source

AI in a Box is built on a foundation of open source models like Whisper and Llama2. In the same vein, we are releasing all the code to accelerate and control the system under an open source license. You can already see our library for optimized transformer inference on the RockChip NPU. This transparency should help with security and privacy auditing, and will allow developers to use the system as a base to build their own real-time voice input applications in Python.

Features & Specifications

• RockChip 3588S SoC with NPU
• 8 GB DRAM
• Built-in display, speakers, and microphone
• Custom enclosure
• Ubuntu 22.04 OS

Support & Documentation

• GitHub documentation

Manufacturing Plan

We will be working with a contract manufacturing partner to produce the complete board, casing, and other accessories. We have working prototypes using off-the-shelf components that we’re using to test our design and software.

Renders of the all-in-one design that combines the microphone, speaker, display, and smarts

Fulfillment & Logistics

After production, AI in a Box will be shipped to Crowd Supply’s fulfillment partner, Mouser Electronics, which will deliver to backers worldwide. Additional information about Crowd Supply’s fulfillment service can be found in the Ordering, Paying, and Shipping section of their guide.

Risks & Challenges

• Manufacturing. There's a risk of unexpected delays caused by supply chain or manufacturing issues. We have experience producing two previous hardware products with a similar workflow, so we'll use that background to try to catch and mitigate these problems early. We've also tried to stick to commodity parts that are available from multiple suppliers to reduce the risk of supply chain holdups.
• Accuracy. The finished product may not produce results that are accurate or fast enough for all potential use cases. This risk is particularly high with language translations where there are a large number of possible combinations of languages to test. We are trying to get our prototypes into the hands of as many external testers as possible so that they can give us feedback on its strengths and weaknesses, and provide outside guidance on the applications it's most suited for.
• Lies. All current AI language models are very good at sounding confident while giving totally wrong answers. This means that you can't entirely trust any output from the device when you ask it factual questions, so it shouldn't be your only source of information. This only affects the LLM part of AI in a Box, the speech to text transcription and translation may make mistakes but they won't be factual errors.