Appendix A: Applications for Open Book Touch
Perhaps the most enduring lesson learned from shipping Sensor Watch was this: the code that I write to bring a device to life is necessary, but the code that enables the community to write their own code is essential in a deeper way. Sensor Watch launched with timekeeping, temperature sensing and calculation of sunset times and moon phases. Useful! But in writing a well-documented, easily extensible framework (which I called Movement), more than a hundred members of the community have now contributed code of their own, and Sensor Watch Pro can roll dice, show TOTP login codes, draw tarot cards and play Wordle (and of course still tell the time).
When planning to launch Open Book Touch, I knew I needed to provide really excellent software for reading books. That much was a requirement, the sine qua non of shipping an e-book reader. But I also knew that I wanted to deliver a really excellent framework for developing other kinds of software for Open Book. That’s where Focus came in, and I already wrote a bit about it a couple of weeks ago.
Alongside, I also wanted the ability to load applications onto Open Book without having you completely reflash it, removing all of its booklike qualities. What I wanted, essentially, was something like what I made for Sensor Watch, where you can mix and match new kinds of functionality, but keep the object in its base state: a watch that tells the time, or a book that you can read. My solution to this was adding an additional item to the Open Book’s "drawer" menu: Applications.
Applications are self-contained binaries that Open Book Touch can load at runtime, jump into, and jump back out from when they exit. This was a wild rabbit hole of a feature to implement: lots of pointer math and linker-fu and crazed exported symbol names like extern char _ZNKSt8__detail20_Prime_rehash_policy14_M_need_rehashEjjj. (I wish that was an exaggeration.)
Anyway, in the end, I was able to get to a point where you could launch an application from the menu, relocate it, jump into it and get "Hello World" to print on the console. And once I get you to Hello World, I’m sure you can figure out the rest, right?
I’m totally joking, of course. If you want to write an application for Open Book Touch, you get the full Focus framework and full access to the display and all the hardware, along with some helpful glue code that gives you touch tracking and display output. Once I got the application loader working, the first thing I did was write a simple Sudoku application. I especially love to demo this one, because it shows off just how much you get for free when you develop an application with Focus. That whole number pad interface at the bottom of the screen? The Sudoku numbers are just Focus TextFields with a numeric type; the appropriate keyboard automagically appears whenever a TextField comes into focus.
As much as I’d love to show you the internals of how the Sudoku app works, the second example I wrote feels more of a piece with the project. Since applications get full access to the hardware, they also get access to the wifi radio. So I made an application that fetches a random poem from poetrydb.org.
It’s not much, but its simplicity makes it the perfect candidate for explaining at a basic level how making an application for the Open Book works. Which, for those of y’all that want to get technical, I’ll get to at the end of the post. But for now, a couple more app ideas.
Appendix P(DF) and N(PR)
One feature request I get a lot is for PDF support. I’m going to be honest: PDF parsing feels like it’s at the very edge of the ESP32-S3’s capability. But we gave it the old college try, and what do you know:
That’s MuPDF, a GPL-licensed PDF parser, decoding a PDF file — the sheet music of the variation that plays behind our campaign video — and displaying it on the Open Book, all from an application built using the exact same framework you just saw demonstrated via the poetry application.
To set expectations: as of now, this is nothing more than a tech demo: PDF parsing is slower than I’d like, and I haven’t implemented zooming, so A4 sized PDFs look super tiny on the Open Book’s 4.26 inch screen. I also can’t build this PDF reader into the Open Book firmware due to license incompatibility, and the app is a bit large as it stands. Still, I think it’s a fantastic flag to plant in the ground as a demonstration of the possible.
Another use case that folks have asked about involves connectivity: OPDS, RSS feeds, even OpenStreetMap. Open Book Touch has wifi, so it’s totally possible to implement all kinds of interesting applications for the device. RSS feeds in particular are something that I can see people being super interested in reading on Open Book.
So I threw together a news reader that pulls in headlines and stories from NPR:
This is a lot like the poetry app from earlier, except it integrates multiple view controllers: a HeadlinesViewController that presents a paginated list of articles, and an ArticleViewController that displays article text. Again: simple, but a demonstration of the possible. And to give you an idea of what it feels like to develop an application with Focus, this next section is going to delve into some of the internals; feel free to skip it and head to the bottom to learn more about the burgeoning Focus ecosystem.
Anatomy of a Focus Application
A Focus application gets a main.cpp, just like the one that the main Open Book firmware has. In the case of the poetry app it looks something like this:
extern "C" int main(int argc, char *argv[]) { auto display = std::make_shared<SSD1608Display>(EPD_BUSY, EPD_RESET, EPD_DC, EPD_CS, epd_spi_handle, epd_framebuffer, epd_staging_buffer, epd_plane1_buffer); display->setRotation(270); auto window = std::make_shared<Window>(display, MakeSize(480, 800)); window->setBackgroundColor(GrayscaleColor::White()); window->setTouchEnabled(); View::SetDefaultBackgroundColor(GrayscaleColor::White()); View::SetDefaultForegroundColor(GrayscaleColor::Black()); Font::addFontSearchPath("/system/fonts/"); Font::setSystemFont(Font::withName("spleen-12x24")); Font::setSystemLargeFont(Font::withName("spleen-16x32")); auto app = std::make_shared<PoetryApp>(window, display); app->run(); return 0; }
To walk through this: first, we create an interface to the e-paper display, which is the same thing that the main firmware does, using the same handles and buffers that the main firmware uses. Then we set the display’s rotation (your app can work in portrait, landscape or even upside down) and create a window matching that geometry. Then we do some setting up of our default colors and fonts, and we create an application, PoetryApp. It gets references to our window (where views are laid out) and our display (where the window gets rendered).
The app->run() line is where the application runs in its own run loop — exactly like the main firmware’s run loop — until the loop ends, and control passes back to the main Libros firmware. That run loop gets set up by PoetryApp:
PoetryApp::PoetryApp(const std::shared_ptr<Window>& window, std::shared_ptr<EPaperDisplay> display) : Application(window), display(display) {} void PoetryApp::setup() { this->addTask(std::make_shared<TouchTrackingTask>()); this->addTask(std::make_shared<RefreshTask>(this->display)); this->addTask(std::make_shared<YieldTask>()); auto viewController = std::make_shared<PoetryViewController>(shared_from_this()); this->setRootViewController(viewController); }
That’s it, in its entirety: the application adds some run loop tasks for touch tracking and screen refresh (all provided by the Open Book’s hardware abstraction layer), and then creates a PoetryViewController. Any touch that comes in gets passed to the PoetryViewController as an event, and when PoetryViewController changes what’s in the window, the display gets refreshed.
PoetryViewController is a little more complex, but at a high level it’s fairly straightforward:
void PoetryViewController::createView() { auto stack = std::make_shared<VStack>(MakeRect(0, 0, 480, 800)); stack->setMargins(24); stack->setSpacing(16); this->view = stack; auto titleLabel = std::make_shared<LabelView>(MakeRect(0, 0, 0, 48), "PoetryDB"); titleLabel->setFont(Font::systemLargeFont()); stack->addSubview(titleLabel); if (auto poem = this->poetryClient.fetchRandomPoem()) { stack->addSubview(std::make_shared<LabelView>(MakeRect(0, 0, 0, 72), poem->title)); stack->addSubview(std::make_shared<LabelView>(MakeRect(0, 0, 0, 32), poem->author)); // A rect height of 0 gives this view a flexible height stack->addSubview(std::make_shared<LabelView>(MakeRect(0, 0, 0, 0), poem->text)); } else { stack->addSubview(std::make_shared<LabelView>(MakeRect(0, 0, 0, 0), "Couldn't fetch a poem")); } auto exitButton = std::make_shared<Button>(MakeRect(0, 0, 0, 60), "Exit"); exitButton->setAction( std::bind(&PoetryViewController::exitApplication, this, std::placeholders::_1, std::placeholders::_2), FOCUS_EVENT_TOUCH_UP_INSIDE); stack->addSubview(exitButton); } void PoetryViewController::exitApplication(Event event, std::weak_ptr<View> sender) { if (auto app = this->application.lock()) app->quit(); }
The view controller has a createView method, which is responsible for laying out the view we want to put on the screen. In this case, we can use the VStack that Focus provides: the app title, poem title and author all have fixed heights, so they sit up top as a header; the poem text has a flexible height and occupies the rest of the vertical space, until we reach the "Exit" button, 60 pixels tall, at the bottom of the stack.
That button also gets a fancy C++ binding that attaches the exitApplication method to an event it might receive, FOCUS_EVENT_TOUCH_UP_INSIDE. When a user touches the button, PoetryViewController::exitApplication is called, and after some C++ smart pointer boilerplate, it simply calls app->quit(), which causes the app to immediately exit and cede the hardware back to the main Open Book firmware.
So yeah: while the poetry app is super small and simple, you can do a lot more with apps.
Let’s Go!
Even beyond Open Book Touch, Focus is making things possible already. My friend Evan has been developing Amigo, a pocket-sized device for playing the game Go. It’s got a different screen, a different input mode, and wildly different goals from a book — yet he’s using Focus as his entire UI framework too, and it’s working well for him.
As I said in the opener: the most important code I can write, as a maker of objects, is the code that enables the community to do more with the object than I ever intended. I’ve been blown away by the incredible things people have done with Sensor Watch, and I want to see the same explosion of possibility with Open Book Touch and devices like it. That all starts with designing a framework that lets your creativity run wild.
Which is all by way of saying: if you want to be the one to implement Wordle for Open Book Touch, you can pick one up today.