Yahoo.com PWA, Part 1: Introduction

Brief History

Yahoo recently embraced Progressive Web App (PWA) and launched a new standalone mobile web app called Yahoo Lite. PWA is an exciting technology that allows our users around the globe to benefit from the enhancements that they can provide.

We already launched in more than eight countries (expanding to a lot more). The performance improvements and user engagement growth of mobile web and Yahoo Lite has given us great confidence in PWA capabilities. However, building a PWA at Yahoo scale required us to do things differently.

We assume you have some familiarity with Progressive Web App (PWA), if not please read Google’s documentation about PWA. We wanted to share our learnings and experiences in three (Post 2 and Post 3) parts. Each part is broken into four sections: What, How, Challenges and Conclusion. We hope you find these posts informative, and we strongly believe PWAs help bridge the gap between native and web capabilities.

What

When we started our journey, we asked ourselves a few questions:

• Can we simultaneously build a PWA for both Android and iOS?
• Should we rebuild our tech stack and start from scratch?
• Do we need to use the app shell model?
• Who should handle our XHR call fetch strategies: service worker or client library?

How

We had already started working on a re-architecture for Yahoo.com mobile web. Our new client library was evolving, an investment that helped us reduce our JS size by 68%. We also managed to reduce our HTML and CSS sizes by 48% and 76% respectively, with good improvements to the following times: AFT (above the fold time), time to first-byte, start render, service response time and DOM interactive time. We knew we could take this further by adopting PWA capabilities. When we started working on our PWA, Safari had just launched support for service worker. However, after some experimentation, we realized Safari support was limited:

• Standalone mode is buggy. We were facing issues with persisting cookies between Safari web and standalone mode.
• All links open in a new window for standalone mode (there is a work around, but it’s not elegant).
• Safari does not install binary like Android Web APK, making add to home screen just another bookmark, limiting its capabilities and resulting in duplicate icons.
• Service worker on Safari is not fully matured yet.

Given Safari standalone mode and service worker limitations, we decided to start with Android

As part of our tech migration, we did not want to introduce any new radical changes. Numerous articles like this one on progressive web app also talk about React. While React is amazing, it unfortunately did not fit our homepage use case.

We decided to continue with our tech migration without making any significant changes. Progressive web app benefits are library and framework agnostic

Yahoo.com mobile web is a partial single page app. We have different tech stacks powering different pages, so moving to an app shell model would require a lot more changes and we wanted to serve Yahoo start_url with content from the server for SEO purposes. We have a good mix of page navigations and client navigations. Given limited benefits for our use case:

We decided to build our progressive web app and pages around it without app shell model

As part of our new client library, we decided not to leverage service worker for XHR calls, as we wanted to improve our experience across OS and browsers. We do not have a service worker on all of our properties and pages. If you have batched prefetch calls like we do, service worker caching might not work.

Adding fetch strategies on client gave us lot more flexibility and reach

As part of our client library, we built a very powerful fetch component. Fetch is what we use to make XHR calls and mount the new DOM using document fragment onto the page. We optimize this further by using GPU for most of our animations (we will post about it in the future).

To add to our fetch component capabilities, we also added “Network First” and “Cache First” fetch strategies.

Network first strategy

• Fetch will first attempt to retrieve data over the network.
• On successful network call, the Fetch component will respond with data and write latest data into cache in asynchronous mode (not blocking the request).
• If network fails, it will look into cache and serve from cache if available.
• Fetch will serve from cache, even if cache is stale.

Cache first strategy

• Fetch will first attempt to obtain data from cache with 1,000ms timeout.
• It will attempt to serve from cache if cache exists and has not expired.
• If cache does not exist, has expired, or cache timed out, Fetch will serve from network.
• On successful network call, will respond with data and write into cache in asynchronous mode (not blocking the request).
• If initial cache lookup and network failed, will look again into cache with higher timeout and serve from cache, if available. We use higher timeout for second lookup to factor in occasional IndexedDB slowness that we have observed as part of our measurements.
• Fetch will serve from cache, even if cache is stale.

Challenges

IndexedDB and our learnings

We are using IndexedDB for caching. IndexedDB is really awesome and gives a transaction-based async database in the browser. However, we still had our share of learnings while using it.

• Indexed DB operations are async, but read/write operations can be slow occasionally. Please find below some of our data from another project.

Time taken for initializing connection across browsers on Android:

< 100ms = 48.9%

> 100ms & < 500ms = 37.7%

> 500ms & < 1000ms = 7.1%

> 1000ms = 6.2%

Time taken for record retrieval from store across browsers on Android

< 100ms = 13.1%

> 100ms & < 500ms = 37.7%

> 500ms & < 1000ms = 64.5%

> 1000ms = 12.8%

Time taken for store record update across browsers on Android

< 100ms = 71%

> 100ms & < 500ms = 22.8%

> 500ms & < 1000ms =3.5%

> 1000ms = 2.7%

• Creating an index on the correct field while creating a store is essential.
• Use ‘onupgradeneeded’ carefully (refer to our code below).
• Don’t use DB for reading until store is created.
• Split your read / write transactions (refer to our code below).

Here is an example of how we handle our connection onsuccess, onerror and onupgradeneeded events. If you notice, we have multiple checks determining when we can perform operations. Ensuring DB and all stores are created is essential, as stores would not be created ever again until we bump the db version.

Another example of how careful we need to be with indexedDB transactions.

• Ensure DB and table are created before performing any operations.
• On transaction error, we highly recommend removing the table and recreating the DB if you don’t care about persistence of data.

Disk space limitations

Temporary storage is shared among all web apps running in the browser

Fetch uses IndexedDB which internally uses disk space. Remember, disk space is shared across websites and each website has limited data available. Read this if you plan to use IndexedDB and service worker caching. Keeping disk space in mind, we purge all data created before 24 hours from most of our stores (unless store needs to be persistent).

Conclusion

For building Progressive Web App, you don’t need to rewrite your tech stack; service worker (we will cover it in out next post) is the backbone of PWA. It’s important you plan ahead and split responsibilities between service worker and client. We started our approach with making our client more powerful and delegating the rest to service worker.

To try our Yahoo Lite experience, just visit https://www.yahoo.com/ on your android phone and add it to your Home-screen. Based on the chrome version, you would be prompted differently. For Firefox and Samsung browsers, add to home-screen is part of navigation bar.

This story is published in Noteworthy, where 10,000+ readers come every day to learn about the people & ideas shaping the products we love.

Follow our publication to see more product & design stories featured by the Journal team.