Introducing MEAN-SEO: A Free, Self hosted, Express.js plugin for SPAs (but mostly for MEAN apps)
One of the biggest issues many devs who build SPAs often hit is the need to make sure their app is crawler-friendly. Since we started doing more and more complex things on the client, using frameworks like Angular.js, a large amount of the content we render on the page is rendered by our Javascript application. That causes an inherent flaw to many JS heavy frameworks.
As part of evolving the MEAN stack towards a production ready state, the MEAN-SEO module makes it pretty simple to make sure your MEAN application ready to serve all of your precious content to those hungry crawlers, without breaking a sweat.
Quick Start
First, install:
Then:
And you’ll be good to go.
Overview
MEAN-SEO is an Express.js middleware, and it intercepts requests that include an escaped fragment reference in the URL. We include a special meta tag that tells the crawler to query our Angular pages again, using the escaped fragment reference.
MEAN-SEO then launches a Phantom.js headless-browser, which then fetches the same page, but makes sure to wait for a while, before serving the fully rendered page back to the server.
This allows the crawler to get our content after our Angular app is fully loaded, and the search engine could then index the page content, even if it requires resolving AJAX requests etc.
Caching
The MEAN-SEO would save pages onto the disk by default for two days. You can choose the duration for the cache to be saved by passing a “cacheDuration” property, with a value in milliseconds.
In addition, the plugin caches the page for the duration you define, either by saving the pages either to the disk or to a Redis instance (requires installing Redis).
Setup
MEAN-SEO is super easy to set up. To install, run:
Next, we’ll create the middleware function, typically close to where we declare other middlewares, such as Passport etc, and wire it to our app:
Make sure you include the following meta tag in the <head> of any page you use Angular (or Ember or any other AJAX-dependant content):
<meta name=”fragment” content=”!”>
Digging in
If you take a quick look at what the module does, you’ll see how extremely simple it is. When crawlers from search engines like Google and Bing come around, they identify your SPA by hitting the fragment we just mentioned. When they do, they request the same page again, this time converting any url fragment (anything following the # or “hashbangs”), so that:
http://localhost:3000/#!/articles
to
http://localhost:3000/?_escaped_fragment_=/articles
So, every time a request comes in through your Express app, the middleware looks at the request to see if it includes an “_escaped_fragment_” parameter. If it does, that means a crawler came by, and MEAN-SEO kicks into action: We make sure the page requested isn’t already in the cache, and in case it is, we serve the crawler that cached result.
If a page was never crawled before — or if the cache was invalidated — a Phantom.js client simply fetches the page, ensuring it is entirely loaded before serving the results back to the crawler.
Why are we doing this?
Simply put, we are trying to make sure MEAN.JS is as production ready as it can be. We saw the challenge of search engine optimization to be one of the most common and yet complicated to solve. By adding MEAN-SEO to the MEAN.JS stack, we make it easier for you to enjoy the benefits of working with awesome SPA frameworks like Angular.js, without worrying about any external limitations.
What’s next?
MEAN-SEO will add some other needed SEO-related features, like building site maps, making sure your meta tags are up to snuff, and others. We hope you join us by contributing code, and adding to the feature list on our Trello board.
Until next time!
The MEAN.js team