Introducing BIGKit

BIGKit was a tool born out of frustration. As a software engineer, I have spent more than my fair share of time writing data access code, managing GUI’s, validation logic, serialisation and marshalling code, etc. There were clear patterns which kept recurring, but yet somehow each time seemed to be manifesting themselves in unique ways, with just enough divergence that all previous progress was somehow not transferable.

My personal view that many of these issues are down to the available tools being defined at a too low level of abstraction. A lot of these common problems required access to complex libraries and frameworks, allowing enough control to be possible, and thus with that comes complexity.

Many of the existing higher level tools which bridge the somewhat poorly defined domain between the CMS/page builders, and real hand-rolled applications seem to compromise at some level. A large part of the problem here is we often start from the foundation of a page editor, and many of the more advanced features, although possible, are not considered first class citizens.

So, What is BIGKit?

Built on top of dotnet core, S.A.F.E stack and React, Bigkit attempts to bridge the gap between a publish-focused CMS and real application development. It was built from the ground up to include some core tenets an opinionated software engineer like myself might expect from a system builder:

• A visual component and page builder (nothing new here)
• Components and composition are first class concepts. The unit of composition (the node) should be simple, and do only 1 thing.
• Hierarchically scoped types everywhere
• type inference where possible
• Ability to write and run code anywhere without needing to deal with files, build pipelines, bundlers, marshalling types between boundaries, etc. The code should be typesafe, and allow typed access to api’s to prevent whole classes of frustrating runtime bugs. Code should be a first class citizen, and not some bolted on afterthought.
• Two way data-binding and change tracking
• Everything is reactive by default, even data access code
• Users and user permissions are a first class concept
• Functional principles: immutability, unidirectional flow, expressions everywhere, reactivity.
• To not have to compromise on scaleout potential for future read performance by having some kind of first class integrated reprojection engine. I should not need to get involved in Kafka, message busses, actors, or any other hardcore server based event engine, just to be able to keep a couple of read reprojections in sync.
• Following on from the point above, events out of the box, at least at the level of field update deltas to allow auditing and data time-travel. This could of-course also be used for reprojection.
• Generally mundane repeatable tasks should be vastly diminished, or if possible: nonexistent, allowing developers to focus their energy real system or domain problems rather than pushing pixels all day long.

The page builder

A page is a visual end user experience, so it makes sense to work on this problem visually. Not having to resort to code to craft your pages brings down the iteration cycle, and allows you to see immediately how any changes will affect the end result.

Any BIGKit application generally starts with pages. Each page comprises of a number of nodes arranged in a tree. There are many node types available, some are simply visual (such as a text or a grid container), and some allow complex retrieval of data. Each node has a number of properties that can be configured, which will change the behaviour of the node in question.

It is important to be able to quickly see how your application responds to edits, and unfortunately the WYSIWYG editor can only take you so far. It is after all a conceptual representation of the page, so there are runtime constructs that cannot be visualised.

In run mode, the application subscribes to changes from the editor, thus allowing you to push changes into the executing preview. This allows something equivalent to what React or Angular developers know as HMR (hot module replacement). This can work in both inline preview, or alternatively across tabs.

A number of pages can then quickly be strung together with navigation into a user journey. Because all pages are effectively data, we can easily start to understand how our pages actually fit together in the real system (rather than some out of date documentation that has to be separately managed).

Components

The same WYSIWYG editor is available with some modifications in component mode, a way to construct reusable building blocks such as a Page layout, or a menu bar.

Currently components have some restrictions, such as not being able to use scopes, but the intent is to lift this in time.

Components, like pages, can take parameters. The parameters can be primitives, or complex object graphs, and are available to bind to throughout the component.

Components can also contain one or more rendering placeholders, allowing you to define areas to render the children of the component in question.

Exploratory Testing

Although we do not yet have means of doing lower level testing, a lot of UI work generally benefits from a more exploratory approach. To facilitate this, and somewhat inspired by storybook we have the concept of Run Cases.

A run case allows you to define a list of cases, each allowing explicit configuration of input parameters to a component or page. It is then possible to click through, and quickly evaluate various scenarios to see how the page/component behaves. We currently do have limited support for user impersonation in run cases, but there is lots of potential to expand here. Some examples of this would be mocking out data, creating automatically verifiable expectations etc.

Types everywhere

The word “type” can mean many things to many people in the development world. I want to specifically point out that what we mean here is the “shape” of data. When you define any data construct, it is thus given a type based off the shape the system is expecting the data to be. As the type is consumed in expressions and threaded through to different parts of the application, this type information is also threaded through. This allows us to reason about what data we can expect at any point in the application. We can validate that — the id passed to an entity scope is actually an id and not a shopping list for example.

Types also allows us to better understand how data is used, thus opening the door to implicitly working out what fields to load on an entity based off what fields you bind to, among other exciting things.

Although there is still some argument as to the absolute value of types, it is generally accepted that types are able to prevent whole classes of bugs. There is also empirical evidence to back this up.

Part of the pain of using types has always been the extra ceremony that is required to achieve this. We at Leansquad use F# for all our core products, and the Hindley Milner type system is perhaps the leading example of how to do type inference right. BIGKit uses a more simplistic type inference algorithm, which only really supports outbound type inference, however this is more than adequate for our specific unidirectional flow situation.

It is also possible to bridge these types between real typescript expressions, and back. we will cover the specifics of this in a later article.

Deployment and DevOps

Although this is still somewhat in its infancy, we can currently deploy the whole application in something which approaches a one-click publish seen by other content publishing platforms. On top of this we are working on tooling such as page-level logs and analytics to help manage your deployed application.

There is much work still to be done here, but there are absolutely cost savings to be made not having to manage complex build and deployment pipelines.

Summary

Thanks for joining us in the first part of our whirlwind tour of the current state of BIGKit. Next time we will be delving deeper into some more advanced concepts such as inputs, handling persistent data, reactivity, code integration, and more. Join is in the coming weeks where we will go into more detail.

We welcome you to check out our samples, create a free account with us, and give us your feedback!

It is our opinion that fastest way to ensure software delivers on the promise is to dogfood. Our main site, documentation site, feature request site, and our client’s site Headroom are entirely built using BIGKit.