Introducing Temporal Resources

Your standard application generally only tracks what is true right now. Updates overwrite previous data, and asking questions about the past requires either introducing a mechanism to track the previous state of your application (versions), or reshaping your entire mental model of application development (event-driven/event-sourcing). Temporal resources are the answer.

Temporal resources are a new, experimental feature in Ash that make dealing with changes over time boring. You can read it as of any moment, write to it as of any moment (yes, including the future), without changing how you use or write your application (almost) at all. Filters, relationships, aggregates, calculations. All of it from one option, as_of.

I announced the new feature at Goatmire Elixir in a talk called Time Travel for Normies, and I've rebuilt it here for the web. These pixel examples are based on a live multi-player game that had the audience split into two teams, competing for territory while traveling and using powerups that travelled through time. You had to be there for that one.

How we keep history today

There's no shortage of ways to keep history around. Lets compare what I would consider the "main four":

A paper trail copies each version of a record into a versions table as it changes. AshPaperTrail does this for you.
An event-driven design records what happened as events, alongside your regular tables, typically driving other behavior in your application via events.
Event sourcing makes the events the source of truth, and works out the current state from them.
AshEvents sits somewhere in the middle: every action is recorded as an event you can replay, while your tables stay the source of truth.

They can all tell you what happened. Fewer make it easy to see what the world looked like at a given moment, none of them let you ask about the past with an ordinary query, and none of them let you make a change that only takes effect later.

✓ yes · ◐ sort of · ✕ no.

Four lines of code

To make a resource temporal, give it a temporal section:

pixel.ex


use Ash.Resource,
  domain: TemporalDemo.Canvas,
  data_layer: AshPostgres.DataLayer

temporal do
  strategy :context
  attribute :valid_at
end

attributes do
  uuid_primary_key :id
  attribute :x, :integer, allow_nil?: false
  attribute :y, :integer, allow_nil?: false
  attribute :color, Color, default: :white
end

That's it. Every row now has a period, valid_at: the span of time it's valid for, from one instant up to (but not including) another, or forever. Each record is now split up across multiple rows, each row showing what the value of each column was for a given span of time. The migration generator sets the table up so no two of them can overlap. You never have to set valid_at yourself.

To read or write it at any moment other than now, pass as_of. Leave it out and you're reading and writing now, so code that doesn't care about time doesn't change at all.

Lets take a tour

Here is a small canvas that three people, Alice, Bob and Carol, have been painting for the last hour. It's 09:45. Every paint is a dot on the timeline, and the white line shows the time that we are reading at (the `as_of`).

How it works

PostgreSQL 18 added the SQL:2011 building blocks for application-time periods: periods in primary keys and foreign keys. The standard's other half, FOR PORTION OF, is how a write changes only part of a row's period.

Plot twist: FOR PORTION OF was supposed to come in PostgreSQL 19, and that's what I built all of this on. Then we found that PostgreSQL had reverted it before release. Under READ COMMITTED. There were bugs w/ concurrent writes.

So now we've hand-rolled the following functionality. For the sake of the demo we're going to keep it using FOR PORTION OF since those are the semantics we've copied. We will use FOR PORTION OF when it releases.

Try it yourself

Reading is the easy part: as of a moment, it's whichever row's period contains that moment, valid_at @> '10:00'. Drag along the timeline to read the pixel as of any time. Then change its history, and watch the rows split.

Read

You won't write any of that SQL yourself. AshPostgres generates the period column, PRIMARY KEY (id, valid_at WITHOUT OVERLAPS), PERIOD foreign keys for relationships between temporal resources, and identities that are unique at every instant rather than across all of time. Writes split periods for you, and read actions filter on the period.

AshPaperTrail: temporal inline mode

A temporal resource already keeps every version of every record, so a separate versions table would be an unnecessary copy. AshPaperTrail's new :temporal_inline mode records all of the same information you might track in paper_trail, but on each temporal row itself.

subscription.ex


temporal do
  strategy :context
  attribute :valid_at
end

paper_trail do
  mode :temporal_inline
  change_tracking_mode :previous_values
  store_action_name? true

  belongs_to_actor :user, MyApp.Accounts.User,
    domain: MyApp.Accounts
  metadata :reason_for_change, :string
end

Caveats

This is experimental, and there are some sharp edges to know about.

It needs PostgreSQL 18 or later. That's where periods in primary and foreign keys arrived.
Our mechanism of getting this behavior is not perfect. Its very very very unlikely to happen in practice, but the strategy that we took to simulate `FOR PORTION OF` writes requires locking records and rechecking that no transaction modified the ranges of those records after writing. If we do, we have to redo the original operation (just the data layer part). This means that we have a "retry loop" built into temporal queries for the sake of consistency. It is capped at 25 retries, and while simulating a whole bunch of writes against a single extremely hot record we had one write that took 3 attempts to complete. So, probably fine, but there is a new theoretical "deadlock-like" error that can be emitted by AshPostgres. This should go away when PostgreSQL actually releases FOR PORTION OF
Every read is a single point in time. There's no read across a range of times. Explaining why is complex but, put simply, there are weird mind bending and complex timey-wimey things you'd have to do to model this correctly. Things we might actually attempt at a later date.
No cascades on PERIOD foreign keys. Postgres forbids ON DELETE and ON UPDATE actions on them, so you have to cascade in your application instead.
The actor is taken as it is. You supply your actor, so anything in your policies that read from the actor (but not things that filter on the data), don't know about the past or future. It is on you to provide actor data appropriate for the time that a change is happening, or to write your policies to account for this. This is also something we may work on improving in later versions.
Changes, validations and preparations must be temporal safe. Anything that runs during a temporal action must explicitly indicate what changes and validations are safe to run with a time that is anything other than "now". Ash's built-in ones are all temporal safe.

Try it out

The temporal resources guide has more information. Hop in the Discord if you'd like to chat more.

Thanks to Mike Buhot, whose talk It's time! PostgreSQL 19 for temporal data gave me the idea for mine. And if you'd like the whole talk's worth of slides, they're all here.