Time Travel Queries in TerminusDB 2.0

Our new release of has both a commit history graph (which can be WOQL queried just like anything else) and a path regular expression operator and a bunch of usability improvements: this snippet shows a bunch of them in action together:

WOQL.using("_commits")
.triple("v:Branch", "ref:ref_commit", "v:Head")
.path("v:Head", "ref:commit_parent+", "v:Tail", "v:Path")

Will return the commit chain of every branch in the database in the Path variable

it reads as: start from branch head, follow commit_parent one or more times (until you get to tail), and store the results in path

add constraints to get a single author’s chain of commits:

.triple("v:Head", "ref:commit_author", "joe")

or to get all commits in a time frame:

.triple("v:Tail", "ref:commit_timestamp", "v:CommitTime").less("v:CommitTime", mytime)

And any id of commit ids that are found in the commit graph can be used to set the client.ref(COMMIT_ID) function - this will mean that all subsequent queries with the client will be to the immutable state of the database as it was at the time that the transaction in question was completed. All queries and API calls should work the same, it’s just that all results are relative to the state of the database at the time at which the transaction was made.(edited)

That’s how time-travel works in a nutshell.
(from @kevin)

2 Likes

This is awesome!

Just to confirm - does this mean that the internal representation of the data is a sort of log of triples, and any insert/update/delete event just adds triples to that log? In other words, information is never deleted from internal storage, just appended?

1 Like

You got it - all state is append only and you can replay the state to any point you like You can also query the database in the state that it was in after each and every update just as if you were querying the latest state.

Here’s a paper that describes the low level mechanics: https://terminusdb.com/t/papers/terminusdb-git.pdf

2 Likes

Thanks @kevin, this paper is great! I’m guessing a merge functionality is planned for the future?

I think the immutable layers are a great design, and will enable many powerful applications of Terminus, including event sourcing like @aschrijver mentioned in the other thread.

However, I do use Git’s history-rewriting features (rebase and amend) all the time. I think people will miss them.

I think the solution is to have two layers of Git-like functionality:

  1. The existing low-level functionality with immutable layers that supports committing, branching, and merging.
  2. A high-level optional plugin that represents the history of a graph as a graph within Terminus itself. Since graphs within Terminus are mutable, it’d be possible to do anything Git can do and more, including rebasing and amending. It would also be possible to query and reason about the history using WOQL and Prolog, which is what’s particularly appealing to me.

I’m willing to try writing the high-level version control plugin in Prolog if you think it’s a good idea. Ideally, I’d write the new versioning commands as an extension to WOQL to allow the greatest flexibility of use. Are there provisions for creating Terminus plugins that extend WOQL? Would such a plugin communicate with the core server application directly by manipulating the triple store, or through JSON-LD messages (or their Prolog equivalents)?

1 Like

The latest master actually has an API for basic rebase functionality but it hasn’t been exposed in the front end yet. We’re going to be rolling out the features in the UI as they are tested and we produce usable interfaces.

We actually have versions of both 1 and 2. We will be writing technical descriptions of them over the next month and since there are so many possibilities here, we would be very happy to have contributions for the man slight variations that can occur. We have not yet written a 3-way merge for instance.

2 Likes