Documentation feedback from a new user

Hey just some feedback on the docs from the perspective of a new user, and with full acknowledgement that the docs are very much WIP.

Most of the docs seem focused on teaching the user all about how to use it from the context of treating it like a shareable Git repository. Pushing, pulling, rebasing, cloning, and all of that kind of stuff.

None of that stuff is of much use to me as a developer until I am feeling confident in how I integrate it into my own systems. There’s also a lot of documentation about how to query and consume graph data once it’s in the system, but less exploration of getting the data in there from the perspective of a typical client application.

The way the documentation emphasises all of the Git-like functionality was leaving me feeling like maybe the database is currently more targeted at people who want to build, share and query data sets than those who want solid integrations into high-performance client applications. I had to spend some time letting things turn over in my head and challenging that reaction I found myself having.

What I, as a developer new to TerminusDB, would have really benefited from, would be stuff like:

  • How can I quickly get a Schema in place without getting bogged down in the intricacies of WOQL and so forth? Can I just start by referring to a fake Schema/prefix and backfill the details later? Sometimes it’s nice to just start fleshing something out and then fill in the details as things take shape.
  • How do I deal with blobs of data? I’ve realised that I should probably be storing blocks of content like text documents, scripts and so forth externally to TerminusDB and then linking to it from my database. I had to figure this out for myself though - it very much wasn’t clear from any of the documentation.
  • Further to the above, let’s say I’m dumping text documents into a local database (LevelDB in my case) so that I can link them together using linked data. Do I need to create a prefix to something local? Do I have to worry about TerminusDB trying to resolve it on my behalf?

Under tutorials, you have:

  • My First Cloned Database
  • My First Shared Database
  • Pushing and Pulling Changes
  • Creating and Merging Branches
  • Taking TerminusDB to the Bank
  • Importing and Cleaning Data From CSVs

Most of those don’t really help me get started using TerminusDB as a database for my application. The second-last item seems useful, but it primarily talks in terms of creating and deleting triples (whereas my application uses records, arrays, etc.), then hurries me along to sharing, branching and all of that. I would have liked to see it talk more from the perspective of “Hey you’re a developer with a bunch of records you’d like to store and link together. Here’s how to do that without having to mentally break it all into triples in advance, or at least here’s how you can do it with a minimum of ceremony”. I honestly had no interest whatsoever at this stage in sharing and publishing data so others can use it. All of that stuff can wait until much later.

The “Reference” section is obviously more comprehensive, but it’s really not clear where basic stuff is, like, “how do I put some JSON data into the database?”. The “JS Client” section has something about “update_triples”, but I’m not yet working with triples directly in memory in my application. I don’t see anything there about adding triples. I found some functions to that effect in the “Python Client” section (assuming that they probably mirror the JS client to some extent?). I then realised the the QL in WOQL is as in SQL, in that it’s not just about performing queries, it’s also about making changes. So then I found myself hunting around and feeling a bit deflated, seeing as all I wanted to do was get some records into the database and to then start building on that.

Even if what I want as a developer new to TerminusDB is not really reasonable on my part (which is fine if the impedance mismatch is too great and thus the learning curve is necessary), I’d like to see a significant focus in the documentation which starts with the assumption that I am a developer wanting to use TerminusDB in my application, and that I am not a data scientist or someone who has spent the last few years dabbling in semantic web concepts. The documentation would demonstrate that it understands the kinds of problems I’m starting with, and then help me remap my mental model by showing me how to transition to doing it the linked-data way - without making me feel like I’m being crushed under the verbosity often associated with linked data.

Kevin’s four-part series on graph databases were excellent. I felt like there was no continuity from there to working with TerminusDB though, and all the documentation seems to want me to use TerminusDB to make some data and then share it with my friends.

Other thoughts:

  • There’s almost nothing in the documentation about working with JSON(-LD) at all.
  • Having to manually add and remove individual triples everywhere feels very “heavy”. Would be good to see the documentation reassure me that this is not an issue. Having to break up my data models into individual triples in the first place also feels like it could get tiresome. Is there no better way to have my cake and eat it too, so to speak?
  • Are there any (forthcoming?) sample applications I could look at that do normal things (persistent todo list, whatever) with the Node.js client and a simple user interface? If so, I’d encourage minimising the use of React, Angular or whatever, with a preference for vanilla JavaScript. Anyone familiar with a particular framework can still get the concepts if the source is just plain old JavaScript. The reverse is not as true.

Just some food for thought.

Edit

Just as a quick follow-up, some of my reactions to the documentation may already be addressed in places I didn’t notice or wasn’t aware of for some other reason. So I guess as a final bit of feedback, it’d be nice if all of the documentation was organised into something that had a proper navigation and search feature, complete with subsections, links between sections and references, etc. Then, even if how to do something is not immediately obvious, it would be easy to quickly sift through the docs to identify pages and subsections that may have what I need. Documentation is such an important thing for developer adoption - I really believe it’d be a worthwhile time investment to help get a bunch of new users on board.

4 Likes

Thanks @axefrog, this is really helpful. I think you are correct that we wanted the documentation to highlight the collaboration features, but we also want to make life as easy as possible for devs building data intense applications.

I am going to open a bunch of issues on the back of your comments as you’ve identified a few places where we need to do better. Working with JSON and blobs are front of my mind.

2 Likes

No worries. I wrote the following on Discord, but am pasting here for reference. If you want to really delight new users and get them invested, consider the following:

Example of trying your features right in the browser in ably.io’s docs: https:/ /www.ably.io/documentation/realtime/channels

Example of the kind of comprehensiveness that developers will absolutely love you for, from Stripe: https:/ /stripe.com/docs/billing/subscriptions/fixed-price

And finally, a fantastic example of things explained well. This is the kind of thing I was referring to regarding leading the developer from their starting point to where you want them to be: https:/ /gist.github.com/staltz/868e7e9bc2a7b8c1f754

Note: I had to deform the links because your spam settings won’t let me post more links as a new user.

2 Likes

I’d like to second @axefrog point. I went through the tutorials and documentation and don’t understand how to store things in the DB using the javascript client libraries and I think that would be very helpful for people building applications on top of TerminusDB

I’ll poke around and figure it out, but a step by step guide would be very helpful. Imagine someone coming from SQL or MongoDB and they just wanna toss a row into a table or a bit of text and a timestamp into a document store. How can you model that as triples? What if I just want to add a node with a property? A name with a timestamp? Again I’ll do more reading, perhaps I missed it, but given @axefrog’s feedback I think this might just be a gap in the docs

Creating an example Todo list application would be helpful, I’m happy to help make some content if someone can chat with me about how to make it

1 Like