Hey just some feedback on the docs from the perspective of a new user, and with full acknowledgement that the docs are very much WIP.
Most of the docs seem focused on teaching the user all about how to use it from the context of treating it like a shareable Git repository. Pushing, pulling, rebasing, cloning, and all of that kind of stuff.
None of that stuff is of much use to me as a developer until I am feeling confident in how I integrate it into my own systems. There’s also a lot of documentation about how to query and consume graph data once it’s in the system, but less exploration of getting the data in there from the perspective of a typical client application.
The way the documentation emphasises all of the Git-like functionality was leaving me feeling like maybe the database is currently more targeted at people who want to build, share and query data sets than those who want solid integrations into high-performance client applications. I had to spend some time letting things turn over in my head and challenging that reaction I found myself having.
What I, as a developer new to TerminusDB, would have really benefited from, would be stuff like:
- How can I quickly get a Schema in place without getting bogged down in the intricacies of WOQL and so forth? Can I just start by referring to a fake Schema/prefix and backfill the details later? Sometimes it’s nice to just start fleshing something out and then fill in the details as things take shape.
- How do I deal with blobs of data? I’ve realised that I should probably be storing blocks of content like text documents, scripts and so forth externally to TerminusDB and then linking to it from my database. I had to figure this out for myself though - it very much wasn’t clear from any of the documentation.
- Further to the above, let’s say I’m dumping text documents into a local database (LevelDB in my case) so that I can link them together using linked data. Do I need to create a prefix to something local? Do I have to worry about TerminusDB trying to resolve it on my behalf?
Under tutorials, you have:
- My First Cloned Database
- My First Shared Database
- Pushing and Pulling Changes
- Creating and Merging Branches
- Taking TerminusDB to the Bank
- Importing and Cleaning Data From CSVs
Most of those don’t really help me get started using TerminusDB as a database for my application. The second-last item seems useful, but it primarily talks in terms of creating and deleting triples (whereas my application uses records, arrays, etc.), then hurries me along to sharing, branching and all of that. I would have liked to see it talk more from the perspective of “Hey you’re a developer with a bunch of records you’d like to store and link together. Here’s how to do that without having to mentally break it all into triples in advance, or at least here’s how you can do it with a minimum of ceremony”. I honestly had no interest whatsoever at this stage in sharing and publishing data so others can use it. All of that stuff can wait until much later.
The “Reference” section is obviously more comprehensive, but it’s really not clear where basic stuff is, like, “how do I put some JSON data into the database?”. The “JS Client” section has something about “update_triples”, but I’m not yet working with triples directly in memory in my application. I don’t see anything there about adding triples. I found some functions to that effect in the “Python Client” section (assuming that they probably mirror the JS client to some extent?). I then realised the the QL in WOQL is as in SQL, in that it’s not just about performing queries, it’s also about making changes. So then I found myself hunting around and feeling a bit deflated, seeing as all I wanted to do was get some records into the database and to then start building on that.
Even if what I want as a developer new to TerminusDB is not really reasonable on my part (which is fine if the impedance mismatch is too great and thus the learning curve is necessary), I’d like to see a significant focus in the documentation which starts with the assumption that I am a developer wanting to use TerminusDB in my application, and that I am not a data scientist or someone who has spent the last few years dabbling in semantic web concepts. The documentation would demonstrate that it understands the kinds of problems I’m starting with, and then help me remap my mental model by showing me how to transition to doing it the linked-data way - without making me feel like I’m being crushed under the verbosity often associated with linked data.
Kevin’s four-part series on graph databases were excellent. I felt like there was no continuity from there to working with TerminusDB though, and all the documentation seems to want me to use TerminusDB to make some data and then share it with my friends.
- There’s almost nothing in the documentation about working with JSON(-LD) at all.
- Having to manually add and remove individual triples everywhere feels very “heavy”. Would be good to see the documentation reassure me that this is not an issue. Having to break up my data models into individual triples in the first place also feels like it could get tiresome. Is there no better way to have my cake and eat it too, so to speak?
Just some food for thought.
Just as a quick follow-up, some of my reactions to the documentation may already be addressed in places I didn’t notice or wasn’t aware of for some other reason. So I guess as a final bit of feedback, it’d be nice if all of the documentation was organised into something that had a proper navigation and search feature, complete with subsections, links between sections and references, etc. Then, even if how to do something is not immediately obvious, it would be easy to quickly sift through the docs to identify pages and subsections that may have what I need. Documentation is such an important thing for developer adoption - I really believe it’d be a worthwhile time investment to help get a bunch of new users on board.