This space is reserved for sponsors that support us to keep the newsletter going! Want to support Console? Send us a note at osh@codesee.io
Browse through open source projects on OpenSourceHub.io, add your project to get more exposure and connect with other maintainers and contributors!
All-in-one OSINT tool for analysing any website
language: TypeScript stars: 3308 last commit: yesterday
repo: github.com/Lissy93/web-check
site: web-check.xyz
Puppet is an automated administrative engine for your Linux, Unix, and Windows systems, performs administrative tasks (such as adding users, installing packages, and updating server configurations) based on a centralized specification.
language: Ruby stars: 7042 last commit: last week
repo: github.com/puppetlabs/puppet
site: puppet.com/open-source/#osp
Trilium Notes is a hierarchical note taking application with focus on building large personal knowledge bases.
language: JS stars: 22,233 last commit: 4 days
repo: github.com/zadam/trilium
Join thousands of other open-source enthusiasts and developers in the Open Source Hub Discord server to continue the discussion on the projects in this week's email!
Hey Zadam! Thanks for joining us! Let us start with your background.
I’m from Central Europe. I started programming when I was 14 years old, trying to build the next big OS in x86 assembler. That proved to be too hard, so I wrote a simple EXE/COM virus instead, infecting my whole computer (but nothing else, I had no internet back then).
Nowadays, I’m a fan of static/rich type systems, immutability and functional programming. From the mainstream languages, Kotlin is my favorite. From the non-mainstream languages, I’m excited about Idris and its dependent types and hope some of that will be eventually trickle down into mainstream. In the process of developing Trilium Notes, modern JavaScript grew onto me. It’s a quite ergonomic scripting language, although its lack of static types means it doesn’t scale well beyond a few people.
What’s your most controversial programming opinion?
Not exactly programming, but I strongly believe that microservice architecture is a big overkill for a great majority of projects and its wide adoption is basically a cargo cult and resume-driven-development. Putting network into your business logic interactions has significant costs (reliability, performance, lack of transactions, refactoring difficulty, “weak typing”), but you often don’t get anything in return (there are better ways to achieve modularization).
What is your favorite software tool?
IntelliJ IDEA. The two areas I love the most are the automatic static analysis and the advanced refactoring features. I spend a lot of my time at work and my free time with IntelliJ and I feel lucky that I get to use a tool I really like.
If I gave you $10 million to invest in one thing right now, where would you put it?
$10 million is both a lot and not a lot of money, so I’d look for places where you get the most bang for the buck.
In my mind, projects like GIMP, Kdenlive or Audacity are underfunded, but “simple enough” that a couple of full time dedicated contributors could make a big difference. These apps are critical for the public perception of Linux (BSDs …) as a complete en user platform, but they lack the necessary corporate backing.
Why was Trilium Notes started?
Back in 2017 there wasn’t a lot of choice in note-taking software. I used to use proprietary software called Notecase Pro. I like its free-form tree based structure, WYSIWYG editor and some other features, but it had only rudimentary sync capabilities, so I decided to build a web app which I could use at work and all my notes would sync with Notecase Pro at home.
But relatively early on I found out that keeping the compatibility with this app was just too difficult for the benefit it brought (there was a lot of “impedance mismatch”, e.g. simple formatting like italics was done using some custom binary markup and translation from/to HTML was painful), so I decided it will be easier to make a separate app for my needs instead. I initially allocated 3 months to develop it, instead it’s been 6 years now …
How does Trilium Notes work?
Hardly exciting stuff. It’s a client-server app. Frontend JavaScript, backend JavaScript (node.js), SQLite used for persistence. Natively it’s a web app which can be deployed on a Linux server, for desktop there’s Electron wrapper. Individual instances (e.g. desktop, server) can sync between each other.
People sometimes criticize that Trilium uses Electron, and that they would prefer a native application instead. But this is a false dichotomy - if Electron did not exist, there would simply be no desktop Trilium at all.
Where did the name for Trilium Notes come from?
Naming things is hard, especially if the name is there to stay, and you can’t easily refactor later. When I started the project, I lived in Ontario, Canada. Trillium grandiflorum is Ontario’s “official flower” (sounds kind of absurd, I know), and as a consequence appears in the names of many institutions and companies. So I kept seeing it all the time just by walking around. I liked the sound of it, the natural theme, and it kinda alludes to “tree” which is the central data structure in Trilium.
Who, or what was the biggest inspiration for Trilium Notes?
As mentioned above, the project originally started as a web app for a commercial application called Notecase Pro. I remember what impressed me back then was how easy it is to create and manipulate the note tree structure - e.g. moving the note in the hierarchy upwards or downwards is just one keyboard shortcut. Unlike many other note-taking tools, it didn’t force you into a particular structure and left it up to you.
Early in the development, I also read “The Tinderbox Way” describing some of the philosophy of knowledge base building and the thinking behind some of the features of the eponymous application. That provided inspiration for some of the more advanced features/concepts in Trilium (e.g. promoted attributes).
These days, inspiration mostly comes from the user feedback/ideas.
Are there any overarching goals of Trilium Notes that drive design or implementation? If so, what trade-offs have been made in Trilium Notes as a consequence of these goals?
One goal which comes to my mind is the long term sustainability as a hobby project. A surprising share of even quite prominent open source projects are largely driven by a single person, and that applies to Trilium as well. So I try to protect myself against burnout, by making sure I do a lot of the fun stuff and limit the boring (or outright frustrating) work to a minimum.
As an example, Trilium is English-only application which might be surprising at first. Adding localization is not that hard, what it adds is the maintenance overhead down the line, managing the translations/strings, making sure all the dialogs are sized flexibly enough to support all kinds of languages etc. You might call me a discriminating a**hole to basically exclude non-English speakers, but for me, it’s one of the anti-burnout preventions focused on keeping the development of Trilium viable.
A different trade-off I can mention is balancing between desktop and mobile. Having a good mobile support is these days considered a necessity, but that drives some compromises, often dumbing down the functionality/interface so that it works for both mobile and desktop.
I’m making a different trade-off, for me the Trilium Notes is a desktop first application, with mobile app (actually just mobile website) being just “best effort” with some of the more complicated functionality completely inaccessible. Personally, I don’t actually use Trilium on a smartphone at all - if I need to note something, I will use an app like Google Keep and then put it to Trilium once on a computer (with proper categorizing etc.).
What is the most challenging problem that’s been solved in Trilium Notes, so far?
It’s a note-taking app, so it’s no rocket science :-D There are no fancy algorithms or anything like that.
Perhaps one challenging engineering problem is how to make things fast - some of the Trilium’s advanced features (such as attribute inheritance, logic/presentation driven by user-set attributes) mean that the application has to consult a non-trivial amount of data for some operations. To do that fast, you need to minimize the amount of IO/database interactions.
It took several iterations over the years, but in the end it converged to a design where most business logic is executed completely in memory without database access. The backend process has a complete data model of the notes’ metadata (tree structure, labels, relations), while frontend has a lazy loaded subset of a similar data model - notes which you can see in the UI, all the referenced notes, all the ancestors of displayed notes (e.g. to correctly evaluate the attribute inheritance).
One of the challenges there is then to keep all these distinct data models (frontend, backend, database + synced data from remote servers) in sync.
Are there any projects similar to Trilium Notes? If so, what were they lacking that made you consider building something new?
Back in 2017 there wasn’t a lot of choice in this area, I believe since then this area experienced a significant boom. It’s quite possible that if there was such a selection back in 2017, I would become a happy user (and perhaps contributor) of some existing application instead of starting my own project. But that ship has sailed.
The reason why there are some many applications seemingly doing the same is that each application and their authors have a specific vision. This then forces some specific trade-offs which then make some other ideas (which are not part of the original vision) difficult to execute. For example, many apps pride themselves to use only a file system (as opposed to a database), but that significantly limits the capabilities of the application (or makes it very difficult to implement). An example - one basic idea I got from Notecase Pro is that for the purpose of note-taking, having a dichotomy of folder and file (folder can’t have text content of its own, files can’t have children) is breaking the “content refactoring” where you want to e.g. extract longer paragraphs into a separate sub-note. But being based on a simple file system forces you to do exactly that (you can work around that, but it will necessarily be ugly and not true to the file system spirit anymore).
What was the most surprising thing you learned while working on Trilium Notes?
Perhaps that most users are actually pretty nice, polite and thankful. You can often read horror stories about entitled angry users, but I’m glad to report I had a mostly positive experience. I can remember only two users who I had to block (on GitHub).
What is your typical approach to debugging issues filed in the Trilium Notes repo?
The most important thing is to be able to reproduce the defect. That is sometimes challenging in such a fundamentally data-driven application, where even behavior is often determined by user data (attributes). Providing a good description is therefore crucial, and the file issues differ greatly, from very vague descriptions without context, to a super detailed analysis, sometimes with a pull request attached.
What is the release process like for Trilium Notes?
It’s very important for me that the actual act of releasing the software is simple and automated. For the most part it’s executing one script with a single parameter (version) which builds all the desktop binaries locally on my Linux system and uploads them to GitHub. Docker images are then built with GitHub Actions, Flatpak on FlatHub.
This limits the number of platforms I’m able to release binaries for - e.g. the build for Apple M1 can’t be automated from Linux.
Is Trilium Notes intended to eventually be monetized if it isn’t monetized already? If so, how? If it’s already monetized, what is your main source of revenue?
No, there’s never been a plan to monetize Trilium. I think it’s very difficult to make a living in this segment of personal note-taking / knowledge base tools and basically impossible to “make it big”.
The only path to that is targeting companies by building in multi-user support, various authentication methods, fine-grained ACL and other enterprise features, but I believe that’s fundamentally a different product which forces some compromises for personal note-taking.
I get donations on GitHub and PayPal which provide “beer money”. It’s also very nice to see that users are seeing value in my work and are willing to support it in real terms.
How do you balance your work on open-source with your day job and other responsibilities?
The day job is actually pretty simple. I work the standard 40 hours a week, so there’s a very clear divide between what is work time and the rest.
I also have clear time allocated for family and personal relationships, and they take precedence.
The part which is challenging is balancing the work on Trilium with other intellectual pursuits like reading. On one side I feel certain responsibility towards users and try to do as much bug fixing / feature implementation as possible. I try to reserve some time for e.g. reading, some days I don’t open the GitHub / IDE at all etc.
Have you ever experienced burnout? How did you deal with it?
Avoiding burnout is the big part of my thinking / strategy outlined above in talking about the long term project sustainability. Basically I’m trying to work on fun things and avoid the boring things (unless really necessary), even at the cost of a wider user adoption. Another aspect I think is important is scope management - I try to always have an achievable scope of the project in mind, so that it doesn’t feel like I’ve set an impossible goal for myself.
The great advantage of building an open source software is that can just do your own thing at your own pace, people using it is “just” a nice bonus. You usually don’t have the same freedom with commercial products where your existence depends on the product success/adoption.
What is the best way for a new developer to contribute to Trilium Notes?
I guess the best way is to use the app, find some bug or identify a feature which is missing, but fits into the project mission.
People also sometimes contribute themes and scripts / extensions. One user created an Awesome Trilium collection of such contributions.
If you plan to continue developing Trilium Notes, where do you see the project heading next?
I don’t expect Trilium to undergo huge changes anymore, conceptually it’s more or less finished. But there’s an unending tail of smaller features and refinements waiting to happen.
To give something specific - in the next version I’d like to implement an in-app offline documentation, I also want to spend time on providing better out-of-the-box data visualizations (basic statistics, but also refinement of “note map” feature).
What motivates you to continue contributing to Trilium Notes?
That’s a surprisingly difficult question. I started the project to scratch my own itch, but the app has long already fulfilled my personal needs. I guess now it’s a combination of being a fun thing to do, feeling some kind of responsibility towards the existing users and a general perfectionism. I heard that OSS projects thrive on shame of their authors and I guess there’s some truth in it.
I came across this interesting photo in your readme. Can you tell us the story?
Not as exciting as it might seem. There’s a website run by Ukrainian volunteers which “sells shells” - you kinda donate money to buy an artillery shell for the Ukrainian army and as a “vanity reward” they will write text of your choosing on it (as I understand, not long before firing it). You can choose different kind of shells for different prices etc., it’s kind of surreal.
I admit that putting it on GitHub is provocative and essentially off-topic. But I think it’s fair that I as an author can put a personal message about a cause I care about. My message is basically that supporting Ukraine and its sovereignty necessarily means providing weapons (there are many people who vocally support Ukraine, but reject providing weapons, which is in my view an inconsistent position). And yes, weapons kill people, but that’s just what you have to do to defend yourself from invaders.
Where do you see software development heading next?
It’s becoming quite obvious that AI will change the field, it’s just not yet clear how exactly. Very simple jobs (like simple websites) will likely be “outsourced” to AIs to a considerable degree. We’ll have domain-trained and project-trained AI coding assistants, static analysis and quality gates. And that’s just with the current state of the art (industry will need a few years to catch up). I used to be an AI skeptic for a long time, but even these relatively low tech LLMs are proving to be powerful enough for a lot of use cases.
Having said that, if there’s some breakthrough towards strong general AI, all bets are off.
Where do you see open-source heading next?
One long term trend I worry about is that open source is getting more dominated by corporate contributions. On one hand it’s good that many corporations see contributing open source code as a “new normal”, on the other hand it seems that the complexity of many projects / areas are getting out of reach for the community (which isn’t bound by commercial interests).
Community open source works best with projects which can be handled by a single person or a small group of people. Getting community OSS work on a bigger scale is quite difficult. KDE would be a notable exception since it’s quite big, quite successful, yet still largely community run.
Some corporate driven projects are pretty well run, though, notable examples would be Linux kernel or Postgres. On the opposite side of the spectrum we have e.g. blink where you have one company dominating the project (even though it’s open source) and having a disproportionate influence on the web platform evolution.
Do you have any suggestions for someone trying to make their first contribution to an open-source project?
The best motivation is scratching your own itch. If contributing to an existing project, try to start with something small. If you want to contribute a bigger thing, check with the maintainers first. The reality is that projects won’t accept any feature, even if it’s well engineered.
If you’re starting a new project, don’t try to make it perfect. Embrace the iterative approach, the first versions may be really crappy and that’s just fine. I think some people may have a sort of stage fright with their code being open, visible to everyone and thus open to critique. Don’t worry about it too much.
With the benefit of the hindsight, what would you do differently?
I started the project with the “move fast and break things” mentality. I iterated very quickly, not caring about bugs or code quality. Back then, I was the main user and the project was only semi-public, so it didn’t matter that much.
With time the code quality and bug rate improved significantly, which was important given that I had some users already. But one thing which was never really improved is the automated test code coverage. I never went back to write tests for the existing functionality, and only few parts of the application have tests.
I still think that it’s fine to not write tests in basically POC stage of the project. But then it’s important to catch the right moment where it starts being important.
Want to join the conversation about one of the projects featured this week? Drop a comment, or see what others are saying!
Leave a comment
Interested in sponsoring the newsletter or know of any cool projects or interesting developers you want us to interview? Reach out at osh@codesee.io or mention us @ConsoleWeekly!
